You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have followed the installation instruction in the main readme file, followed by the instructions to run the segment_anything_fast example. I am encountering an odd error.
Error logs
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1765) ~[?:?]
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:435) ~[?:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:229) ~[model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1570) [?:?]
2024-06-11T13:46:54,390 [WARN ] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: sam-fast, error: Worker died.
2024-06-11T13:46:54,390 [DEBUG] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-sam-fast_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2024-06-11T13:46:54,390 [WARN ] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2024-06-11T13:46:54,390 [INFO ] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9006 in 55 seconds.
2024-06-11T13:46:54,390 [INFO ] W-9006-sam-fast_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9006-sam-fast_1.0-stdout
2024-06-11T13:46:54,390 [INFO ] W-9006-sam-fast_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9006-sam-fast_1.0-stderr
2024-06-11T13:46:54,396 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Listening on addr:port: 127.0.0.1:9009
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Successfully loaded /opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/configs/metrics.yaml.
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - [PID]86770
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Torch worker started.
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Python runtime: 3.10.14
2024-06-11T13:46:54,401 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-sam-fast_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2024-06-11T13:46:54,402 [INFO ] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /127.0.0.1:9009
2024-06-11T13:46:54,402 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD repeats 1 to backend at: 1718138814402
2024-06-11T13:46:54,402 [INFO ] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Looping backend response at: 1718138814402
2024-06-11T13:46:54,402 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Connection accepted: ('127.0.0.1', 9009).
2024-06-11T13:46:54,403 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - model_name: sam-fast, batchSize: 1
2024-06-11T13:46:54,463 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Backend worker process died.
2024-06-11T13:46:54,463 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 108, in load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module, function_name = self._load_handler_file(handler)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 153, in _load_handler_file
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/importlib/init.py", line 126, in import_module
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1050, in _gcd_import
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1027, in _find_and_load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1006, in _find_and_load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 688, in _load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 883, in exec_module
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/Users/yousof/Desktop/serve/examples/large_models/segment_anything_fast/model_store/sam-fast/custom_handler.py", line 11, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from segment_anything_fast import SamAutomaticMaskGenerator, sam_model_fast_registry
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/init.py", line 7, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .build_sam import (
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/build_sam.py", line 11, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .modeling import ImageEncoderViT, MaskDecoder, PromptEncoder, Sam, TwoWayTransformer
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/modeling/init.py", line 7, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .sam import Sam
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/modeling/sam.py", line 13, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .image_encoder import ImageEncoderViT
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/modeling/image_encoder.py", line 15, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from segment_anything_fast.flash_4 import _attention_rel_h_rel_w
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/flash_4.py", line 23, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - import triton
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'triton'
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG -
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - During handling of the above exception, another exception occurred:
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG -
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 263, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - worker.run_server()
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 231, in run_server
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 194, in handle_connection
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 131, in load_model
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - service = model_loader.load(
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 110, in load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module = self._load_default_handler(handler)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 159, in _load_default_handler
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name, "ts.torch_handler")
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/importlib/init.py", line 126, in import_module
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-11T13:46:54,464 [INFO ] nioEventLoopGroup-5-28 org.pytorch.serve.wlm.WorkerThread - 9009 Worker disconnected. WORKER_STARTED
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1050, in _gcd_import
2024-06-11T13:46:54,464 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1027, in _find_and_load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 992, in _find_and_load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1050, in _gcd_import
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1027, in _find_and_load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1004, in _find_and_load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'ts.torch_handler.custom_handler'
2024-06-11T13:46:54,464 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died., responseTimeout:300sec
java.lang.InterruptedException: null
Installation instructions
Ihave also tried to follow the docker instructions and am encountering the same error. I am on a MacBook, so no access to a GPU, but as far as I can tell, that shouldn't be an issue as
./build_image.sh
installs a CPU version by default.
Model Packaging
the model was packaged according to the instructions in there sam_fast README.MD file.
config.properties
config.properties is unchanged from the original cloned file.
Versions
I am running python 3.10 and the followng torch packages
Hi @yousofaly, sorry, but the example will not run on a MacBook as the segment_anything_fast fork is specifically optimized for running on GPU (specifically A100s).
In you case it fails because the triton package is missing which is used to create a custom CUDA kernel to replace an auto generated on in torch.compile:
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - import triton
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'triton'
You can try to run the original segment_anything version in the handler but this might need some modifications to the original example.
🐛 Describe the bug
I have followed the installation instruction in the main readme file, followed by the instructions to run the segment_anything_fast example. I am encountering an odd error.
Error logs
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1765) ~[?:?]
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:435) ~[?:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:229) ~[model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1570) [?:?]
2024-06-11T13:46:54,390 [WARN ] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: sam-fast, error: Worker died.
2024-06-11T13:46:54,390 [DEBUG] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-sam-fast_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2024-06-11T13:46:54,390 [WARN ] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2024-06-11T13:46:54,390 [INFO ] W-9006-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9006 in 55 seconds.
2024-06-11T13:46:54,390 [INFO ] W-9006-sam-fast_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9006-sam-fast_1.0-stdout
2024-06-11T13:46:54,390 [INFO ] W-9006-sam-fast_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9006-sam-fast_1.0-stderr
2024-06-11T13:46:54,396 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Listening on addr:port: 127.0.0.1:9009
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Successfully loaded /opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/configs/metrics.yaml.
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - [PID]86770
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Torch worker started.
2024-06-11T13:46:54,401 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Python runtime: 3.10.14
2024-06-11T13:46:54,401 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-sam-fast_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2024-06-11T13:46:54,402 [INFO ] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /127.0.0.1:9009
2024-06-11T13:46:54,402 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD repeats 1 to backend at: 1718138814402
2024-06-11T13:46:54,402 [INFO ] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Looping backend response at: 1718138814402
2024-06-11T13:46:54,402 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Connection accepted: ('127.0.0.1', 9009).
2024-06-11T13:46:54,403 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - model_name: sam-fast, batchSize: 1
2024-06-11T13:46:54,463 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Backend worker process died.
2024-06-11T13:46:54,463 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 108, in load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module, function_name = self._load_handler_file(handler)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 153, in _load_handler_file
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/importlib/init.py", line 126, in import_module
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1050, in _gcd_import
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1027, in _find_and_load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1006, in _find_and_load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 688, in _load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 883, in exec_module
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/Users/yousof/Desktop/serve/examples/large_models/segment_anything_fast/model_store/sam-fast/custom_handler.py", line 11, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from segment_anything_fast import SamAutomaticMaskGenerator, sam_model_fast_registry
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/init.py", line 7, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .build_sam import (
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/build_sam.py", line 11, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .modeling import ImageEncoderViT, MaskDecoder, PromptEncoder, Sam, TwoWayTransformer
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/modeling/init.py", line 7, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .sam import Sam
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/modeling/sam.py", line 13, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from .image_encoder import ImageEncoderViT
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/modeling/image_encoder.py", line 15, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - from segment_anything_fast.flash_4 import _attention_rel_h_rel_w
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/segment_anything_fast/flash_4.py", line 23, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - import triton
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'triton'
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG -
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - During handling of the above exception, another exception occurred:
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG -
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 263, in
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - worker.run_server()
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 231, in run_server
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 194, in handle_connection
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_service_worker.py", line 131, in load_model
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - service = model_loader.load(
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 110, in load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module = self._load_default_handler(handler)
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/site-packages/ts/model_loader.py", line 159, in _load_default_handler
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name, "ts.torch_handler")
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "/opt/anaconda3/envs/trchsrv/lib/python3.10/importlib/init.py", line 126, in import_module
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-11T13:46:54,464 [INFO ] nioEventLoopGroup-5-28 org.pytorch.serve.wlm.WorkerThread - 9009 Worker disconnected. WORKER_STARTED
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1050, in _gcd_import
2024-06-11T13:46:54,464 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1027, in _find_and_load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 992, in _find_and_load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1050, in _gcd_import
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1027, in _find_and_load
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - File "", line 1004, in _find_and_load_unlocked
2024-06-11T13:46:54,464 [INFO ] W-9009-sam-fast_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'ts.torch_handler.custom_handler'
2024-06-11T13:46:54,464 [DEBUG] W-9009-sam-fast_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died., responseTimeout:300sec
java.lang.InterruptedException: null
Installation instructions
Ihave also tried to follow the docker instructions and am encountering the same error. I am on a MacBook, so no access to a GPU, but as far as I can tell, that shouldn't be an issue as
./build_image.sh
installs a CPU version by default.
Model Packaging
the model was packaged according to the instructions in there sam_fast README.MD file.
config.properties
config.properties is unchanged from the original cloned file.
Versions
I am running python 3.10 and the followng torch packages
pytorch-labs-segment-anything-fast @ git+https://github.com/pytorch-labs/segment-anything-fast.git@3e9c47d2ef18ddf4f179128e8c0f677dd5e989b8
torch==2.2.2
torch-model-archiver @ file:///usr/share/miniconda/envs/__setup_conda/conda-bld/torch-model-archiver_1715885178714/work
torch-workflow-archiver @ file:///usr/share/miniconda/envs/__setup_conda/conda-bld/torch-workflow-archiver_1715885227278/work
torchao==0.1
torchserve @ file:///usr/share/miniconda/envs/__setup_conda/conda-bld/torchserve_1715885095944/work
torchvision==0.17.2
Repro instructions
Install dependencies
cuda is optional
python ./ts_scripts/install_dependencies.py --cuda=cu121
Latest release
pip install torchserve torch-model-archiver torch-workflow-archiver
Nightly build
pip install torchserve-nightly torch-model-archiver-nightly torch-workflow-archiver-nightly
cd to the example folder examples/large_models/segment_anything_fast
cd ../examples/large_models/segment_anything_fast
install segment_anything_fast
download weights
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
If you are not using A100 for inference, turn off the A100 specific optimization using
export SEGMENT_ANYTHING_FAST_USE_FLASH_4=0
generate model archive
start and run
Possible Solution
No response
The text was updated successfully, but these errors were encountered: