[BUG] nexa run bark-small:fp16 fails with ValueError: Failed to load model from file: /Users/user/.cache/nexa/hub/official/bark-small/fp16.gguf

[hammer-ai]

Issue Description

I am trying to run TTS, but getting ValueError: Failed to load model from file: /Users/user/.cache/nexa/hub/official/bark-small/fp16.gguf when I run.

Steps to Reproduce

~/repo nexa run bark-small:fp16


Downloading bark-small/fp16.gguf...
fp16.gguf: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 804M/804M [00:19<00:00, 43.8MB/s]
Successfully downloaded bark-small/fp16.gguf to /Users/user/.cache/nexa/hub/official/bark-small/fp16.gguf
Successfully pulled model bark-small:fp16 to /Users/user/.cache/nexa/hub/official/bark-small/fp16.gguf, run_type: NLP
Error loading GGUF models, please refer to our docs to install nexaai package: https://docs.nexaai.com/getting-started/installation
Traceback (most recent call last):
  File "nexa/gguf/nexa_inference_text.py", line 100, in _load_model
  File "nexa/gguf/llama/llama.py", line 364, in __init__
  File "nexa/gguf/llama/_internals_transformers.py", line 55, in __init__
ValueError: Failed to load model from file: /Users/user/.cache/nexa/hub/official/bark-small/fp16.gguf

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "nexa/cli/entry.py", line 27, in run_ggml_inference
  File "nexa/gguf/nexa_inference_text.py", line 72, in __init__
  File "nexa/utils.py", line 155, in wrapper
  File "nexa/gguf/nexa_inference_text.py", line 110, in _load_model
  File "nexa/gguf/llama/llama.py", line 364, in __init__
  File "nexa/gguf/llama/_internals_transformers.py", line 55, in __init__
ValueError: Failed to load model from file: /Users/user/.cache/nexa/hub/official/bark-small/fp16.gguf

I can see it though when I look at the list:

~/repo nexa list
+-----------------------------------+------+-----------------+------------------------------------------------------------------------------------+
| Model Name                        | Type | Run Type        | Location                                                                           |
+-----------------------------------+------+-----------------+------------------------------------------------------------------------------------+
| FLUX.1-schnell:flux1-schnell-q4_0 | gguf | Computer Vision | /Users/user/.cache/nexa/hub/official/FLUX.1-schnell/flux1-schnell-q4_0.gguf |
| FLUX.1-schnell:t5xxl-q4_0         | gguf | Computer Vision | /Users/user/.cache/nexa/hub/official/FLUX.1-schnell/t5xxl-q4_0.gguf         |
| FLUX.1-schnell:ae-fp16            | gguf | Computer Vision | /Users/user/.cache/nexa/hub/official/FLUX.1-schnell/ae-fp16.gguf            |
| FLUX.1-schnell:clip_l-fp16        | gguf | Computer Vision | /Users/user/.cache/nexa/hub/official/FLUX.1-schnell/clip_l-fp16.gguf        |
| bark-small:fp16                   | gguf | NLP             | /Users/user/.cache/nexa/hub/official/bark-small/fp16.gguf                   |
+-----------------------------------+------+-----------------+------------------------------------------------------------------------------------+

OS

MacOS 14.4.1

Python Version

N/A

Nexa SDK Version

0.0.8.5

GPU (if using one)

Apple M2

[Davidqian123]

update to the latest version v0.0.9.1 of nexa-sdk and it should work:

1. pip uninstall nexaai
2. CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai==0.0.9.1 --prefer-binary --index-url https://github.nexa.ai/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir