Skip to content

Commit

Permalink
Merge pull request #42 from dusty-nv/20231106-llamaspeak
Browse files Browse the repository at this point in the history
added llamaspeak page
  • Loading branch information
dusty-nv authored Nov 7, 2023
2 parents 988bb70 + 11604bb commit 296fbf1
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 2 deletions.
15 changes: 15 additions & 0 deletions docs/tutorial_llamaspeak.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Tutorial - llamaspeak

Talk live with Llama using Riva ASR/TTS, and chat about images with Llava!

<a href="https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/llamaspeak" target=”_blank”><img src="https://raw.githubusercontent.com/dusty-nv/jetson-containers/docs/docs/images/llamaspeak_voice_clip.gif"></a>

* [`llamaspeak:v1`](https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/llamaspeak) - uses text-generation-webui loaders for LLM models (llama.cpp, exllama, AutoGPTQ, Transformers)
* [`llamaspeak:v2`](https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/local_llm) - uses AWQ/MLC from [`local_llm`](https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/local_llm) package, web chat voice agent

llamaspeak v2 has multimodal support for chatting about images with quantized Llava-1.5:

<a href="https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/local_llm" target=”_blank”><img src="https://raw.githubusercontent.com/dusty-nv/jetson-containers/docs/docs/images/llamaspeak_llava_clip.gif"></a>
> [Multimodal Voice Chat with LLaVA-1.5 13B on NVIDIA Jetson AGX Orin](https://www.youtube.com/watch?v=9ObzbbBTbcc) (container: [`local_llm`](https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/local_llm))
See the [`Voice Chat`](https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/local_llm#voice-chat) section of the [`local_llm`](https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/local_llm) documentation to run llamaspeak v2.
3 changes: 1 addition & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ nav:
- About NVIDIA Jetson: tutorial-intro.md
- Text (LLM):
- text-generation-webui: tutorial_text-generation.md
# - LLM + ASR/TTS:
# - LlamaSpeak: tutorial_llamaspeak.md
- llamaspeak: tutorial_llamaspeak.md
- Text + Vision (VLM):
- Mini-GPT4: tutorial_minigpt4.md
- LLaVA: tutorial_llava.md
Expand Down

0 comments on commit 296fbf1

Please sign in to comment.