Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can we load model on demand as Ollama did #767

Open
6 tasks done
liaoweiguo opened this issue Dec 19, 2024 · 2 comments
Open
6 tasks done

can we load model on demand as Ollama did #767

liaoweiguo opened this issue Dec 19, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@liaoweiguo
Copy link

Self Checks

  • I have thoroughly reviewed the project documentation (installation, training, inference) but couldn't find any relevant information that meets my needs. English 中文 日本語 Portuguese (Brazil)
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell us your story.

deploy a service in office for reading my clipboard, but when I don't use it, I want it to automatically unload, maybe 10 minutes

2. What is your suggested solution?

load model on demand as Ollama

3. Additional context or comments

No response

4. Can you help us with this feature?

  • I am interested in contributing to this feature.
@liaoweiguo liaoweiguo added the enhancement New feature or request label Dec 19, 2024
@liaoweiguo
Copy link
Author

ollama.chat has a parameter keep_alive

@None1024Outlook
Copy link

ollama.chat has a parameter keep_alive

Models should not be supported for use on ollama

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants