Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

haystack/components/generators/hugging_face_local.py "stop_words" not behaving as expected #8672

Open
sephcodes opened this issue Dec 24, 2024 · 0 comments

Comments

@sephcodes
Copy link

Describe the bug
according to the documentation: "param stop_words: If the model generates a stop word, the generation stops."
however, stop_words is just being used to remove whatever is defined as a stopword from the final output. It is not stopping generation when a stop word is encountered.
If possible, a way to stop generation using stop words would be useful

Error message
N/A

Expected behavior
Expecting the stop_words argument to stop generation of output when a value in stop_words list is encountered

Additional context
N/A

To Reproduce
generator = self.generator(model="HuggingFaceTB/SmolLM-1.7B-Instruct",
task="text-generation",
stop_words=self.stop_words,
generation_kwargs={
"max_new_tokens": 150,
# "do_sample": False,
"do_sample": True,
"temperature": 0.5,
# "top_p": 0.9,
# "eos_token_id": self.tokenizer.eos_token_id,
# "stopping_criteria": self.stopping_criteria
})
whatever is defined in self.stop_words will just be dropped from the output but not stop generation of text as expected

FAQ Check
N/A

System:
All

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant