Stop Sequences
A stop sequence is a user-defined string that, when generated, terminates output. Unlike EOS (model-learned), stop sequences are provided by the application to enforce structural boundaries, such as ending a JSON block or a chat turn.
What is Stop Sequences?
A stop sequence is a user-defined string that, when generated, terminates output. Unlike EOS (model-learned), stop sequences are provided by the application to enforce structural boundaries, such as ending a JSON block or a chat turn.
A stop sequence is a user-defined string that, when generated, terminates output. Unlike EOS (model-learned), stop sequences are provided by the application to enforce structural boundaries, such as ending a JSON block or a chat turn.
Where is it used?
OpenAI's `stop` parameter, Anthropic's `stop_sequences`, and HuggingFace TGI all support this. A coding assistant might use `stop=['\\n\\n```']` to stop after a code block. Chat templates use `stop=['<|im_end|>']` to end turns.
How to build it
Decode the running output incrementally: `text = tokenizer.decode(ids)`. Check `if any(s in text for s in stop_sequences): break` and truncate the output before the matched sequence before returning.