Stop Conditions
A stop condition is any criterion that terminates the generation loop. Common conditions include emitting the EOS token, reaching the maximum token limit, or encountering a user-specified stop sequence.
What is Stop Conditions?
A stop condition is any criterion that terminates the generation loop. Common conditions include emitting the EOS token, reaching the maximum token limit, or encountering a user-specified stop sequence.
A stop condition is any criterion that terminates the generation loop. Common conditions include emitting the EOS token, reaching the maximum token limit, or encountering a user-specified stop sequence.
Where is it used?
Production LLM APIs (OpenAI, Anthropic, HuggingFace TGI) implement multiple stop conditions simultaneously. For example, GPT-4 stops on EOS, max_tokens, or a custom stop string like '\\n\\nHuman:'. This ensures controlled, bounded output.
How to build it
Combine checks: `if next == eos_id or len(ids) >= max_tokens or stop_str in decoded: break`. Decode incrementally and substring-match against a list of stop strings. Return output up to (but excluding) the stop sequence.