Repetition Penalty
Repetition penalty reduces the probability of tokens that have already appeared in the generated text. It scales down the logits of previously seen tokens, discouraging the model from repeating the same phrases or words.
What is Repetition Penalty?
Repetition penalty reduces the probability of tokens that have already appeared in the generated text. It scales down the logits of previously seen tokens, discouraging the model from repeating the same phrases or words.
Repetition penalty reduces the probability of tokens that have already appeared in the generated text. It scales down the logits of previously seen tokens, discouraging the model from repeating the same phrases or words.
Where is it used?
Used in HuggingFace `generate()` (`repetition_penalty=1.1`), CTRL paper, and LLaMA inference. Values of 1.1-1.3 are typical; higher values aggressively suppress repetition but can harm coherence. OpenAI API does not expose this directly.
How to build it
For each token already in the sequence, divide its logit by the penalty if positive, multiply if negative: `score = logit / penalty if logit > 0 else logit * penalty`. Apply before softmax in the generation loop.