Greedy Decoding
Greedy decoding selects the single highest-probability token at each step. It is deterministic and fast but tends to produce repetitive, generic text because it never explores lower-probability but more creative options.
What is Greedy Decoding?
Greedy decoding selects the single highest-probability token at each step. It is deterministic and fast but tends to produce repetitive, generic text because it never explores lower-probability but more creative options.
Greedy decoding selects the single highest-probability token at each step. It is deterministic and fast but tends to produce repetitive, generic text because it never explores lower-probability but more creative options.
Where is it used?
Greedy decoding is the default for simple LLM demos and when determinism is required. It is rarely used in production chatbots (GPT-4, Claude) because it yields dull, repetitive output. It can get stuck in loops like 'I am I am I am'.
How to build it
At each step: `next_token = logits.argmax(dim=-1)`. Append to the sequence and repeat: `for _ in range(max_new): next = model(ids).argmax(-1)[-1:]; ids = cat([ids, next])`. No sampling needed.