Autoregressive Generation
Autoregressive generation produces text one token at a time, conditioning each new token on all previously generated tokens. The model's own outputs become part of its input, creating a feedback loop that extends the sequence.
What is Autoregressive Generation?
Autoregressive generation produces text one token at a time, conditioning each new token on all previously generated tokens. The model's own outputs become part of its input, creating a feedback loop that extends the sequence.
Autoregressive generation produces text one token at a time, conditioning each new token on all previously generated tokens. The model's own outputs become part of its input, creating a feedback loop that extends the sequence.
Where is it used?
All decoder-only LLMs (GPT-4, Claude, LLaMA, Mistral) generate text this way. Autoregressive generation is also used in time-series forecasting and music generation. The step-by-step nature limits speed (sequential, not parallelizable).
How to build it
Implement a generation loop: `for _ in range(max_tokens): logits = model(ids); next = sample(logits[-1]); ids = cat(ids, next); break if next == eos`. Use KV-caching to avoid recomputing past attention.