Test-Time Compute
Test-time compute is the extra inference-time computation (more reasoning tokens, search, or verification) spent to improve answer quality, trading latency for accuracy on hard problems.
What is Test-Time Compute?
Test-time compute is the extra inference-time computation (more reasoning tokens, search, or verification) spent to improve answer quality, trading latency for accuracy on hard problems.
Test-time compute is the extra inference-time computation (more reasoning tokens, search, or verification) spent to improve answer quality, trading latency for accuracy on hard problems.
Where is it used?
OpenAI o1 dynamically scales test-time compute by generating more reasoning tokens for harder problems; AlphaCode and ToT also use search at inference to boost accuracy.
How to build it
Increase `max_new_tokens` for the reasoning phase and use a `"<|end_reasoning|>"` delimiter; log tokens generated vs answer accuracy to measure the compute-quality scaling curve.