The Value of Thinking: Learning When to Reason in Hybrid Decision Models
THE QUICK READ
The authors propose zero-or-T thinking, in which a single model either answers from implicit thinking in one forward pass without emitting tokens, or performs full autoregressive reasoning when that cost is worth paying. A hybrid decision model implements this with one frozen backbone and a lightweight self-routing block that estimates the value of thinking and runs the slow mode only when that value exceeds a per-request price. Calibrated Decision Policy Optimisation trains the block from episode-level returns.
KEY POINTS
- The paper frames reasoning control as choosing between zero-token implicit answering and full autoregressive reasoning, rather than shortening or skipping explicit reasoning.
- A single frozen backbone serves both modes, and a lightweight self-routing block compares estimated thinking value against a per-request price λ.
- Calibrated Decision Policy Optimisation is introduced to train the routing block from episode-level returns where per-decision labels are unavailable.
READING LIMITS
This is based only on the title and abstract; experimental setup, baselines and the reported numbers are the authors' claims and were not independently verified.
Source abstract [EN]
Existing adaptive-reasoning methods shorten the explicit reasoning a model writes, or skip it by switching modes; either way the answer is still written out token by token. Rather than deciding how much to think, we explore a different paradigm, zero-or-T thinking: a single model either answers from the implicit thinking inside one forward pass, generating no token, or performs full autoregressive reasoning at a cost of T tokens when that is worth paying. We realise it in a hybrid decision model (HDM), where one frozen backbone serves both modes and a lightweight self-routing block chooses between them. The block estimates the value of thinking, the utility the slow mode is expected to add over the fast one, from what that implicit thinking has already produced, and the slow mode runs only where that value exceeds a price λ set per request. Where no per-decision label exists, as in episodic agentic tasks, we introduce Calibrated Decision Policy Optimisation (CDPO), a reinforcement-learning recipe that trains the block from episode-level returns. We evaluate HDM on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. Specifically, on JevBench HDM-4B scores 0.857 against Jev's 0.851 (+0.7%) at a median 0.179\,s against 0.652 (-72.5%). On the agentic venue MiniWoB HDM-27B cuts token cost by 80.1%.
AI summaries use public titles and abstracts only. Research claims belong to the authors; consult the original paper.