StoryboardAI Engineering

Thinking before answering

Letting the model work a problem through before it writes the reply.

What is thinking before answering?

A model writes one token after another and cannot go back. If the first word of its reply has to be the final answer, it must commit before doing any working out, and on anything with several steps it often commits wrongly.

Giving it room to reason first fixes this. The model writes out its working, then writes the answer based on it. The working is kept separate, so the user sees only the answer.

The cost is time and tokens. Simple questions do not need it. Arithmetic, comparisons and anything with rules to apply usually do.

Thinking first: the model reasons in text before it commits to an answer.Step 1: A question with several steps. Step 2: The model writes its working. Step 3: It answers from the working. Step 4: Only the answer is shown.1A question withseveral steps2The model writesits working3It answers fromthe working4Only the answer isshown
Thinking first: the model reasons in text before it commits to an answer.

Why does thinking before answering matter?

A model writes its answer one token at a time and cannot go back. On a problem with several steps, starting to answer at once means committing to a conclusion before the working is done. Letting it reason first, in text it writes for itself, measurably improves answers to anything that needs weighing, planning or arithmetic.

How does thinking before answering work?

  1. The call is made with thinking switched on, or the prompt asks the model to work through the problem first.
  2. The model writes its working: the facts, the steps, the options it considers.
  3. Having written that, it produces the final answer, with its own working in front of it.
  4. The working is usually hidden from the person. It is still counted, and paid for, as tokens written.

When should you use thinking before answering?

Use it when

  • The question has several steps or conditions to hold at once
  • A wrong answer is costly and a few seconds are acceptable
  • The task involves planning what to do next

Think twice when

  • The question is a simple lookup or a greeting
  • The reply must begin within a fraction of a second, as in a voice call

Where does thinking before answering sit in a real system?

It is a setting on the model call, chosen per request. Products that route requests by difficulty turn it on only for the hard ones, because it multiplies both the wait and the cost of a reply.

What goes wrong with thinking before answering?

  • Turning it on for everything, and paying eight times as much for answers that are no better.
  • Showing the raw working to customers as though it were the answer.
  • Assuming the written working is a faithful account of how the answer was reached. It is a useful aid, not a proof.

What does thinking before answering look like in practice?

Asked which of three tents suits a tall person on a windy trip under a budget, a model answering directly was right 68 times in 100. With thinking on, it wrote about 2,000 tokens of private working first and was right 91 times. On simple questions, thinking changed nothing but the bill.

Watch it happen

These stories show thinking before answering at work, one step at a time.