AI Engineering, story 19Designing a tool

The model picked the wrong tool

1 of 14

Designing a tool: AI Engineering, story 19

The model keeps choosing the wrong tool. It is not being stupid. See what it was given to go on, and fix the tools instead of the model.

A model chooses and uses a tool from its name, description, inputs and results. Designing those well fixes most tool mistakes.

The story, step by step

  1. The model picked the wrong tool

    HM Nayem asks whether a refund has arrived. The assistant has a tool for exactly that. The model uses a different one, looks up the order, and reports that the jacket was delivered last week. Every word is true and none of it answers the question. The model is not being stupid. Let us look at what it had to go on.

  2. All the model knows about a tool

    This is everything the model was shown about its two tools. Two names that could mean anything, and two descriptions of four words each. Given this, a careful new employee would guess too. A model chooses and uses a tool from its name, its description, its inputs and its results. Those four things are the whole design.

  3. A name that says what it does

    The first fix costs nothing. A name is read every time the model considers a tool, so it should say what the tool is for in the words a customer would use. Verb first, then the thing. The model now has a strong hint before it reads a word of description.

  4. A description for a newcomer

    A good description is written for someone bright who started this morning. It says what the tool returns, when to use it, and when not to, which is the part people leave out. Naming the neighbouring tool in the "when not to" line removes most confusion between the two.

  5. Inputs that are hard to get wrong

    The old tool took one input called "q", any text at all. The model filled it with whole sentences. Each input should have a clear name, a stated shape and, where possible, a fixed list of choices. A model cannot misspell an option it has to pick from a list. The harness refuses anything that does not fit, before the tool runs.

  6. Too many tools

    Pleased with tools, the shop adds more: forty in all. Each one's description goes into every prompt, and the model must choose between forty. HM Nayem measures how often it picks the right one. The more tools, the more often it picks a near miss. A team is better off with a dozen sharp tools than forty blunt ones.

  7. Two tools that overlap

    Among the forty are find_order and search_orders. One takes a number, the other takes words, and for most questions either would do. When two tools could both answer, the model's choice between them is close to a coin toss, and the two return different shapes. They are merged into one tool with two optional inputs.

  8. A tool for the whole job

    The first tools were copies of the database: one to find a customer, one to list their orders, one to get an order's items. To answer "what did I buy last month?" the model needed three calls in the right order. Tools should match the jobs people ask for, not the tables behind them. One tool now does the whole lookup.

  9. Results the model can use

    What a tool returns is read by the model, so it is part of the design too. Raw results are full of internal codes and fields nobody asked for. A good result uses the words a person would use, leaves out what is not needed, and says plainly when there is more than it has shown.

  10. Errors that teach

    When a tool fails, its error message goes to the model as the result. "Error 400" teaches nothing, and the model tries the same thing again. A good error says what was wrong, what would be right, and what else to try. The model reads it and corrects itself on the next round.

  11. Tools that change things say so

    Looking up a refund harms nobody. Issuing one moves money. The harness needs to know which is which, so each tool carries a label: reads only, changes something, or cannot be undone. The label decides whether a person is asked first. A tool that changes something is also built so that running it twice does no more than running it once.

  12. Test the tools like any other part

    How does HM Nayem know the new tools are better? He writes sixty customer questions, each with the tool that should be used and the inputs it should get. For every version of the tools he counts two things: right tool, and right inputs. No model changed between these rows. Only the names, descriptions and inputs did.

  13. Ask the model what confused it

    For the two questions still failing, HM Nayem does something simple. He shows a model the tool list and the failed conversation, and asks: what about these tools made the wrong choice likely? The answer points at one word. "Status" appears in two descriptions. Models are good at spotting what trips a model.

  14. The same question, at real speed

    The first question again, with nothing changed but the tools. The model reads the names, picks the refund tool without hesitation, fills in a number and a kind, and gets back three plain lines. The model was always able to do this. It needed to be told, clearly, what its tools were for.

When a model misuses a tool, look at the tool first.

Ideas in this story