AI Engineering, story 20One line of the prompt changes

One small change

1 of 13

One line of the prompt changes: AI Engineering, story 20

HM Nayem adds a single sentence to an assistant's instructions. It looks better. Follow the check that shows it quietly got worse.

How a team knows a change made things better, not just different.

The story, step by step

  1. One small change

    HM Nayem built a little assistant for the hiking club, called Trail Buddy. Members like it but say the answers ramble. So HM Nayem adds one line to its instructions. What could go wrong with one line?

  2. Try it once

    A quick test. HM Nayem asks the question members ask most. The answer is short and correct. It looks like a clear improvement, and it is very tempting to stop here.

  3. The trouble with trying it once

    Members ask hundreds of kinds of question, and HM Nayem tried one. A line in a prompt touches every answer the assistant will ever give. And because models pick words by chance, even this one answer could differ next time.

  4. A set of test questions

    So HM Nayem keeps a list: forty real questions from members, each with a note on what a good answer must include. This is called an eval set. It is the most valuable thing an AI product owns after its prompt.

  5. Checks that code can do

    Every answer has to be graded, forty at a time, again and again. Some checks are easy for ordinary code. Does the answer contain the word Saturday? Is it under two sentences? Those take no time at all.

  6. Checks that need a judge

    Whether gear advice is safe cannot be found by matching words. So a second model does that grading. It gets the question, the answer and a written rubric, and returns pass or fail with a reason. It is used only as a judge.

  7. First, the score before

    Before judging the new line, HM Nayem needs something to compare it with. So all forty questions are run with the old instructions. This is the baseline: 36 of 40.

  8. Then, the score after

    Now the same forty questions with the new line added. Same questions, same graders, one thing changed. The score drops to 31. The line that looked like an obvious improvement made the assistant worse.

  9. Which ones broke

    A score alone is not much use. The list of failures is. The new line did exactly what it said: shorter answers. And in doing so it squeezed the safety details out of five of them.

  10. A better line

    Now HM Nayem knows what to fix. Not “be short”, but “be short without cutting safety”. The failures wrote the new instruction.

  11. Run it again

    Forty questions once more. This time 38 pass. Shorter answers, and better than the baseline. That is what an improvement looks like when it is measured rather than hoped for.

  12. Every change, every time

    HM Nayem makes it automatic. Any change to the instructions, and any move to a newer model, runs all forty questions first. If the score falls, the change does not go live.

  13. The list keeps growing

    One last habit. Next month a member reports a bad answer about dogs on trails. That question goes straight into the list, as number 41. Every real mistake becomes a test, so it can never quietly come back.

You cannot tell if a prompt is better by trying it once.

Ideas in this story