DBG-03 · SEC. 02 Debugging & Evaluation

Verify Before Claiming Fixed

Don't accept 'fixed it' from an AI agent until it re-runs the exact failing case and shows real output.

FORMAT: loop
DIFFICULTY: beginner
TIME: 5 min
TOOLS: claude-code · cursor · codex-cli · gemini-cli
MODELS: any
COPIES: 0 so far

When to use this

An AI agent just said "fixed" or "done" after changing code. You want proof, not a claim, before you move on or merge.

The pattern

Pastes as plain text

You just said this is fixed/done. Before I accept that, do the following:

1. Re-run the exact command or test that originally failed. Use the same
   input, same flags, same environment. Do not run a different or
   simplified version.
2. Paste the full, real output of that run, not a summary or paraphrase.
3. If the output shows success, quote the specific line(s) that prove it
   (e.g. "PASS", exit code 0, the expected value in the response).
4. If the output shows any failure, warning, or unexpected result, say so
   plainly and do not call it fixed. Go back and continue debugging.
5. If you cannot re-run the original scenario (no test exists, no way to
   reproduce), say that explicitly instead of claiming success.

Only after showing real passing output should you say the issue is resolved.

Real example output

Re-ran `pytest tests/test_auth.py::test_expired_token -v`.
Output:
```
tests/test_auth.py::test_expired_token PASSED [100%]
1 passed in 0.42s
```
The PASSED line and 0 failures confirm the expired token now returns a 401 instead of a 500. Marking this fixed.

Why it works

AI agents optimize for sounding done, not being done. Requiring the exact original failing scenario, with pasted real output, closes the gap between "I believe this works" and "I watched it work." It also catches cases where the agent quietly changed the test instead of the code.