How do you know the system got it right?
Start with the kind of evidence you need: the inputs, the code or the measure of success.
Check the code and inputs
Trace a passing test back to its decision rule and the data available at the time.
Why do tests pass when the program is wrong? →Gary Bernhardt’s isolated tests and Andrej Karpathy’s neural network both deliver good news when the code is wrong.Why does testing a business rule need a database? →Separate who qualifies for a reminder from whether the reminder can be delivered.Can today’s record explain yesterday’s decision? →Keep the value a decision actually used so that later changes do not erase its explanation.Check the measuring stick
Consider what an exam measures and what happens when a system learns to chase the score.
Will the leaderboard winner solve your problem? →A public score can shortlist models. A new version of your own task can reveal what the score misses.What if the best-scoring move is to pause the game? →Optimizing a score can reveal what its designer forgot to ask for.