HackerTube 中文?
Representation, intuition and feedback →

How do you update on evidence—and choose what to ask next?

A test result and a Wordle clue both rule out possibilities. The crucial difference is whether you already know the result or still have to choose the question.

Questions in this reading · 3
  1. First remove the cases that do not fit the evidence
  2. Why is the calculation different before the colors appear?
  3. Why can the same light or sound prompt a different judgment?

First remove the cases that do not fit the evidence

Imagine a thousand cards. Ten represent people with a condition. A test marks nine of those ten red, but also marks 89 of the other 990 red. Given a red card, look for those nine among the 98 red cards, not among the ten cards representing the condition. Both 9/10 and 9/98 are correct calculations. They answer questions pointing in opposite directions. These are hypothetical teaching figures, not the performance of a particular test.

Passages and timestamps · 1
And then suppose that among the remainder without cancer, 89 get false positives, and 901 correctly get negative results.
The medical test paradox, and redesigning Bayes' rule · ▶ 1:37–1:44

Another talk replaces the cards with areas. The prior sets the initial size of each region. Once you see the description, compare only the parts consistent with it. Use the pictures to check each other: if the area left in your drawing does not represent every red card, a group is missing from your denominator.

Passages and timestamps · 1
In the standard notation, this vertical bar means given that, as in we're restricting our view only to the possibilities where the evidence holds.
Bayes theorem, the geometry of changing beliefs · ▶ 5:10–5:18
Lay out the thousand cards
RedNot red
Condition present91
Condition absent89901
Check yourself: handed a red card, where do you look?

Read down the red column: the denominator is 9 + 89. For the fraction marked red among those with the condition, read across the first row instead: 9 + 1.

Why is the calculation different before the colors appear?

Before a Wordle guess you do not know which pattern will light up. Judging a word by its luckiest possible feedback is misleading. Look at how much uncertainty each result would remove and average by the chance of getting it. Updating after seeing the colors is the counterpart of keeping only the red cards. The first calculation designs a question; the second interprets an answer.

Passages and timestamps · 1
The way we measure the quality of this guess will be to take the expected value of this information.
Solving Wordle using information theory · ▶ 10:43–10:48

Try four candidate words with prior probabilities of 70%, 10%, 10% and 10%. A guess that splits them into two groups of two still splits the probability 80–20. Ask instead whether it is the most common word. One word versus three gives a more balanced 70–30 probability split. Of these two yes-or-no questions, the second has higher expected information. In the talk, a group containing 32 unlikely words is about as probable as one containing just eight. Even after weighting the groups, specify the goal: maximizing expected information and winning on this guess are different objectives.

Passages and timestamps · 1
So a quarter as many matches, but it's about as likely.
Solving Wordle using information theory · ▶ 23:06–23:09

Why can the same light or sound prompt a different judgment?

Lotto’s tiles send the same light to the eye while the surround suggests shadow on one tile. Seth supplies a speech cue and replays a distorted sound that now seems intelligible. The examples change expectations about illumination and sound source respectively. Hoffman’s desktop-icon analogy asks a further question: does usefulness for acting imply resemblance to what a representation stands for? The demonstrations test particular changes in perception. His broader evolutionary claim requires a separate argument.

Passages and timestamps · 3
Because if the tile in that shadow were in fact in shadow, and reflecting the same amount of light to your eye as the one outside the shadow, it would have to be more reflective -- just the laws of physics.
Optical illusions show how we see | Beau Lotto · ▶ 7:29–7:40
All that's changed is your brain's best guess of the causes of that sensory information. And that changes what you consciously hear.
Your Brain Hallucinates Your Conscious Reality | Anil Seth | TED · ▶ 7:00–7:07
Anyone who thought that misinterprets the purpose of the interface. It's not there to show you the reality of the computer. In fact, it's there to hide that reality.
Do we see reality as it is? | Donald Hoffman | TED · ▶ 12:25–12:35

Try it on your own example

Design two yes-or-no questions for four cards with unequal prior probabilities. Compare the questions before asking, then update assuming a yes. Did the two calculations use the same denominator?