HackerTube 中文?
Learning signals and model behavior →

How much does a successful demonstration establish?

A program gives the expected output. Between “it did” and “it will” lie the test distribution, formal assumptions and deployment context. Those boundaries are easy to miss under pressure to ship.

Questions in this reading · 4
  1. Does running a model explain its decision?
  2. What if the model has already seen the test?
  3. What can a formal check actually verify?
  4. How does cheaper automation become a deployment race?

Does running a model explain its decision?

Executing a model yields an answer. Explaining why that answer followed, or predicting its behavior under a new input, requires further evidence.

Compare original passages and timestamps · 2
you connect that inscrutable thing to important real life things like financial markets
USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? · ▶ 16:27–16:37
we don't know how these parameters collaborate to actually perform that
[1hr Talk] Intro to Large Language Models · ▶ 12:25–12:33

If two curves agree, can you tell the mechanisms apart?

Replace a correct answer with a correct curve. In the first model, exponential growth follows from daily additions proportional to the current count. Make each contact depend on encountering someone still susceptible and the early curves may look almost identical before diverging. Showing only the start makes a poor test between those explanations.

Passages and timestamps · 1
For a random shuffling model like this, that could mean including a factor like 1 minus the proportion of people in the world who are already infected.
Exponential growth and epidemics · ▶ 4:39–4:46

The moving-dot simulation gives the converse example. Isolating a dot or shortening the time it can transmit can have the same effect inside the model. A finer plot of total cases need not distinguish the mechanisms. You need a different observation, such as whether the dot is still meeting neighbors. This is a lesson about distinguishing models, not an estimate of real intervention effects.

Passages and timestamps · 1
If there exists an antiviral therapeutic that can move people out of this infectious category quickly, it has the same effect on the model as isolating those cases.
Simulating an epidemic · ▶ 10:15–10:24

What if the model has already seen the test?

A public score becomes harder to interpret when the training corpus is unknown. Novel questions probe a different failure mode, without proving what the original corpus contained.

Compare original passages and timestamps · 2
we don't know what gpt4 was trained on I don't know what gpt4 was trained on
Sparks of AGI: early experiments with GPT-4 · ▶ 17:35–17:49
some of the questions from these things actually leaks through to some of the training sets
A Hackers' Guide to Language Models · ▶ 59:53–60:04

What can a formal check actually verify?

A tool needs a precise property to check. Translating what someone intended into a formal statement is itself a source of error.

Compare original passages and timestamps · 2
in the limit I'll like you know formally verify all of my protocols.
Nicholas Carlini - Black-hat LLMs | [un]prompted 2026 · ▶ 25:17–25:25
Before we can start proving new theorems, we need to be able to state old theorems
The Future of Mathematics? · ▶ 53:37–53:42

How does cheaper automation become a deployment race?

One account considers incentives to replace costly work. The other asks how competition changes the decision to release a technology whose capabilities are still emerging.

Compare original passages and timestamps · 2
A tenth the speed is still cost effective when it's a hundredth the price.
What Happened to Horses Is Happening to Us · ▶ 2:27–2:36
if that technology confers power it will start a race.
The A.I. Dilemma - March 9, 2023 · ▶ 6:42–7:04

Try it on your own example

State the conditions on the demonstration: what survives a new question or a change of assumptions?