What improved when a game agent got better?

A rising score might reflect a better network, deeper search or a loophole. Trace what the agent observes, what it tries and why the resulting move earns a reward.

Questions in this reading · 5

Can a network evolve its own wiring?
Reward and fitness both give feedback. What changes?
A better network or a longer look ahead?
Did the game agent see too little or optimize the wrong thing?
What cannot be recovered from the current observation?

Can a network evolve its own wiring?

First follow activations through a fixed network. In MarI/O, selection changes the connections themselves. Following a signal and searching for a useful topology are different operations.

Compare original passages and timestamps · 2

the heart of the network as an information processing mechanism

But what is a neural network? | Deep learning chapter 1 · ▶ 4:39–4:43

without presupposing the best structure for the neurons and their connections

MarI/O - Machine Learning for Video Games · ▶ 4:58–5:06

Reward and fitness both give feedback. What changes?

In reinforcement learning, rewards inform action choices. MarI/O uses a fitness score to select entire networks across generations.

Compare original passages and timestamps · 2

there's no supervisor there's just this reward signal

RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning · ▶ 9:52–10:04

Only the neural networks that produce the highest fitness are selected

MarI/O - Machine Learning for Video Games · ▶ 3:47–3:53

A better network or a longer look ahead?

Evolving a network changes its structure. Looking farther ahead changes how an agent evaluates future moves. Each addresses a different limitation.

Compare original passages and timestamps · 2

without presupposing the best structure for the neurons and their connections

MarI/O - Machine Learning for Video Games · ▶ 4:58–5:06

this algorithm does not think very far ahead

Computer program that learns to play classic NES games · ▶ 15:25–15:34

Did the game agent see too little or optimize the wrong thing?

A limited input can hide distinctions needed for a decision. A misleading objective can reward a bad move even when the state is visible.

Compare original passages and timestamps · 2

the representation is ambiguous and of course when it gets it wrong

30 Weird Chess Algorithms: Elo World · ▶ 31:51–32:17

playing Tetris well requires some thinking ahead, and this algorithm does not think very far ahead.

Computer program that learns to play classic NES games · ▶ 15:23–15:34

What cannot be recovered from the current observation?

Ask which distinctions the input representation discards. Earlier observations, feedback or prior knowledge may change what can be inferred.

Compare original passages and timestamps · 3

this has to have some mistakes since the representation is ambiguous

30 Weird Chess Algorithms: Elo World · ▶ 32:02–32:08

the future is independent of the past given the present

RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning · ▶ 43:06–43:33

The white squares stand for blocks the player can stand on

MarI/O - Machine Learning for Video Games · ▶ 1:10–1:17

Try it on your own example

To test your explanation, would you first change the observations, the search depth or the reward?