HackerTube 中文?

The spelled-out intro to neural networks and backpropagation: building micrograd

Andrej KarpathyYouTube ↗

Further reading 2

Why do tests pass when the program is wrong? →Gary Bernhardt’s isolated tests and Andrej Karpathy’s neural network both deliver good news when the code is wrong.How does a network change after a wrong answer? →Forward propagation computes a prediction. In micrograd, a loss and its gradients connect that prediction to the next parameter update.

Caption notes

Build automatic differentiation from scalar operations, apply the chain rule backward and train a small classifier. Reusing accumulated gradients reveals why training loops clear them.

Original passages · 6
back propagation is is just a recursive application of chain rule backwards through the computation graph
The spelled-out intro to neural networks and backpropagation: building micrograd · ▶ 51:04–51:12
we have to accumulate these gradients these gradients add
The spelled-out intro to neural networks and backpropagation: building micrograd · ▶ 85:23–85:32
these gradients just add up and they never get reset to zero
The spelled-out intro to neural networks and backpropagation: building micrograd · ▶ 131:39–131:47
you may have lots of bugs in the code and uh your network might actually work
The spelled-out intro to neural networks and backpropagation: building micrograd · ▶ 133:44–133:51
efficiently evaluate the gradient of some kind of a loss function
The spelled-out intro to neural networks and backpropagation: building micrograd · ▶ 0:55–1:07
changing this variable is not changing the loss function
The spelled-out intro to neural networks and backpropagation: building micrograd · ▶ 31:16–31:27

Hacker News comments

View original on HN ↗