But what is a neural network? | Deep learning chapter 1
3Blue1BrownYouTube ↗
Further reading 3
Can a network evolve its own wiring? →First follow activations through a fixed network. In MarI/O, selection changes the connections themselves. Following a signal and searching for a useful topology are different operations.How does a network change after a wrong answer? →Forward propagation computes a prediction. In micrograd, a loss and its gradients connect that prediction to the next parameter update.What does attention add to layered activations? →A feedforward network propagates activations through its layers. Causal attention lets each position combine information from earlier tokens.
Caption notes
A network for handwritten digits turns pixels into activations using learned weights, biases and nonlinearities. This chapter explains the architecture before training.
Original passages · 5
Whether or not this is what our final network actually does is another questionthe bias tells you how high the weighted sum needs to bethis makes the relevant code both a lot simpler and a lot fasterthe heart of the network as an information processing mechanismthe entire network is just a function