HackerDeep Dive into LLMs like ChatGPT
Andrej KarpathyYouTube ↗
Further reading 2
What if the best-scoring move is to pause the game? →Optimizing a score can reveal what its designer forgot to ask for.Where does learning get a signal without an answer key? →Start with an unlabeled patch of an image, then switch to a game that requires a sequence of actions. Neither has an answer key, but finding structure and choosing a move need different signals.This video helps answer · Where do strategies absent from demonstrations come from?
Caption notes
Follow language models from tokenization and pretraining to supervised tuning and reward optimization, including hallucinations and ways a learned reward can be exploited.
Original passages · 7
it doesn't have to recall it it just has access to itdistribute our reasoning and our computation across many tokensthe models they don't see the characters they see tokenswe're going to check them and we're going to do more of what workedoptimization will start to game it and you basically crop itthis process of converting from raw text into these symbolslabelers are asked to come up with prompts