A Hackers' Guide to Language Models
Jeremy HowardYouTube ↗
Further reading 3
Will the leaderboard winner solve your problem? →A public score can shortlist models. A new version of your own task can reveal what the score misses.From predicting a token to using a language model →Building a predictor makes its training target concrete. Using a trained model shows how context and instructions put that predictor to work.What if the model has already seen the test? →A public score becomes harder to interpret when the training corpus is unknown. Novel questions probe a different failure mode, without proving what the original corpus contained.
Caption notes
Howard connects next-token training to practical assistants through code: prompting, function calls, local quantized models and retrieval.
Original passages · 5
these metrics are not particularly well aligned with real life usagethere's no context here being sent to the embedding model so it's actually going to have no idea I'm talking about new lmfitit does not actually return the number nine instead it returns something saying please call a functionknows how to predict the next wordthis is called retrieval augmented generation