HackerSparks of AGI: early experiments with GPT-4
Sebastien BubeckYouTube ↗
Further reading 2
Will the leaderboard winner solve your problem? →A public score can shortlist models. A new version of your own task can reveal what the score misses.How much does a successful demonstration establish? →A program gives the expected output. Between “it did” and “it will” lie the test distribution, formal assumptions and deployment context. Those boundaries are easy to miss under pressure to ship.This video helps answer · What if the model has already seen the test?
Caption notes
Bubeck demonstrates early GPT-4 on unfamiliar tasks and considers what these qualitative results do and do not establish about intelligence.
Original passages · 5
the version we had access to was not multimodal it was text input only and text output onlywe don't know what gpt4 was trained on I don't know what gpt4 was trained onit's frozen in time okay it doesn't update itself every day is a new day for gpt4there is some intelligence in this systemgpt4 cannot plan and this is a very subtle and delicate issue