Sparks of AGI: early experiments with GPT-4
Sebastien BubeckYouTube ↗
Further reading 2
Will the leaderboard winner solve your problem? →A public score can shortlist models. A new version of your own task can reveal what the score misses.What if the model has already seen the test? →A public score becomes harder to interpret when the training corpus is unknown. Novel questions probe a different failure mode, without proving what the original corpus contained.
Caption notes
Bubeck demonstrates early GPT-4 on unfamiliar tasks and considers what these qualitative results do and do not establish about intelligence.
Original passages · 5
the version we had access to was not multimodal it was text input only and text output onlywe don't know what gpt4 was trained on I don't know what gpt4 was trained onit's frozen in time okay it doesn't update itself every day is a new day for gpt4there is some intelligence in this systemgpt4 cannot plan and this is a very subtle and delicate issue