Francois Chollet — Why the biggest AI models can't solve simple puzzles

Dwarkesh Podcast

Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today.

I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through.

It was really fun discussing/debating the cruxes. Enjoy!

Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

Timestamps

(00:00:00) – The ARC benchmark

(00:11:10) – Why LLMs struggle with ARC

(00:19:00) – Skill vs intelligence

(00:27:55) - Do we need “AGI” to automate most jobs?

(00:48:28) – Future of AI progress: deep learning + program synthesis

(01:00:40) – How Mike Knoop got nerd-sniped by ARC

(01:08:37) – Million $ ARC Prize

(01:10:33) – Resisting benchmark saturation

(01:18:08) – ARC scores on frontier vs open source models

(01:26:19) – Possible solutions to ARC Prize



This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com
More description

Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today.

I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through.

It was really fun discussing/debating the cruxes. Enjoy!

Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

Timestamps

(00:00:00) – The ARC benchmark

(00:11:10) – Why LLMs struggle with ARC

(00:19:00) – Skill vs intelligence

(00:27:55) - Do we need “AGI” to automate most jobs?

(00:48:28) – Future of AI progress: deep learning + program synthesis

(01:00:40) – How Mike Knoop got nerd-sniped by ARC

(01:08:37) – Million $ ARC Prize

(01:10:33) – Resisting benchmark saturation

(01:18:08) – ARC scores on frontier vs open source models

(01:26:19) – Possible solutions to ARC Prize



This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com
2024-06-11 93 min
Listen elsewhere

Available Results

Generated results are saved to your library for reuse and search.

No generated results are available for this episode yet.

Transcript

No transcript is available for this episode yet.
Sign in to generate a transcript for review.
Sign in

Chapters

No chapters available.