Ilya Sutskever full talk “Sequence to sequence learning with neural networks: what a decade” at NeurIPS 2024 in Vancouver, Canada.
Pre-training as we know it will end” and what comes next is superintelligence: agentic, reasons, understands and is self aware.
NeurIPS 2024 — 2024 Conference on Neural Information Processing Systems.
00:00:00 - Opening Remarks
00:01:00 - Reflecting on 10 Years
00:02:30 - The Deep Learning Hypothesis
00:05:30 - Autoregressive Models Explained
00:07:45 - The Era of LSTMs
00:10:30 - Scaling Hypothesis
00:12:30 - Challenges with Data
00:15:00 - Future of AI: Agents and Synthetic Data
00:18:45 - Insights from Biology
00:22:00 - The Path to Superintelligence
00:26:30 - Reasoning and Hallucinations
00:30:00 - Incentives for Ethical AI Development
00:34:30 - Generalization in AI
00:39:00 - Closing Reflections