Part I. What is a Language Model?
IN THIS LESSON
This is the third lecture in the Language Models and Intelligent Agentic Systems course, run by Meridian Cambridge in collaboration with the Cambridge Centre for Data Driven Discovery (C2D3).
This lecture covers neural scaling laws - mathematical relationships between the loss achieved by a model and the model size, dataset size, and the total compute used to train the model. We also discuss historical trends in compute allocation for frontier models, and new capabilities which arise as models scale.