8. Optimisation and Reasoning

This is the eigth lecture in the Language Models and Intelligent Agentic Systems course, run by Meridian Cambridge in collaboration with the Cambridge Centre for Data Driven Discovery (C2D3).

This lecture covers how strong optimisation pressure can lead to goal-driven agentic systems with sophisticated planning and reasoning capabilities. We start by discussing examples of agents, and how RL training can lead to emergent capabilities. We then step back and consider a broader notion of 'optimising system' that applies beyond a single-agent setup. We discuss how agentic systems are likely to pursue certain goals that are instrumental to their aims, but not of inherent value. Finally, we examine processes that develop and select agentic systems, and how these selected systems may pursue objectives that conflict with their creators' intentions.