CS 395T: Foundations of Modern Generative AI

Course information: here

This course will cover many of the foundational ideas behind the remarkable progress in generative AI in the last few years, focusing primarily on recent research on large language models (LLMs) and diffusion models. Topics include: the transformer architecture, pretraining/scaling laws, optimization, post-training & reinforcement learning, diffusion models, and selected topics on AI safety/alignment.

In general, the course will approach these topics from a “CS theory-friendly” perspective: for most of the topics, we will partition the material into two parts, describing both (a) what is done empirically and (b) some theoretical models that have been developed to help us better understand the former. The hope is to illuminate what is known (and what remains to be done) around how ideas from theory can help us to develop predictive “toy” models of training and inference for generative AI.

• Stanford CS336: Language Modeling from Scratch
• UC Berkeley EE 290/194: Scalable AI
• CMU 10-423/623/723: Generative AI
• Cornell CS5788: Deep Generative Models
• Berkeley CS194/294-267: Understanding LLMs — Foundations and Safety
• Berkeley CS294-280: Advanced LLM Agents
• UIUC CS598: Machine Learning Algorithms for LLMs
• Princeton COS597G: Understanding Large Language Models
• Cornell CS4787/5777: Principles of Large-Scale Machine Learning Systems
• Stanford CME296: Fundamentals of Diffusion Models
• Washington CSE599J: Social Reinforcement Learning
• Anthropic Alignment Science Blog
• Austin et al., How to Scale Your Model: A Systems View of LLMs on TPUs (2025)