Architecture
Design choices that make a model trainable, scalable, and expressive.
Foundational Models
A working notebook on the systems behind frontier AI — experiments, paper notes, and reading logs, organized by aspect. Filled in over time.
Aspects
Pre-training
Data, objectives, scaling laws, and the infrastructure behind training at frontier scale.
Capabilities
What models can and cannot do — evaluations, emergent skills, real-world gaps.
Post-training
SFT, preference optimization, RL from human and verifiable feedback, distillation, tool-use, reasoning.
Synthesis
Cross-cutting reads — model families compared, how the aspects interact, where the frontier is heading.
Architecture
- PlaceholderNotes coming soon.
Pre-training
- PlaceholderNotes coming soon.
Capabilities
- PlaceholderNotes coming soon.
Post-training
- PlaceholderNotes coming soon.
Synthesis
- PlaceholderNotes coming soon.