Katie Everett

I am a Research Scientist at Google DeepMind where I work on understanding and improving training dynamics in large-scale neural networks. My research focuses on high-dimensional optimization, scaling laws and parameterization. I am also interested in reasoning and the role of discrete compositional structures in representations.

In addition, I am a final-year PhD student in Electrical Engineering and Computer Science at MIT advised by Leslie Pack Kaelbling. I completed my M.Eng. in electrical engineering and computer science and my B.S. with a double major in computer science and mathematics, both at MIT.

Google Scholar X / Twitter GitHub LinkedIn

Selected Research

High-Dimensional Optimization and Momentum

Optimizer Memory Schedules for Outscaling the Overtraining Axis
Katie Everett and Shikai Qiu
Preprint 2026
Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
Katie Everett and Elliot Paquette
GitHub with Lean autoformalization
Preprint 2026
Logarithmic-time Schedules for Scaling Language Models with Momentum
Damien Ferbach, Courtney Paquette, Gauthier Gidel, Katie Everett, and Elliot Paquette
GitHub
Preprint 2026
Dimension-adapted Momentum Outscales SGD
Damien Ferbach, Katie Everett, Gauthier Gidel, Elliot Paquette, and Courtney Paquette
NeurIPS 2025 Spotlight

Transformer Scaling and Architectures

Scaling Exponents Across Parameterizations and Optimizers
Katie Everett, Lechao Xiao, Mitchell Wortsman, Alexander A. Alemi, Roman Novak, Peter J. Liu, Izzeddin Gur, Jascha Sohl-Dickstein, Leslie Pack Kaelbling, Jaehoon Lee, and Jeffrey Pennington
ICML 2024
Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth
Katie Everett
GitHub
Preprint 2026
Small-scale proxies for large-scale Transformer training instabilities
Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie Everett, Alex Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, and Simon Kornblith
ICLR 2024 Oral
NanoDO: A minimal Transformer decoder-only language model implementation in JAX
Peter J. Liu, Roman Novak, Jaehoon Lee, Mitchell Wortsman, Lechao Xiao, Katie Everett, Alexander A. Alemi, Mark Kurzeja, Pierre Marcenac, Izzeddin Gur, Simon Kornblith, Kelvin Xu, Gamaleldin Elsayed, Ian Fischer, Jeffrey Pennington, Ben Adlam, and Jascha Sohl-Dickstein
GitHub 2024

Causal and Compositional Representation Learning

Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA
Sébastien Lachapelle, Pau Rodríguez López, Yash Sharma, Katie Everett, Rémi Le Priol, Alexandre Lacoste, and Simon Lacoste-Julien
CLeaR 2022
Nonparametric partial disentanglement via mechanism sparsity: Sparse actions, interventions and sparse temporal dependencies
Sébastien Lachapelle, Pau Rodríguez López, Yash Sharma, Katie Everett, Rémi Le Priol, Alexandre Lacoste, and Simon Lacoste-Julien
JMLR 2026
GFlowNet-EM for learning compositional latent variable models
Edward J. Hu, Esmeralda S. Whitammer, Moksh Jain, Katie Everett, Alexandros Graikos, and Yoshua Bengio
ICML 2023
GFlowNets and variational inference
Esmeralda S. Whitammer, Salem Lahlou, Tristan Deleu, Xu Ji, Edward Hu, Katie Everett, Dinghuai Zhang, and Yoshua Bengio
ICLR 2023