Research

Learning Mechanics

I study how optimization dynamics shape what neural networks learn. I am interested in how training time, model scale, and numerical precision influence learning trajectories and generalization. My aim is to develop mechanistic explanations that make testable predictions, connecting mathematical analysis with controlled training experiments.

In our work on Slingshot loss spikes, we identify a mechanism in which finite-precision errors break the zero-sum constraint of cross-entropy gradients. The resulting drift creates a positive feedback loop between classifier and feature means, which we call Numerical Feature Inflation. This explains rapid norm growth and the onset of loss spikes in the settings studied, while also showing that numerical drift can drive parameter growth without visible spikes. This work motivates my broader interest in how the numerical implementation of a learning rule changes its dynamics.

Generative Models

My current focus is on the theoretical properties of deep generative models, especially generalization and memorization in diffusion models. I want to understand how denoisers learn reusable statistical structure from finite data, and how this supports samples that are both novel and faithful to the underlying distribution. I am particularly interested in the roles of architectural inductive bias, noise level, and the local and multiscale structure of natural images.

Physics of Neural Networks

I draw on statistical mechanics, Bayesian inference, and differential geometry to formulate questions about high-dimensional learning systems. I am interested in how noise level and dataset size change posterior concentration, and whether ideas from disordered systems and nonequilibrium physics can clarify transitions between generalization and memorization. In particular, I want to understand when predictions draw on many training examples and when a few examples begin to dominate.