Jan 03, 2025 Fantastic Pretraining Optimizers and Where to Find Them 2.2: The Hitchhiker's Guide to the Weight Norm Theory Jan 02, 2025 Fantastic Pretraining Optimizers and Where to Find Them 2.1: Hyperball Optimization