Wen's Blog

My thoughts on deep learning and AI

Fantastic Pretraining Optimizers and Where to Find Them II (Part 2)

A deep dive into hyperball geometry and pretraining optimization strategies.

1 min read · January 02, 2025

2025 · weight decay hyperball optimization deep learning · research
Fantastic Pretraining Optimizers and Where to Find Them II (Part 1)

A deep dive into hyperball geometry and pretraining optimization strategies.

1 min read · January 02, 2025

2025 · weight decay hyperball optimization deep learning · research