About Me
Hello! I am Kaiyue Wen. I am a first-year Phd student at Stanford University. I am currently rotating with Tengyu Ma while working together with Percy Liang. I graduated from Tsinghua University, where I was a member of Yao’s pilot class. Here are my CV and Publications. During my undergraduate study, I am fortunate to be advised by Tengyu Ma, Zhiyuan Liu, Andrej Risteski, Jingzhao Zhang, Yuhao Wang and Zhiyuan Li.
My research interest spreads broadly in deep learning, including theory and applications. My long-term goal is to understand the physics behind deep learning and I believe a combination of theoretical analysis and empirical study is essential for this goal.
Recent News
- Jan, 2025 New preprint (Global Load Balancing Helps Expert Specialization) on arxiv!
- Jan, 2025 3 papers (River Valley Landscape, RNNs are not Transformers (Yet), Optimization Analysis on Chain-of-Thought) accepted at ICLR 2025!
- Dec, 2024 Residual Permutation Test is accepted at AoS!
- Oct, 2024 New preprints River Valley Landscape and Optimization Analysis on Chain-of-Thought on arxiv!
- Sep, 2024 Start my Ph.D. study at Stanford University! I am currently rotating with Percy Liang.
- Jul, 2024 Graduated from Tsinghua University with a Bachelor’s degree in Computer Science.
- May, 2024 Receive and accept the offer from Stanford University! I am honored to receive the Stanford Graduate Fellowship.
- Feb, 2024 New preprint RNNs are not Transformers (Yet) on arxiv!
- Oct, 2023 Awarded the National Scholarship (top 0.2%)!
- Sep, 2023 2 papers (Sharpness&Generalization, (Un)interpretability of Transformers) accepted at Neurips 2023! Sharpness&Generalization is received as oral.
- Sep, 2023 Receive the silver medal for Yao Award (Top 4 in Yao’s pilot class)!
- Aug, 2023 Return to China for my senior year in Tsinghua.
- Jul, 2023 Visit Hawaii for ICML 2023! Always great to see old friends.
- Jun, 2023 Residual Permutation Test receive Major Revision from AoS.
- Jun, 2023 Visiting Tengyu Ma at Stanford!
- May, 2023 Visit Rwanda for ICLR 2023!
- Mar, 2023 Reviewing ICML for the first time!
- Mar, 2023 New preprint Solving LPN with Neural Networks on arxiv!
- Feb, 2023 Visiting Andrej Risteski at CMU!
- Jan, 2023 2 papers (Understanding SAM, Not Benign Overfitting) accepted at ICLR 2023!
- Dec, 2022 New preprint Residual Permutation Test on arxiv!
- Dec, 2022 New preprint Understanding SAM on arxiv!
- Oct, 2022 1 paper (Skill Neurons) accepted at EMNLP 2022.
- Jun, 2022 New preprint Not Benign Overfitting on arxiv!
One More Thing
I keep a firm faith in analytical thinking, hard work, and consistent self-improvement. Any advice or feedback is welcome. You can use this Anonymous Form or discuss with me in person.