From the Ball-proximal (Broximal) Point Method to Efficient Training of LLM

on YouTube ↗52m

" summary

Proposes the Ball-Proximal (Broximal) Point Method as a ball-constrained analogue of proximal methods and proves surprising guarantees: linear convergence and finite termination for nonsmooth convex problems, extendable to non‑Euclidean norms. Shows practical approximations—layerwise LMO minimization and momentum—that recover Muon/Scion behavior (Muon uses ~five Newton–Schulz orthogonalization steps) and yields theoretical step sizes matching tuned values; measures layerwise smoothness constants mostly between 1 and 100 and validates faster wall‑clock training (NanoGPT, CNN) with compression and error‑feedback.

AI-written summary · about

" chapters (6)

1. 0:00Welcome
2. 9:49Broximal point method
3. 22:49Broximal convergence
4. 31:40Theory to Muon optimizers
5. 43:43Muon foundations
6. 51:03Broximal in federated learning

" related talks

• 46mSmooth Nonconvex Min-Max OptimizationMeisam Razaviyayn · Adversarial Approaches in Machine Learning 2022
• 51mAre Single-Loop Algorithms Sufficient for Unbalanced Minimax Optimization?Niao He · Adversarial Approaches in Machine Learning 2022
• 42mOptimization Crash CourseAshia Wilson · Geometric Methods in Optimization and Sampling Boot Camp
• 01h 18mOptimization IBen Recht · Big Data Boot Camp 2013
• 12mEquilibrium Computation and the Foundations of Multi-Agent Machine LearningEmmanouil Zampetakis · Meet the Fellows Welcome Event
• 01h 01mML for discrete optimization: Theoretical foundationsEllen Vitercik · Graph Learning Meets Theoretical Computer Science 2025
• 01h 00mMin-Max Optimization from a Dynamical Systems ViewpointPanayotis Mertikopoulos · Adversarial Approaches in Machine Learning 2022
• 01h 05mTutorial: Federated Optimization, Part IIIPeter Richtarik · Federated and Collaborative Learning Boot Camp 2026