https://www.newabdullah.com/tags/optimization/ Recent content in Optimization on Abdullah Al Mamun https://www.newabdullah.com/og-default.png https://www.newabdullah.com/og-default.png Hugo en newabdullah 2025 Tue, 15 Sep 2026 00:00:00 +0000 https://www.newabdullah.com/posts/optimization-algorithms-ml-interview/ Tue, 15 Sep 2026 00:00:00 +0000 https://www.newabdullah.com/posts/optimization-algorithms-ml-interview/ Every optimizer worth knowing, from SGD and momentum through AdaGrad, RMSProp, Adam and AdamW to Lion, Shampoo and Muon (as used in production LLM pretraining), plus learning rate schedules, warmup, gradient clipping, accumulation, checkpointing, batch size scaling, optimizer memory, and how to debug a loss that will not go down.