Beyond Adam: Meet Yogi – The Optimizer That Tames Noisy Gradients
Yogi adds a tiny bit of compute per step and may need slightly more memory. In practice, it's negligible for most models. yogi optimizer
Yogi won't replace Adam everywhere, but it's an excellent tool to keep in your optimizer toolbox – especially when gradients get wild. Beyond Adam: Meet Yogi – The Optimizer That
Developed by researchers at Google and Stanford, Yogi modifies Adam's adaptive learning rate mechanism to make it more robust to noisy gradients. yogi optimizer