Jethro's Braindump

LARS Optimizer

Layer-wise Adaptive Rate Scaling (LARS) is a Neural Network Optimizer. The technique allows Large Batch Training without significant decrease in accuracy (You, Gitman, and Ginsburg 2017). One of the secondary goals is Fast Neural Network Training.



You, Yang, Igor Gitman, and Boris Ginsburg. 2017. “Large Batch Training of Convolutional Networks.” CoRR.