Gradient Parkour Learning Rate

Gradient when using cosine annealing with warm restarts learning rate scheduler Parkour!

Steve Carell as Michael Scott from The Office throws his arms outward and shouts “Parkour!” while sitting in a living room. The joke compares the dramatic excitement of the character’s failed parkour enthusiasm to using a cosine annealing learning-rate scheduler with warm restarts, a reaction for sudden or over-the-top machine-learning enthusiasm. EN: Steve Carell as Michael Scott from The Office throws his arms outward and shouts “Parkour!” while sitting in a living room. The joke compares the dramatic excitement of the character’s failed parkour enthusiasm to using a cosine annealing learning-rate scheduler with warm restarts, a reaction for sudden or over-the-top machine-learning enthusiasm.

• @profunctor_io / 3631 2019-12-10