At first glance, "Learning Rate Decay" might sound like an undesirable side effect or something that happens inadvertently during model training.
Learning Rate Decay: The Misnomer
At first glance, "Learning Rate Decay" might sound like an undesirable side effect or something that happens inadvertently during model training.