Evolving Deep Learning Optimizers [R]

via

Machine Learning

We present a genetic algorithm framework for automatically discovering deep learning optimization algorithms. Our approach encodes optimizers as genomes that specify combinations of primitive update terms (gradient, momentum, RMS normalization, Adam-style adaptive terms, and sign-based updates) along with hyperparameters and scheduling options. Through evolutionary search over 50 generations with a population of 50 individuals, evaluated across multiple vision tasks, we discover an evolved optim

More from Machine Learning

Visualizing Loss Landscapes of Neural Networks [P]

Machine Learning·12

Built a efficient and fast MRI compression program called KMRI [P]

Machine Learning·4

Should I follow-up with the editor for a TMLR paper awaiting final decision? [D]

Machine Learning·4

How do I actually learn AI/ML deeply enough to build systems (not just follow tutorials)? [D]

Machine Learning·4

ICML final decisions rant [D]

Machine Learning·9

Real World Physics-Informed AI Applications [D]

Machine Learning·5