Accession Number : ADP007159
Title : Note on Learning Rate Schedules for Stochastic Optimization,
Corporate Author : YALE UNIV NEW HAVEN CT DEPT OF COMPUTER SCIENCE
Personal Author(s) : Darken, Christian ; Moody, John
Report Date : 1992
Pagination or Media Count : 5
Abstract : We present and compare learning rate schedules for stochastic gradient descent, a general algorithm which includes LMS, on-line back-propagation and k-means clustering as special cases. We introduce search-then-converge type schedules which outperform the classical constant and running average (l/t) schedules both in speed of convergence and quality of solution.
Descriptors : *ALGORITHMS, *GRADIENTS, *LEARNING, CLUSTERING, CONSTANTS, CONVERGENCE, DESCENT, QUALITY, RATES, VELOCITY.
Subject Categories : Statistics and Probability
Distribution Statement : APPROVED FOR PUBLIC RELEASE