Accession Number : ADP007159

Title :   Note on Learning Rate Schedules for Stochastic Optimization,

Corporate Author : YALE UNIV NEW HAVEN CT DEPT OF COMPUTER SCIENCE

Personal Author(s) : Darken, Christian ; Moody, John

Report Date : 1992

Pagination or Media Count : 5

Abstract : We present and compare learning rate schedules for stochastic gradient descent, a general algorithm which includes LMS, on-line back-propagation and k-means clustering as special cases. We introduce search-then-converge type schedules which outperform the classical constant and running average (l/t) schedules both in speed of convergence and quality of solution.

Descriptors :   *ALGORITHMS, *GRADIENTS, *LEARNING, CLUSTERING, CONSTANTS, CONVERGENCE, DESCENT, QUALITY, RATES, VELOCITY.

Subject Categories : Statistics and Probability

Distribution Statement : APPROVED FOR PUBLIC RELEASE