Accession Number : ADA290097

Title :   Priors, Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines.

Descriptive Note : Memo rept.,

Corporate Author : MASSACHUSETTS INST OF TECH CAMBRIDGE ARTIFICIAL INTELLIGENCE LAB

Personal Author(s) : Girosi, Federico ; Jones, Michael ; Poggio, Tomaso

PDF Url : ADA290097

Report Date : JUN 1993

Pagination or Media Count : 29

Abstract : We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models, Breiman's hinge functions and some forms of Projection Pursuit Regression. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In summary, different multilayer networks with one hidden layer, which we collectively call Generalized Regularization Networks, correspond to different classes of priors and associated smoothness functionals in a classical regularization principle. Three broad classes are Radial Basis Functions that generalize into Hyper Basis Functions, some tensor product splines, and additive splines that generalize into schemes of the type of ridge approximation, hinge functions and one-hidden-layer perceptrons. (AN)

Descriptors :   *KNOWLEDGE BASED SYSTEMS, MATHEMATICAL MODELS, FOURIER TRANSFORMATION, LINEAR SYSTEMS, DATA PROCESSING, PROBLEM SOLVING, REGRESSION ANALYSIS, SOLUTIONS(GENERAL), APPROXIMATION(MATHEMATICS), PATTERN RECOGNITION, BAYES THEOREM, NUMERICAL METHODS AND PROCEDURES, SPARSE MATRIX, SPECIAL FUNCTIONS(MATHEMATICS), VARIATIONAL PRINCIPLES, CUBIC SPLINE TECHNIQUE.

Subject Categories : Numerical Mathematics
      Cybernetics

Distribution Statement : APPROVED FOR PUBLIC RELEASE