Accession Number : AD0706910

Title :   MATHEMATICAL PROGRAMMING AND THE CONTROL OF MARKOV CHAINS,

Corporate Author : BROWN UNIV PROVIDENCE R I DIV OF APPLIED MATHEMATICS

Personal Author(s) : Kushner,H. J. ; Kleinman,A. J.

Report Date : MAY 1970

Pagination or Media Count : 45

Abstract : Linear programming versions of some control problems on Markov chains are derived, and are studied under conditions which occur in typical problems which arise by discretizing continuous time and state systems, or in discrete state systems. Control interpretations of the dual variables and simplex multipliers are given. The formulation allows the treatment of 'state space' like constraints which cannot be handled conveniently with dynamic programming. The relation between dynamic programming on Markov chains, and the deterministic discrete maximum is explored, and some insight is obtained into the problem of singular stochastic controls (with respect to a stochastic maximum principle). (Author)

Descriptors :   (*CONTROL SYSTEMS, MATHEMATICAL MODELS), (*STOCHASTIC PROCESSES, *MATHEMATICAL PROGRAMMING), PROBABILITY, DYNAMIC PROGRAMMING, SIMPLEX METHOD, ITERATIONS, ALGORITHMS

Subject Categories : Statistics and Probability

Distribution Statement : APPROVED FOR PUBLIC RELEASE