Accession Number : AD0720299

Title :   Undiscounted Markov Renewal Programming via Modified Successive Approximations.

Descriptive Note : Research rept.,

Corporate Author : CARNEGIE-MELLON UNIV PITTSBURGH PA MANAGEMENT SCIENCES RESEARCH GROUP

Personal Author(s) : Morton,Thomas E.

Report Date : JUN 1970

Pagination or Media Count : 16

Abstract : An efficient class of procedures is described for finding a solution to the functional equations of undiscounted Markov renewal programming. First, for the special case of a single possible policy, the problem is proved equivalent to solving two related ordinary Markov chain problems. This leads to an algorithm for the general problem whose exact form depends on the specification of a decision rule for alternation of two types of iterations. At one extreme the technique is exactly 'policy iteration' with iterative techniques replacing solution of N equations for each improved policy. At the other extreme, the algorithm becomes essentially value iteration, generalizing the method of successive approximations proposed by D. J. White for Markovian decision processes. The latter version of the technique is related to another generalization being currently proposed by Paul J. Schweitzer; the methods being proposed here, however, do not deteriorate when the minimum transition time between states becomes very small. (Author)

Descriptors :   (*DECISION THEORY, *MATHEMATICAL PROGRAMMING), STOCHASTIC PROCESSES, RANDOM VARIABLES, MATRICES(MATHEMATICS), ITERATIONS

Subject Categories : Operations Research

Distribution Statement : APPROVED FOR PUBLIC RELEASE