Accession Number : ADA192228

Title :   Asymptotically Efficient Adaptive Allocation Schemes for Controlled Markov Chains: Finite Parameter Space.

Descriptive Note : Technical rept.,

Corporate Author : MICHIGAN UNIV ANN ARBOR COMMUNICATIONS AND SIGNAL PROCESSING LAB

Personal Author(s) : Agrawal, Rajeev ; Teneketzis, Demosthenis ; Anantharam, Venkatachalam

PDF Url : ADA192228

Report Date : Feb 1988

Pagination or Media Count : 38

Abstract : Consider a controlled Markov chain whose transition probabilities and initial distribution are parametrized by an unknown parameter Theta belonging to some known parameter space Theta. There is a one-step reward associated with each pair of control and the following state of the process. The objective is to maximize the expected value of the sum of one step rewards over an infinite horizon. By introducing the Loss associated with a control scheme, the problem is equivalent to minimizing this Loss. Define a uniformly good adaptive control schemes and restrict attention to these schemes. A lower bound is developed on the Loss associated with any uniformly good control scheme. Finally, an adaptive control scheme is constructed whose Loss equals the lower bound, and is therefore optimal. Keywords: Adaptive control scheme; Controlled Markov chain; Asymptotically optimal.

Descriptors :   *ADAPTIVE CONTROL SYSTEMS, *MARKOV PROCESSES, ADAPTIVE SYSTEMS, ALLOCATIONS, CONTROL, EFFICIENCY, PROBABILITY, TRANSITIONS

Subject Categories : Statistics and Probability

Distribution Statement : APPROVED FOR PUBLIC RELEASE