On the Controller Synthesis for Finite-State Markov Decision Processes

Investor logo


This publication doesn't include Institute of Computer Science. It includes Faculty of Informatics. Official publication website can be found on muni.cz.


Year of publication 2005
Type Article in Proceedings
Conference 25th International Conference on Foundations of Software Technology and Theoretical Computer Science
MU Faculty or unit

Faculty of Informatics

Field Informatics
Keywords Controller Synthesis; Markov Decision Processes
Description We study the problem of effective controller synthesis for finite-state Markov decision processes (MDPs) and the class of properties definable in the logic PCTL extended with long-run average propositions. We show that the existence of such a controller is decidable, and we give an algorithm which computes the controller if it exists. We also address the issue of ``controller robustness'', i.e., the problem whether there is a controller which still guarantees the satisfaction of a given property when the probabilities in the considered MDP slightly deviate from their original values. From a practical point of view, this is an important aspect since the probabilities are often determined empirically and hence they are inherently imprecise. We show that the existence of robust controllers is also decidable, and that such controllers are effectively computable if they exist. .
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info