Markov Decision Processes are the major model of controlled stochastic processes in discrete time. Value iteration (VI) is one of the major methods for finding optimal policies. For each discount factor, starting from a finite number of iterations, which is called the turnpike integer, value iteration algorithms always generate decision rules which are deterministic optimal policies for the infinite-horizon problems. This fact justifies the rolling horizon approach for computing infinite-horizon optimal policies by conducting a finite number of value iterations. In this talk, we will first discuss the complexity of using VI to approximately solve MDPs, and then introduce properties of turnpike integers and provide their upper bounds.

We will present recent results obtained with J. Hounie on the removability sets of singularities for bounded solutions and the propagation of zeros across rough (nondifferentiable) boundaries for solutions of systems of complex vector fields including CR vector fields.