Linear–quadratic stochastic leader–follower differential games for Markov jump-diffusion models

Abstract

This paper considers the linear–quadratic (LQ) leader–follower Stackelberg differential game for Markov jump-diffusion stochastic differential equations (SDEs). We first obtain the open-loop type optimal solutions for the leader and the follower by establishing the general stochastic maximum principle for (indefinite) LQ control with Markovian jumps. Then we obtain the state-feedback representation of the open-loop type optimal solutions for the leader and the follower in terms of the coupled Riccati differential equations (CRDEs) by generalizing the classical Four-Step Scheme to the case with Markovian jumps. Unlike the existing literature, the Four-Step Scheme in our paper is not symmetric due to the presence of the nonsymmetric quadratic variation induced by the Markov chain. We develop the decoupling approach to find the simplified expression of the corresponding quadratic variation. Under the well-posedness of the CRDEs, the state-feedback type optimal solutions for the leader and the follower constitute the Stackelberg equilibrium. We also show the well-posedness of the CRDEs for the follower under suitable assumptions of the coefficients. Finally, we demonstrate the well-posedness of the CRDEs for the leader and the follower via numerical simulations for both indefinite and definite cost parameter cases.