A Causal Network-Based Markov Decision Process Model for Intervention Planning

Abstract

Patients suffering from chronic diseases often receive interventions after they are discharged from hospitals for their health management. The intervention decisions can be made sequentially during the course of disease progression and treatment process, and timely and proper interventions can lead to better health condition and quality of life and reduce the rate of re-hospitalization. Moreover, to determine effective intervention plans, personalized care should be considered. However, in many cases, intervention plans are not tailored to patients' health conditions or dynamically adjusted based on their status changes due to the lack of adequate analytical methods. To address this issue, we introduce a causal Bayesian network-based Markov decision process (CNMDP) model to dynamically evaluate the health status and effectiveness of interventions for each patient and update the corresponding intervention strategy. In such a model, a causal Bayesian network is integrated in order to predict the effect of interventions in each patient's health status. The resulting risk at each decision point is considered in a Markov decision process model by examining diverse factors of a patient to make a personalized optimal decision on intervention planning to minimize the risk of health deterioration or readmission. To illustrate the applicability of the model and develop a specific formulation, an intervention decision model for chronic obstructive pulmonary disease (COPD) patients is presented using the proposed method. Such a model can be applied to other diseases as well to design personalized, prompt, and effective intervention plans.