A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton–Jacobi Integrodifferential Systems on Networks
Tóm tắt
We study optimal control problems in infinite horizon whxen the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (corresponding to a toy traffic model). We adapt the results in Soner (SIAM J Control Optim 24(6):1110–1122, 1986) to prove the regularity of the value function and the dynamic programming principle. Extending the networks and Krylov’s “shaking the coefficients” method, we prove that the value function can be seen as the solution to a linearized optimization problem set on a convenient set of probability measures. The approach relies entirely on viscosity arguments. As a by-product, the dual formulation guarantees that the value function is the pointwise supremum over regular subsolutions of the associated Hamilton–Jacobi integrodifferential system. This ensures that the value function satisfies Perron’s preconization for the (unique) candidate to viscosity solution.
Tài liệu tham khảo
Achdou, Y., Camilli, F., Cutrì, A., Tchou, N.: Hamilton–Jacobi equations constrained on networks. Nonlinear Differ. Equ. Appl. NoDEA 20(3), 413–445 (2013)
Achdou, Y., Oudet, S., Tchou, N.: Hamilton–Jacobi equations for optimal control on junctions and networks. ESAIM Control Optim. Calc. Var. 21(3), 876–899 (2015)
Bardet, J.-B., Christen, A., Guillin, A., Malrieu, F., Zitt, P.-A.: Total variation estimates for the TCP process. Electron. J. Probab. 18(10), 1–21 (2013)
Barles, G., Jakobsen, E.R.: On the convergence rate of approximation schemes for Hamilton–Jacobi–Bellman equations. ESAIM Math. Model. Numer. Anal. 36(1), 33–54 (2002)
Barles, G., Briani, A., Chasseigne, E.: A Bellman approach for two-domains optimal control problems in R–N. ESAIM Control Optim. Calc. Var. 19, 710–739 (2013)
Barles, G., Briani, A., Chasseigne, E.: A Bellman approach for regional optimal control problems in \(\mathbb{R}^N\). SIAM J. Control Optim. 52(3), 1712–1744 (2014)
Barnard, R.C., Wolenski, P.R.: Flow invariance on stratified domains. Set-Valued Var. Anal. 21(2), 377–403 (2013)
Biswas, I.H., Jakobsen, E.R., Karlsen, K.H.: Viscosity solutions for a system of integro-PDEs and connections to optimal switching and control of jump-diffusion processes. Appl. Math. Optim. 62(1), 47–80 (2010)
Boxma, O., Kaspi, H., Kella, O., Perry, D.: On/off storage systems with state-dependent input, output, and switching rates. Probab. Eng. Inf. Sci. 19, 1–14 (2005)
Bressan, A., Hong, Y.: Optimal control problems on stratified domains. Netw. Heterog. Media 2(2), 313–331 (2007)
Camilli, F., Marchi, C.: A comparison among various notions of viscosity solution for Hamilton–Jacobi equations on networks. J. Math. Anal. Appl. 407(1), 112–118 (2013)
Cook, D.L., Gerber, A.N., Tapscott, S.J.: Modelling stochastic gene expression: implications for haploinsufficiency. Proc. Natl. Acad. Sci. USA 95, 15641–15646 (1998)
Crudu, A., Debussche, A., Radulescu, O.: Hybrid stochastic simplifications for multiscale gene networks. BMC Syst. Biol. 3, 89 (2009)
Crudu, A., Debussche, A., Muller, A., Radulescu, O.: Convergence of stochastic gene networks to hybrid piecewise deterministic processes. Ann. Appl. Probab. 22(5), 1822–1859 (2012)
Davis, M.H.A.: Piecewise-deterministic Markov-processes—a general-class of non-diffusion stochastic-models. J. R. Stat. Soc. Ser. B Methodol. 46(3), 353–388 (1984)
Davis, M.H.A.: Control of piecewise-deterministic processes via discrete-time dynamic-programming. Stochastic Differential Systems. Lecture Notes in Control and Information Sciences, vol. 78, pp. 140–150. Springer, Berlin (1986)
Davis, M.H.A.: Markov Models and Optimization. Monographs on Statistics and Applied Probability, vol. 49. Chapman & Hall, London (1993)
Dufour, F., Stockbridge, R.H.: On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time. Stochastics 84(1), 55–78 (2012)
Dufour, F., Dutuit, Y., Gonzalez, K., Zhang, H.: Piecewise deterministic Markov processes and dynamic reliability. Proc. Inst. Mech. Eng. Part O J. Risk Reliab. 222, 545–551 (2008)
Finlay, L., Gaitsgory, V., Lebedev, I.: Linear programming solutions of periodic optimization problems: approximation of the optimal control. J. Ind. Manag. Optim. 3(2), 399–413 (2007)
Frankowska, H., Plaskacz, S.: Semicontinuous solutions of Hamilton–Jacobi–Bellman equations with state constraints. In: Differential Inclusions and Optimal Control. Lecture Notes in Nonlinear Analysis, vol. 2, pp. 145–161 (1998)
Frankowska, H., Vinter, R.: Existence of neighbouring trajectorie: applications to dynamic programming for state constraints optimal control problems. J. Optim. Theory Appl. 104(1), 20–40 (2000)
Goreac, D.: Viability, invariance and reachability for controlled piecewise deterministic Markov processes associated to gene networks. ESAIM Control Optim. Calc. Var. 18(2), 401–426 (2012)
Goreac, D., Serea, O.-S.: Linearization techniques for controlled piecewise deterministic Markov processes: application to Zubov’s method. Appl. Math. Optim. 66, 209–238 (2012)
Goreac, D., Serea, O.-S.: Optimality issues for a class of controlled singularly perturbed stochastic systems. J. Optim. Theory Appl. 1–31 (2015). doi:10.1007/s10957-015-0738-4
Graham, C., Robert, P.: Interacting multi-class transmissions in large stochastic networks. Ann. Appl. Probab. 19(6), 2334–2361 (2009)
Imbert, C., Monneau, R.: Flux-Limited Solutions for quasi-convex Hamilton–Jacobi Equations on Networks (2014) (submitted)
Imbert, C., Monneau, R., Zidani, H.: A Hamilton–Jacobi approach to junction problems and application to traffic flows. ESAIM Control Optim. Calc. Var. 19, 129–166 (2013)
Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab. Theory Relat. Fields 117(1), 1–16 (2000)
Plaskacz, S., Quincampoix, M.: Discontinuous Mayer control problem under state-constraints. Topol. Methods Nonlinear Anal. 15, 91–100 (2000)
Rao, Z., Siconolfi, A., Zidani, H.: Transmission conditions on interfaces for Hamilton–Jacobi–Bellman equations. J. Differ. Equ. 257(11), 3978–4014 (2014)
Rolski, T., Schmidli, H., Schmidt, V., Teugels, J.: Stochastic Processes for Insurance and Finance. Wiley Series in Probability and Statistics, vol. 505. Wiley, Chichester (2009)
Schieborn, D., Camilli, F.: Viscosity solutions of Eikonal equations on topological networks. Calc. Var. Partial Differ. Equ. 46(3–4), 671–686 (2013)
Soner, H.M.: Optimal control with state-space constraint. I. SIAM J. Control Optim. 24(6), 552–561 (1986)
Soner, H.M.: Optimal control with state-space constraint. II. SIAM J. Control Optim. 24(6), 1110–1122 (1986)
Wainrib, G., Michèle, T., Pakdaman, K.: Intrinsic variability of latency to first-spike. Biol. Cybern. 103(1), 43–56 (2010)