Using multi-step proposal distribution for improved MCMC convergence in Bayesian network structure learning
Tóm tắt
Bayesian networks have become popular for modeling probabilistic relationships between entities. As their structure can also be given a causal interpretation about the studied system, they can be used to learn, for example, regulatory relationships of genes or proteins in biological networks and pathways. Inference of the Bayesian network structure is complicated by the size of the model structure space, necessitating the use of optimization methods or sampling techniques, such Markov Chain Monte Carlo (MCMC) methods. However, convergence of MCMC chains is in many cases slow and can become even a harder issue as the dataset size grows. We show here how to improve convergence in the Bayesian network structure space by using an adjustable proposal distribution with the possibility to propose a wide range of steps in the structure space, and demonstrate improved network structure inference by analyzing phosphoprotein data from the human primary T cell signaling network.
Tài liệu tham khảo
N Friedman, M Linial, I Nachman, D Pe’er, Using Bayesian networks to analyze expression data. J. Comput. Biol. 7(3–4), 601–620 (2000).
N Friedman, Inferring cellular networks using probabilistic graphical models. Science. 303(5659), 799–805 (2004).
AJ Hartemink, DK Gifford, TS Jaakkola, RA Young, in The Pacific Symposium on Biocomputing (PSB01). Using graphical models and genomic expression data to statistically validate models of genetic regulatory networks (Hawaii, 2001), pp. 422–33.
S Imoto, S Kim, T Goto, S Miyano, S Aburatani, K Tashiro, S Kuhara, Bayesian network and nonparametric heteroscedastic regression for nonlinear modeling of genetic network. J. Bioinforma. Comput. Biol. 1(2), 231–52 (2003).
K Sachs, O Perez, D Pe’er, DA Lauffenburger, GP Nolan, Causal protein-signaling networks derived from multiparameter single-cell data. Science. 308(5721), 523–529 (2005).
D Nikovski, Constructing Bayesian networks for medical diagnosis from incomplete and partially correct statistics. IEEE Trans. Knowl. Data Eng. 12, 509–516 (2000).
AV Nefian, L Liang, X Pi, X Liu, K Murphy, Dynamic Bayesian networks for audio-visual speech recognition. EURASIP J. Appl. Signal Process. 11(4), 1–15 (2002).
P Weber, G Medina-Oliva, C Simon, B Iung, Overview on Bayesian networks applications for dependability, risk analysis and maintenance areas. Eng. Appl. Artif. Intell. 25(4), 671–682 (2012).
O Pourret, P Naïm, B Marcot (eds.), Bayesian Networks: A Practical Guide to Applications (Wiley, Chichester, UK, 2008).
J Pearl, Causality: Models, Reasoning and Inference, 2nd edn. (Cambridge University Press, New York, NY, USA, 2009).
M Grzegorczyk, D Husmeier, Improving the structure MCMC sampler for Bayesian networks by introducing a new edge reversal move. Mach. Learn. 71(2–3), 265–305 (2008).
N Friedman, D Koller, Being Bayesian about network structure – a Bayesian approach to structure discovery in Bayesian networks. Mach. Learn. 50(1–2), 95–125 (2003).
B Ellis, WH Wong, Learning causal Bayesian network structures from experimental data. J. Am. Stat. Assoc. 103(482), 778–789 (2008).
T Niinimäki, M Koivisto, in Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Annealed importance sampling for structure learning in Bayesian networks (AAAI PressBeijing, China, 2013), pp. 1579–1585.
M Koivisto, K Sood, Exact Bayesian structure discovery in Bayesian networks. J. Mach. Learn. Res. 5, 549–573 (2004).
D Eaton, K Murphy, in UAI 2007, Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence. Bayesian structure learning using dynamic programming and MCMC (Morgan KaufmannVancouver, 2007), pp. 101–108.
J Pearl, in Proceedings of the 7th Conference of the Cognitive Science Society. Bayesian networks: a model of self-activated memory for evidential reasoning (University of CaliforniaIrvine, 1985), pp. 329–334.
GF Cooper, E Herskovits, A Bayesian method for the induction of probabilistic networks from data. Mach. learn. 9(4), 309–347 (1992).
D Heckerman, D Geiger, DM Chickering, Learning Bayesian networks: the combination of knowledge and statistical data. Mach. learn. 20(3), 197–243 (1995).
D Geiger, D Heckerman, A characterization of the bivariate Wishart distribution. Probab. Math. Stat. 18(1), 119–131 (1998).
D Madigan, J York, Bayesian graphical models for discrete data. Int. Stat. Rev. 63(2), 215–232 (1995).
MK Cowles, BP Carlin, Markov chain Monte Carlo convergence diagnostics: a comparative review. J. Am. Stat. Assoc. 91(434), 883–904 (1996).
A Gelman, in Practical Markov Chain Monte Carlo, ed. by W Gilks, S Richardson, and D Spiegelhalter. Inference and monitoring convergence (Chapman and HallLondon, 1996), pp. 131–143.
R Castelo, T Koc̆ka, On inclusion-driven learning of Bayesian networks. J. Mach. Learn. Res. 4, 527–574 (2003).
AW Moore, W-K Wong, in Proceedings of the Twentieth International Conference on Machine Learning. Optimal reinsertion: a new search operator for accelerated and more accurate Bayesian network structure learning (Washington D.C., US, 2003), pp. 552–559.
DM Chickering, Learning equivalence classes of Bayesian-network structures. J. Mach. Learn. Res. 2, 445–498 (2002).
D Madigan, SA Andersson, MD Perlman, CT Volinsky, Bayesian model averaging and model selection for Markov equivalence classes of acyclic digraphs. Commun. Stat. Theory and Methods. 25, 2493–2519 (1996).
P Giudici, R Castelo, Improving Markov chain Monte Carlo model search for data mining. Mach. Learn. 50(1–2), 127–158 (2003).
H Haario, M Laine, A Mira, E Saksman, DRAM: Efficient adaptive MCMCDRAM: Efficient adaptive MCMC. Stat. Comput. 16, 339–354 (2006).
IA Beinlich, HJ Suermondt, RM Chavez, GF Cooper, in Second European Conference on Artificial Intelligence in Medicine, 38. The ALARM monitoring system: a case study with two probabilistic inference techniques for belief networks (SpringerLondon, Great Britain, 1989), pp. 247–256.