The geometry of Sloppiness

Keywords: sloppiness, structural identifiability, inference, metric geometry

Abstract

The use of mathematical models in the sciences often requires the estimation of unknown parameter values from data. Sloppiness provides information about the uncertainty of this task. In this paper, we develop a precise mathematical foundation for sloppiness and define rigorously its key concepts, such as `model manifold', in relation to concepts of structural identifiability. We redefine sloppiness conceptually as a comparison between the premetric on parameter space induced by measurement noise  and a reference metric. This opens up the possibility of alternative quantification of sloppiness, beyond the standard use of the Fisher Information Matrix, which assumes that parameter space is equipped with the usual Euclidean and the measurement error is infinitesimal. Applications include parametric statistical models, explicit time dependent models, and ordinary differential equation models.

Author Biographies

Emilie Dufresne, University of Nottingham
Anne McLaren Fellow, School of Mathematical Sciences
Heather A Harrington, University of Oxford
Royal Society University Research Fellow, Mathematical Institute
Dhruva V Raman, University of Cambridge
Department of engineering

References

[1] Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191 of Translations of Mathematical Monographs. American Mathematical Society, Providence, RI; Oxford University Press, Oxford, 2000. Translated from the 1993 Japanese original by Daishi Harada.

[2] Carlos Améndola, Jean-Charles Faugère, and Bernd Sturmfels. Moment Varieties of Gaussian Mixtures. J. Algebr. Stat., 7(1):14–28, 2016. URL: http://dx.doi.org/10.18409/jas.v7i1. 42.

[3] Milena Anguelova. Observability and identifiability of nonlinear systems with applications in biology. PhD thesis, Chalmers University of Technology, 2007. URL: http://www.gu.se/english/research/publication?publicationId=57985.

[4] Joshua F. Apgar, David K. Witmer, Forest M. White, and Bruce Tidor. Sloppy models, parameter uncertainty, and the role of experimental design. Molecular Biosystems, 6(10):1890–1900, oct 2010. URL: http://dx.doi.org/10.1039/b918098b.

[5] Giuseppina Bellu, Maria Pia Saccomani, Stefania Audoly, and Leontina DAngi`o. Daisy: A new software tool to test global identifiability of biological and physiological systems. Computer Methods and Programs in Biomedicine, 88(1):52–61, 2007. URL: http://dx.doi.org/10.1016/j.cmpb.2007.07.002.

[6] Louis J. Billera, Susan P. Holmes, and Karen Vogtmann. Geometry of the space of phylogenetic trees. Advances in Applied Mathematics, 27(4):733–767, 2001. URL: http://dx.doi. org/10.1006/aama.2001.0759.

[7] François Boulier. Differential elimination and biological modelling. In Gröbner bases in symbolic analysis, volume 2 of Radon Series on Computational and Applied Mathematics, pages 109–137. Walter de Gruyter, Berlin, 2007. URL: https://hal.archives-ouvertes.fr//hal.00139364/document.

[8] Kevin S. Brown and James P. Sethna. Statistical mechanical approaches to models with many poorly known parameters. Physical Review E, 68(2):021904, 2003. URL: http://dx.doi.org/10.1103/PhysRevE.68.021904.

[9] George Casella and Roger L. Berger. Statistical Inference. Duxbury, 2nd edition, 2002. URL: http://statistics.columbian.gwu.edu/sites/statistics.columbian. gwu.edu/files/downloads/Syllabus6202- Spring2013- Li.pdf.

[10] E. A. Catchpole and B. J. T. Morgan. Detecting parameter redundancy. Biometrika, 84(1):187–196, 1997. URL: http://dx.doi.org/10.1093/biomet/84.1.187.

[11] Oana-Teodora Chis, Alejandro F. Villaverde, Julio R. Banga, and Eva Balsa-Canto. On the relationship between sloppiness and identifiability. Mathematical Biosciences, 282(Complete):147–161, 2016. URL: http://dx.doi.org/10.1016/j.mbs.2016.10.009.

[12] Gilles Clermont and Sven Zenker. The inverse problem in mathematical biology. Mathematical Biosciences, 260:11–15, 2015. URL: http://dx.doi.org/10.1016/j.mbs.2014.09.001.

[13] Claudio Cobelli and Joseph J. DiStefano III. Parameter and structural identifiability concepts and ambiguities: a critical review and analysis. American Journal of Physiology, 239(1):R7– R24, 1980. URL: http://ajpregu.physiology.org/content/ajpregu/239/1/R7.full.pdf.

[14] Thomas M. Cover and Joy A. Thomas. Elements of information theory. Wiley, 2012.

[15] Bryan C Daniels, Yan-Jiun Chen, James P Sethna, Ryan N Gutenkunst, and Christopher R Myers. Sloppiness, robustness, and evolvability in systems biology. Current Opinion in Biotechnology, 19(4):389 – 395, 2008. URL: http://dx.doi.org/10.1016/j.copbio.2008. 06.008.

[16] John Duchi. Derivations for linear algebra and optimization. URL: http://web.stanford.edu/~jduchi/projects/general_notes.pdf, 2007.

[17] Kamil Erguler and Michael P H Stumpf. Practical limits for reverse engineering of dynamical systems: a statistical analysis of sensitivity and parameter inferability in systems biology models. Molecular BioSystems, 7(5):1593–1602, May 2011. URL: http://dx.doi.org/10. 1039/C0MB00107D.

[18] R. A. Fisher. Theory of statistical estimation. Mathematical Proceedings of the Cam- bridge Philosophical Society, 22(5):700?725, 1925. URL: http://dx.doi.org/10.1017/
S0305004100009580.

[19] Matan Gavish and David L. Donoho. The optimal hard threshold for singular values is 4/√3. IEEE Trans. Inform. Theory, 60(8):5040–5053, 2014. URL: http://dx.doi.org/10.1109/TIT.2014.2323359.

[20] Elizabeth Gross, Brent Davis, Kenneth L. Ho, Daniel J. Bates, and Heather A. Harrington. Numerical algebraic geometry for model selection and its application to the life sciences. J. R. Soc. Interface, 13, 2016. URL: http://dx.doi.org/10.1098/rsif.2016.0256.

[21] Ryan N. Gutenkunst, Jordan C. Atlas, Fergal P. Casey, Brian C. Daniels, Robert S. Kuczenski, Joshua J. Waterfall, Chris R. Myers, and James P. Sethna. Sloppycell. URL: http://sloppycell.sourceforge.net, 2007.

[22] Ryan N. Gutenkunst, Joshua J. Waterfall, Fergal P. Casey, Kevin S. Brown, Christopher R. Myers, and James P. Sethna. Universally Sloppy Parameter Sensitivities in Systems Biology Models. PLoS Computational Biology, 3(10):e189–1878, oct 2007. URL: http://dx.doi.org/10.1371/journal.pcbi.0030189.

[23] Keegan E. Hines, Thomas R. Middendorf, and Richard W. Aldrich. Determination of pa-
rameter identifiability in nonlinear biophysical models: A Bayesian approach. The Journal of
general physiology, 143(3):401–16, 2014. URL:http://dx.doi.org/10.1085/jgp.201311116.

[24] Serkan Ho ̧sten, Amit Khetan, and Bernd Sturmfels. Solving the likelihood equations. Foun- dations of Computational Mathematics, 5(4):389–407, 2005. URL: http://dx.doi.org/10.1007/s10208- 004- 0156- 8.

[25] Joseph DiStefano III. Dynamic Systems Biology Modeling and Simulation, 1st Edition. Elsevier: Academic Press, Amsterdam, 2013.

[26] M. Joshi, A. Seidel-Morgenstern, and A. Kremling. Exploiting the bootstrap method for quantifying parameter confidence intervals in dynamical systems. Metabolic Engineering, 8(5):447–455, 2006. URL: http://dx.doi.org/10.1016/j.ymben.2006.04.003.

[27] Johan Karlsson, Milena Anguelova, and Mats Jirstrand. An efficient method for structural identifiability analysis of large dynamic systems*. IFAC Proceedings Volumes, 45(16):941–946, 2012. URL: http://dx.doi.org/10.3182/20120711-3-BE-2027.00381.

[28] Gregor Kemper. Separating invariants. Journal of Symbolic Computation, 44(9):1212–1222, 2009. URL: http://dx.doi.org/10.1016/j.jsc.2008.02.012.

[29] S. Kullback and R.A. Leibler. On information and sufficiency. The annals of mathematical statistics, 1951. URL: http://www.jstor.org/stable/2236703.

[30] Daniel Lazard. Injectivity of real rational mappings: the case of a mixture of two Gaussian laws. Math. Comput. Simulation, 67(1-2):67–84, 2004. URL: http://dx.doi.org/10.1016/j.matcom.2004.05.009.

[31] Lennart Ljung. System identification: theory for the user. Prentice Hall Information and System Sciences Series. Prentice Hall, Inc., Englewood Cliffs, NJ, 1987.

[32] Lennart Ljung and Torkel Glad. On global identifiability for arbitrary model parametrizations. Automatica, 30(2):265 – 276, 1994. URL: http://dx.doi.org/10.1016/0005-1098(94)90029- 9.

[33] Brian K. Mannakee, Aaron P. Ragsdale, Mark K. Transtrum, and Ryan N. Gutenkunst.
Sloppiness and the geometry of parameter space. In Liesbet Geris and David Gomez- Cabrero, editors, Uncertainty in Biology: A Computational Modeling Approach, pages 271– 299. Springer International Publishing, Cham, 2016. URL: http://dx.doi.org/10.1007/ 978- 3- 319- 21296- 8_11.

[34] Gabriella Margaria, Eva Riccomagno, Michael J. Chappell, and Henry P. Wynn. Differential algebra methods for the study of the structural identifiability of rational function state- space models in the biosciences. Mathematical Biosciences, 174(1):1–26, 2001. URL: http://dx.doi.org/10.1016/S0025- 5564(01)00079-7.

[35] Nicolette Meshkat, Chris Anderson, and Joseph J Distefano. Finding identifiable parameter combinations in nonlinear ODE models and the rational reparameterization of their input- output equations. Mathematical Biosciences, 233(1):19–31, sep 2011. URL: http://dx.doi.org/10.1016/j.mbs.2011.06.001.

[36] Nicolette Meshkat, Marisa Eisenberg, and Joseph J. DiStefano, III. An algorithm for finding globally identifiable parameter combinations of nonlinear ODE models using Gröbner bases. Mathematical Biosciences, 222(2):61–72, 2009. URL: http://dx.doi.org/10.1016/j.mbs. 2009.08.010.

[37] Eva Balsa-Canto Oana-Teodora Chis, Julio R. Banga. Structural identifiability of systems biology models: A critical comparison of methods. PLoS ONE, 6(11), 2011. URL: http://dx.doi.org/10.1371/journal.pone.0027755.

[38] Franc ̧ois. Ollivier. Le Problème de l’Identifiabilité Structurelle Globale: Étude Théorique, Méthodes Effectives et Bornes de Complexité. PhD thesis, École Polytéchnique, 1990. URL: www.theses.fr/1990EPXX0009.

[39] Dhruva V. Raman, James Anderson, and Antonis Papachristodoulou. On the performance of nonlinear dynamical systems under parameter perturbation. Automatica, 63:265–273, 2016. URL: http://dx.doi.org/10.1016/j.automatica.2015.10.009.

[40] Dhruva V. Raman, James Anderson, and Antonis Papachristodoulou. Delineating parameter unidentifiabilities in complex models. Phys. Rev. E, 95:032314, Mar 2017. URL: http://link.aps.org/doi/10.1103/PhysRevE.95.032314.

[41] C Radhakrishna Rao. Information and accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc, 37(3):81–91, 1945.

[42] A. Raue, C. Kreutz, T. Maiwald, J. Bachmann, M. Schilling, U. Klingmu ̈ller, and J. Timmer. Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics, 25(15):1923–9, 2009. URL: https://doi.org/10.1093/bioinformatics/btp358.

[43] Andreas Raue, Johan Karlsson, Maria Pia Saccomani, Mats Jirstrand, and Jens Tim- mer. Comparison of approaches for parameter identifiability analysis of biological systems. Bioinformatics (Oxford, England), 30(10):1440–8, 2014. URL: http://dx.doi.org/10.1093/ bioinformatics/btu006.

[44] Thomas J. Rothenberg. Identification in parametric models. Econometrica, 39(3):577–591, 1971. URL: http://www.jstor.org/stable/1913267.

[45] Alexandre Sedoglavic. A probabilistic algorithm to test local algebraic observability in polynomial time. In Proceedings of the International Symposium on Symbolic and Algebraic Computation. ACM, 2001. URL: http://dx.doi.org/10.1145/384101.384143.

[46] Jim Sethna. Fitting Polynomials: Where is sloppiness from? Webpage last modified June 11, 2008, http://www.lassp.cornell.edu/sethna/Sloppy/FittingPolynomials.html.

[47] Daniel Silk, Paul D W Kirk, Christopher P Barnes, Tina Toni, and Michael P H Stumpf. Model selection in systems biology depends on experimental design. PLOS Computational Biology, 10(6), 2014. URL: http://dx.doi.org/10.1371/journal.pcbi.1003650.

[48] E. D. Sontag. For differential equations with r parameters, 2r + 1 experiments are enough for identification. Journal of Nonlinear Science, 12(6):553–583, 2002. URL: http://dx.doi. org/10.1007/s00332-002-0506-0.

[49] Seth Sullivant. Algebraic Statistics. In preparation. URL: http://www4.ncsu.edu/~smsulli2/Pubs/ asbook.html.

[50] Christian Tönsing, Jens Timmer, and Clemens Kreutz. Cause and cure of sloppiness in ordinary differential equation models. Phys. Rev. E, 90:023303, Aug 2014. URL: http://link.aps.org/doi/10.1103/PhysRevE.90.023303.

[51] Mark K. Transtrum, Benjamin B. Machta, Kevin S. Brown, Bryan C. Daniels, Christo- pher R. Myers, and James P. Sethna. Perspective: Sloppiness and emergent theories in physics, biology, and beyond. The Journal of Chemical Physics, 143(1):010901, jul 2015. URL: http://dx.doi.org/10.1063/1.4923066.

[52] Mark K. Transtrum, Benjamin B. Machta, and James P. Sethna. Why are nonlinear fits to data so challenging? Physical Review Letters, 104(6):060201, feb 2010. URL: https://doi.org/10.1103/PhysRevLett.104.060201.

[53] Mark K. Transtrum, Benjamin B. Machta, and James P. Sethna. Geometry of nonlinear least squares with applications to sloppy models and optimization. Physical Review E, 83(3):036701, mar 2011. URL: https://doi.org/10.1103/PhysRevE.83.036701.

[54] Mark K. Transtrum and Peng Qiu. Model reduction by manifold boundaries. Physical Review Letters, 113(9):098701, aug 2014. URL: https://doi.org/10.1103/PhysRevLett.113.098701.

[55] Sandor Vajda, Herschel Rabitz, Eric Walter, and Yves Lecourtier. Qualitative and quantitative identifiability analysis of nonlinear chemical kinetic models. Chemical Engineering Communications, 83(1):191–219, 1989. URL: http://dx.doi.org/10.1080/00986448908940662.

[56] Michele Vallisneri. Use and abuse of the fisher information matrix in the assessment of gravitational-wave parameter-estimation prospects. Phys. Rev. D, 77:042001, Feb 2008. URL: http://link.aps.org/doi/10.1103/PhysRevD.77.042001.

[57] Joshua J. Waterfall, Fergal P. Casey, Ryan N. Gutenkunst, Kevin S. Brown, Christopher R. Myers, Piet W. Brouwer, Veit Elser, and James P. Sethna. The sloppy model universality class and the Vandermonde matrix. Physical Review Letters, 97(15):150601, 2006. URL: https: //doi.org/10.1103/PhysRevLett.97.150601.
Published
2018-09-24
Section
Articles