Assessing convergence in global sensitivity analysis: a review of methods for assessing and monitoring convergence
Article Full Text (PDF)

Keywords

global sensitivity analysis
convergence
good modeling practice

How to Cite

Sun, X., Jakeman, A. J., Croke, B. F., Roberts, S. G., & Jakeman, J. D. (2024). Assessing convergence in global sensitivity analysis: a review of methods for assessing and monitoring convergence. Socio-Environmental Systems Modelling, 6, 18678. https://doi.org/10.18174/sesmo.18678

Abstract

In global sensitivity analysis (GSA) of a model, a proper convergence analysis of metrics is essential for ensuring a level of confidence or trustworthiness in sensitivity results obtained, yet is somewhat deficient in practice. The level of confidence in sensitivity measures, particularly in relation to their influence and support for decisions from scientific, social and policy perspectives, is heavily reliant on the convergence of GSA. We review the literature and summarize the available methods for monitoring and assessing convergence of sensitivity measures based on application purposes. The aim is to expose the various choices for convergence assessment and encourage further testing of available methods to clarify their level of robustness. Furthermore, the review identifies a pressing need for comparative studies on convergence assessment methods to establish a clear hierarchy of effectiveness and encourages the adoption of systematic approaches for enhanced robustness in sensitivity analysis.

Article Full Text (PDF)

References

Akomeah, E., Lindenschmidt, K. E., & Chapra, S. C. (2019). Comparison of aquatic ecosystem functioning between eutrophic and hypereutrophic cold-region river-lake systems. Ecological Modelling, 393, 25–36. https://doi.org/10.1016/j.ecolmodel.2018.12.004

Anstett-Collin, F., Goffart, J., Mara, T., & Denis-Vidal, L. (2015). Sensitivity analysis of complex models: Coping with dynamic and static inputs. Reliability Engineering and System Safety, 134, 268–275. https://doi.org/10.1016/j.ress.2014.08.010

Archer, G. E. B., Saltelli, A., & Sobol, I. M. (1997). Sensitivity measures, anova-like techniques and the use of bootstrap. Journal of Statistical Computation and Simulation, 58(2), 99–120. https://doi.org/10.1080/00949659708811825

Asensio-Sevilla, M. I., Santos-Martín, M. T., Álvarez-León, D., & Ferragut-Canals, L. (2020). Global sensitivity analysis of fuel-type-dependent input variables of a simplified physical fire spread model. Mathematics and Computers in Simulation, 172, 33–44. https://doi.org/10.1016/j.matcom.2020.01.001

Awad, M., Senga Kiesse, T., Assaghir, Z., & Ventura, A. (2019). Convergence of sensitivity analysis methods for evaluating combined influences of model inputs. Reliability Engineering and System Safety, 189, 109–122. https://doi.org/10.1016/j.ress.2019.03.050

Bajracharya, A., Awoye, H., Stadnyk, T., & Asadzadeh, M. (2020). Time variant sensitivity analysis of hydrological model parameters in a cold region using flow signatures. Water, 12(4), 961. https://doi.org/10.3390/W12040961

Baroni, G., Scheiffele, L. M., Schrön, M., Ingwersen, J., & Oswald, S. E. (2018). Uncertainty, sensitivity and improvements in soil moisture estimation with cosmic-ray neutron sensing. Journal of Hydrology, 564, 873–887. https://doi.org/10.1016/j.jhydrol.2018.07.053

Baroni, G., & Francke, T. (2020). An effective strategy for combining variance- and distribution-based global sensitivity analysis. Environmental Modelling and Software, 134, 104851. https://doi.org/10.1016/j.envsoft.2020.104851

Benedetti, L., Claeys, F., Nopens, I., & Vanrolleghem, P. A. (2011). Assessing the convergence of LHS Monte Carlo simulations of wastewater treatment models. Water Science and Technology, 63(10), 2219–2224. https://doi.org/10.2166/wst.2011.453

Berezowski, T., Nossent, J., Chormaåski, J., & Batelaan, O. (2015). Spatial sensitivity analysis of snow cover data in a distributed rainfall-runoff model. Hydrology and Earth System Sciences, 19(4), 1887–1904. https://doi.org/10.5194/hess-19-1887-2015

Borroni, C. G. (2013). A new rank correlation measure. Statistical Papers, 54(2), 255–270. https://doi.org/10.1007/s00362-011-0423-0

Branger, F., Giraudet, L. G., Guivarch, C., & Quirion, P. (2015). Global sensitivity analysis of an energy-economy model of the residential building sector. Environmental Modelling and Software, 70, 45–54. https://doi.org/10.1016/j.envsoft.2015.03.021

Brembilla, E., Mardaljevic, J., & Hopfe, C. J. (2015). Sensitivity analysis studying the impact of reflectance values assigned in climate-based daylight modelling. 14th International Conference of IBPSA - Building Simulation 2015, BS 2015, Conference Proceedings, September, 1197–1204.

Budamala, V., & Baburao Mahindrakar, A. (2020). Integration of Adaptive Emulators and Sensitivity Analysis for Enhancement of Complex Hydrological Models. Environmental Processes, 7(4), 1235–1253. https://doi.org/10.1007/s40710-020-00468-x

Campolongo, F., Cariboni, J., & Saltelli, A. (2007). An effective screening design for sensitivity analysis of large models. Environmental Modelling and Software, 22(10), 1509–1518. https://doi.org/10.1016/j.envsoft.2006.10.004

Castaings, W., Borgonovo, E., Morris, M. D., & Tarantola, S. (2012). Sampling strategies in density-based sensitivity analysis. Environmental Modelling and Software, 38, 13–26. https://doi.org/10.1016/j.envsoft.2012.04.017

Chaney, N. W., Herman, J. D., Ek, M. B., & Wood, E. F. (2016). Deriving global parameter estimates for the Noah land surface model using FLUXNET and machine learning. Journal of Geophysical Research, 121(22), 13,218-13,235. https://doi.org/10.1002/2016JD024821

Chisari, C., Rizzano, G., Amadio, C., & Galdi, V. (2018). Sensitivity analysis and calibration of phenomenological models for seismic analyses. Soil Dynamics and Earthquake Engineering, 109, 10–22. https://doi.org/10.1016/j.soildyn.2018.02.024

Ciric, C., Ciffroy, P., & Charles, S. (2012). Use of sensitivity analysis to identify influential and non-influential parameters within an aquatic ecosystem model. Ecological Modelling, 246(C), 119–130. https://doi.org/10.1016/j.ecolmodel.2012.06.024

Cloke, H. L., Pappenberger, F., & Renaud, J. P. (2008). Multi-Method Global Sensitivity Analysis (MMGSA) for modelling floodplain hydrological processes. Hydrological Processes, 22(11), 1660–1674. https://doi.org/10.1002/hyp.6734

Confalonieri, R., Bellocchi, G., Bregaglio, S., Donatelli, M., & Acutis, M. (2010a). Comparison of sensitivity analysis techniques: A case study with the rice model WARM. Ecological Modelling, 221(16), 1897–1906. https://doi.org/10.1016/j.ecolmodel.2010.04.021

Confalonieri, R., Bregaglio, S., & Acutis, M. (2012). Quantifying plasticity in simulation models. Ecological Modelling, 225, 159–166. https://doi.org/10.1016/j.ecolmodel.2011.11.022

Confalonieri, R., Bellocchi, G., Tarantola, S., Acutis, M., Donatelli, M., & Genovese, G. (2010b). Sensitivity analysis of the rice model WARM in Europe: Exploring the effects of different locations, climates and methods of analysis on model sensitivity to crop parameters. Environmental Modelling & Software, 25(4), 479–488. https://doi.org/10.1016/j.envsoft.2009.10.005

Coolen-Maturi, T. (2014). A new weighted rank coefficient of concordance. Journal of Applied Statistics, 41(8), 1721–1745. https://doi.org/10.1080/02664763.2014.889664

Cosenza, A., Mannina, G., Vanrolleghem, P. A., & Neumann, M. B. (2013). Global sensitivity analysis in wastewater applications: A comprehensive comparison of different methods. Environmental Modelling and Software, 49, 40–52. https://doi.org/10.1016/j.envsoft.2013.07.009

Cosenza, A., Mannina, G., Vanrolleghem, P. A., & Neumann, M. B. (2014). Variance-based sensitivity analysis for wastewater treatment plant modelling. Science of the Total Environment, 470–471, 1068–1077. https://doi.org/10.1016/j.scitotenv.2013.10.069

Crosetto, M., Tarantola, S., & Saltelli, A. (2000). Sensitivity and uncertainty analysis in spatial modelling based on GIS. Agriculture, Ecosystems and Environment, 81(1), 71–79. https://doi.org/10.1016/S0167-8809(00)00169-9

Cruz May, E., Bassam, A., Ricalde, L. J., Escalante Soberanis, M. A., Oubram, O., May Tzuc, O., Alanis, A. Y., & Livas-García, A. (2021). Global sensitivity analysis for a real-time electricity market forecast by a machine learning approach: A case study of Mexico. International Journal of Electrical Power and Energy Systems, 135, 107505. https://doi.org/10.1016/j.ijepes.2021.107505

Dai, H., & Ye, M. (2015). Variance-based global sensitivity analysis for multiple scenarios and models with implementation using sparse grid collocation. Journal of Hydrology, 528, 286–300. https://doi.org/10.1016/j.jhydrol.2015.06.034

Devak, M., & Dhanya, C. T. (2017). Sensitivity analysis of hydrological models: Review and way forward. Journal of Water and Climate Change, 8(4), 557–575. https://doi.org/10.2166/wcc.2017.149

Do, N. C., & Razavi, S. (2020). Correlation Effects? A Major but Often Neglected Component in Sensitivity and Uncertainty Analysis. Water Resources Research, 56(3). https://doi.org/10.1029/2019WR025436

Douglas-Smith, D., Iwanaga, T., Croke, B. F. W., & Jakeman, A. J. (2020). Certain trends in uncertainty and sensitivity analysis: An overview of software tools and techniques. Environmental Modelling and Software, 124, 104588. https://doi.org/10.1016/j.envsoft.2019.104588

Estrada, V., & Diaz, M. S. (2010). Global sensitivity analysis in the development of first principle-based eutrophication models. Environmental Modelling and Software, 25(12), 1539–1551. https://doi.org/10.1016/j.envsoft.2010.06.009

Faggianelli, G. A., Mora, L., & Merheb, R. (2017). Uncertainty quantification for Energy Savings Performance Contracting: Application to an office building. Energy and Buildings, 152, 61–72. https://doi.org/10.1016/j.enbuild.2017.07.022

Garcia, D., Arostegui, I., & Prellezo, R. (2019). Robust combination of the Morris and Sobol methods in complex multidimensional models. Environmental Modelling and Software, 122, 104517. https://doi.org/10.1016/j.envsoft.2019.104517

Ge, Q., & Menendez, M. (2014). An Efficient Sensitivity Analysis Approach for Computationally Expensive Microscopic Traffic Simulation Models. International Journal of Transportation, 2(2), 49–64. https://doi.org/10.14257/ijt.2014.2.2.04

Ghasemizade, M., Baroni, G., Abbaspour, K., & Schirmer, M. (2017). Combined analysis of time-varying sensitivity and identifiability indices to diagnose the response of a complex environmental model. Environmental Modelling and Software, 88, 22–34. https://doi.org/10.1016/j.envsoft.2016.10.011

Gilardelli, C., Confalonieri, R., Cappelli, G. A., & Bellocchi, G. (2018). Sensitivity of WOFOST-based modelling solutions to crop parameters under climate change. Ecological Modelling, 368, 1–14. https://doi.org/10.1016/j.ecolmodel.2017.11.003

Gilquin, L., Prieur, C., Arnaud, E., & Monod, H. (2021). Iterative estimation of Sobol’ indices based on replicated designs. Computational and Applied Mathematics, 40(1), 1–23. https://doi.org/10.1007/s40314-020-01402-5

Gokarakonda, S., van Treeck, C., & Rawal, R. (2019). Influence of building design and control parameters on the potential of mixed-mode buildings in India. Building and Environment, 148, 157–172. https://doi.org/10.1016/j.buildenv.2018.10.043

Gómez-Delgado, M., & Tarantola, S. (2006). GLOBAL sensitivity analysis, GIS and multi-criteria evaluation for a sustainable planning of a hazardous waste disposal site in Spain. International Journal of Geographical Information Science, 20(4), 449–466. https://doi.org/10.1080/13658810600607709

Gschwend, D., Soltic, P., Edinger, P., Wokaun, A., & Vogel, F. (2017). Performance evaluation of gasoline alternatives using a thermodynamic spark-ignition engine model. Sustainable Energy and Fuels, 1(9), 1991–2005. https://doi.org/10.1039/C7SE00276A

Guillaume, J. H. A., Jakeman, J. D., Marsili-Libelli, S., Asher, M., Brunner, P., Croke, B., Hill, M. C., Jakeman, A. J., Keesman, K. J., Razavi, S., & Stigter, J. D. (2019). Introductory overview of identifiability analysis: A guide to evaluating whether you have the right type of data for your modeling purpose. Environmental Modelling and Software, 119, 418–432. https://doi.org/10.1016/j.envsoft.2019.07.007

Gupta, H. V., & Razavi, S. (2018). Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models. Water Resources Research, 54(11), 8692–8717. https://doi.org/10.1029/2018WR022668

Hart, J., & Gremaud, P. (2019). Robustness of the Sobol’ indices to distributional uncertainty. International Journal for Uncertainty Quantification, 9(5), 453–469. https://doi.org/10.1615/Int.J.UncertaintyQuantification.2019030553

Hartmann, A., Šimůnek, J., Aidoo, M. K., Seidel, S. J., & Lazarovitch, N. (2018). Implementation and Application of a Root Growth Module in HYDRUS. Vadose Zone Journal, 17(1), 170040. https://doi.org/10.2136/vzj2017.02.0040

Helton, J. C., Davis, F. J., & Johnson, J. D. (2005). A comparison of uncertainty and sensitivity analysis results obtained with random and Latin hypercube sampling. Reliability Engineering and System Safety, 89(3), 305–330. https://doi.org/10.1016/j.ress.2004.09.006

Herman, J. D., Kollat, J. B., Reed, P. M., & Wagener, T. (2013). Technical Note: Method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models. Hydrology and Earth System Sciences, 17(7), 2893–2903. https://doi.org/10.5194/hess-17-2893-2013

Homma, T., & Saltelli, A. (1996). Importance measures in global sensitivity analysis of nonlinear models. Reliability Engineering & System Safety, 52(1), 1–17. https://doi.org/10.1016/0951-8320(96)00002-6

Hornberger, G., & Spear, R. (1981). Approach to the preliminary analysis of environmental systems. J. Environ. Manage.; (United States), 12:1, 7–18. https://www.osti.gov/biblio/6396608

Hsieh, N. H., Bois, F. Y., Tsakalozou, E., Ni, Z., Yoon, M., Sun, W., Klein, M., Reisfeld, B., & Chiu, W. A. (2021). A Bayesian population physiologically based pharmacokinetic absorption modeling approach to support generic drug development: application to bupropion hydrochloride oral dosage forms. Journal of Pharmacokinetics and Pharmacodynamics, 5. https://doi.org/10.1007/s10928-021-09778-5

Hsieh, N. H., Reisfeld, B., Bois, F. Y., & Chiu, W. A. (2018). Applying a global sensitivity analysis workflow to improve the computational efficiencies in physiologically-based pharmacokinetic modeling. Frontiers in Pharmacology, 9, 1–17. https://doi.org/10.3389/fphar.2018.00588

Hsieh, N. H., Reisfeld, B., & Chiu, W. A. (2020). pksensi: An R package to apply global sensitivity analysis in physiologically based kinetic modeling. SoftwareX, 12, 100609. https://doi.org/10.1016/j.softx.2020.100609

Hu, Y., Garcia-Cabrejo, O., Cai, X., Valocchi, A. J., & DuPont, B. (2015). Global sensitivity analysis for large-scale socio-hydrological models using Hadoop. Environmental Modelling & Software, 73, 231–243. https://doi.org/10.1016/j.envsoft.2015.08.015

Huo, X., Gupta, H., Niu, G. Y., Gong, W., & Duan, Q. (2019). Parameter Sensitivity Analysis for Computationally Intensive Spatially Distributed Dynamical Environmental Systems Models. Journal of Advances in Modeling Earth Systems, 11(9), 2896–2909. https://doi.org/10.1029/2018MS001573

Iman, R. L., & Conover, W. J. (1987). A Measure of Top-Down Correlation. Technometrics, 29(3), 351. https://doi.org/10.2307/1269344

Isaksson, A., Wallman, M., Göransson, H., & Gustafsson, M. G. (2008). Cross-validation and bootstrapping are unreliable in small sample classification. Pattern Recognition Letters, 29(14), 1960–1965. https://doi.org/10.1016/j.patrec.2008.06.018

Jabloun, M., Li, X., Zhang, X., Tao, F., Hu, C., & Olesen, J. E. (2018). Sensitivity of simulated crop yield and nitrate leaching of the wheat-maize cropping system in the North China Plain to model parameters. Agricultural and Forest Meteorology, 263, 25–40. https://doi.org/10.1016/j.agrformet.2018.08.002

Jakeman, A. J., Letcher, R. A., & Norton, J. P. (2006). Ten iterative steps in development and evaluation of environmental models. Environmental Modelling and Software, 21(5), 602–614. https://doi.org/10.1016/j.envsoft.2006.01.004

Jakeman, J. (2023). PyApprox: A software package for sensitivity analysis, Bayesian inference, optimal experimental design, and multi-fidelity uncertainty quantification and surrogate modeling. Environmental Modelling and Software, 170, 105825. https://doi.org/10.1016/j.envsoft.2023.105825

Janssen, H. (2013). Monte-Carlo based uncertainty analysis: Sampling efficiency and sampling convergence. Reliability Engineering and System Safety, 109, 123–132. https://doi.org/10.1016/j.ress.2012.08.003

Kavetski, D., & Clark, M. P. (2010). Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction. Water Resources Research, 46(10), 2009WR008896. https://doi.org/10.1029/2009WR008896

KC, U., Aryal, J., Garg, S., & Hilton, J. (2021). Global sensitivity analysis for uncertainty quantification in fire spread models. Environmental Modelling and Software, 143, 105110. https://doi.org/10.1016/j.envsoft.2021.105110

KC, U., Garg, S., Hilton, J., & Aryal, J. (2020). A cloud-based framework for sensitivity analysis of natural hazard models. Environmental Modelling and Software, 134, 104800. https://doi.org/10.1016/j.envsoft.2020.104800

Kelleher, C., & Wagener, T. (2011). Ten guidelines for effective data visualization in scientific publications. Environmental Modelling & Software, 26(6), 822–827. https://doi.org/10.1016/j.envsoft.2010.12.006

Khan, S., & Kaklis, P. (2021). From regional sensitivity to intra-sensitivity for parametric analysis of free-form shapes: Application to ship design. Advanced Engineering Informatics, 49, 101314. https://doi.org/10.1016/j.aei.2021.101314

Khorashadi Zadeh, F., Nossent, J., Sarrazin, F., Pianosi, F., van Griensven, A., Wagener, T., & Bauwens, W. (2017). Comparison of variance-based and moment-independent global sensitivity analysis approaches by application to the SWAT model. Environmental Modelling and Software, 91, 210–222. https://doi.org/10.1016/j.envsoft.2017.02.001

Koo, H., Iwanaga, T., Croke, B. F. W., Jakeman, A. J., Yang, J., Wang, H. H., Sun, X., Lü, G., Li, X., Yue, T., Yuan, W., Liu, X., & Chen, M. (2020). Position paper: Sensitivity analysis of spatially distributed environmental models- a pragmatic framework for the exploration of uncertainty sources. Environmental Modelling and Software, 134, 104857. https://doi.org/10.1016/j.envsoft.2020.104857

Krishnan, P., & Aggarwal, P. (2018). Global sensitivity and uncertainty analyses of a web based crop simulation model (web InfoCrop wheat) for soil parameters. Plant and Soil, 423(1–2), 443–463. https://doi.org/10.1007/s11104-017-3498-0

Krishnan, P., Maity, P. P., & Kundu, M. (2021). Sensitivity analysis of cultivar parameters to simulate wheat crop growth and yield under moisture and temperature stress conditions. Heliyon, 7(7), e07602. https://doi.org/10.1016/j.heliyon.2021.e07602

Kroll, S., Dirckx, G., Donckels, B. M. R., Van Dorpe, M., Weemaes, M., & Willems, P. (2016). Modelling real-time control of WWTP influent flow under data scarcity. Water Science and Technology, 73(7), 1637–1643. https://doi.org/10.2166/wst.2015.641

Lebedeva, G., Sorokin, A., Faratian, D., Mullen, P., Goltsov, A., Langdon, S. P., Harrison, D. J., & Goryanin, I. (2012). Model-based global sensitivity analysis as applied to identification of anti-cancer drug targets and biomarkers of drug resistance in the ErbB2/3 network. European Journal of Pharmaceutical Sciences, 46(4), 244–258. https://doi.org/10.1016/j.ejps.2011.10.026

Leolini, L., Bregaglio, S., Moriondo, M., Ramos, M. C., Bindi, M., & Ginaldi, F. (2018). A model library to simulate grapevine growth and development: software implementation, sensitivity analysis and field level application. European Journal of Agronomy, 99, 92–105. https://doi.org/10.1016/j.eja.2018.06.006

Li, P., & Ren, L. (2019). Evaluating the effects of limited irrigation on crop water productivity and reducing deep groundwater exploitation in the North China Plain using an agro-hydrological model: I. Parameter sensitivity analysis, calibration and model validation. Journal of Hydrology, 574(2), 497–516. https://doi.org/10.1016/j.jhydrol.2019.04.053

Likhachev, D. V. (2019). Parametric sensitivity analysis as an essential ingredient of spectroscopic ellipsometry data modeling: An application of the Morris screening method. Journal of Applied Physics, 126(18). https://doi.org/10.1063/1.5126074

Liu, J., Liu, Z., Zhu, A. X., Shen, F., Lei, Q., & Duan, Z. (2019a). Global sensitivity analysis of the APSIM-Oryza rice growth model under different environmental conditions. Science of the Total Environment, 651, 953–968. https://doi.org/10.1016/j.scitotenv.2018.09.254

Liu, K., Xin, X., Ma, J., Zhang, J., & Yu, Q. (2019b). Sensitivity analysis of ship traffic in restricted two-way waterways considering the impact of LNG carriers. Ocean Engineering, 192, 106556. https://doi.org/10.1016/j.oceaneng.2019.106556

Locatelli, T., Tarantola, S., Gardiner, B., & Patenaude, G. (2017). Variance-based sensitivity analysis of a wind risk model - Model behaviour and lessons for forest modelling. Environmental Modelling and Software, 87, 84–109. https://doi.org/10.1016/j.envsoft.2016.10.010

Mai, J., & Tolson, B. A. (2019). Model Variable Augmentation (MVA) for Diagnostic Assessment of Sensitivity Analysis Results. Water Resources Research, 55(4), 2631–2651. https://doi.org/10.1029/2018WR023382

Mailier, J., Delmotte, A., Cloutier, M., Jolicoeur, M., & VandeWouwer, A. (2011). Sensitivity analysis for parameter identification of a plant growth model. In IFAC Proceedings Volumes (IFAC-PapersOnline) (Vol. 44, Issue 1 PART 1). IFAC. https://doi.org/10.3182/20110828-6-IT-1002.00315

Marino, S., Hogue, I. B., Ray, C. J., & Kirschner, D. E. (2008). A methodology for performing global uncertainty and sensitivity analysis in systems biology. Journal of Theoretical Biology, 254(1), 178–196. https://doi.org/10.1016/j.jtbi.2008.04.011

Matthews, C. J., Newton, D. B., Braddock, R. D., & Yu, B. (2007). Analysing the sensitivity behaviour of two hydrology models. Environmental Modeling and Assessment, 12(1), 27–41. https://doi.org/10.1007/s10666-006-9049-3

Medina, Y., & Muñoz, E. (2020a). A simple time-varying sensitivity analysis (TVSA) for assessment of temporal variability of hydrological processes. Water, 12(9), 2463. https://doi.org/10.3390/w12092463

Medina, Y., & Muñoz, E. (2020b). Analysis of the relative importance of model parameters in watersheds with different hydrological regimes. Water, 12(9), 2376. https://doi.org/10.3390/W12092376

Moreau, P., Viaud, V., Parnaudeau, V., Salmon-Monviola, J., & Durand, P. (2013). An approach for global sensitivity analysis of a complex environmental model to spatial inputs and parameters: A case study of an agro-hydrological model. Environmental Modelling and Software, 47, 74–87. https://doi.org/10.1016/j.envsoft.2013.04.006

Neumann, M. B. (2012). Comparison of sensitivity analysis methods for pollutant degradation modelling: A case study from drinking water treatment. Science of the Total Environment, 433, 530–537. https://doi.org/10.1016/j.scitotenv.2012.06.026

Nguyen, A. T., & Reiter, S. (2015). A performance comparison of sensitivity analysis methods for building energy models. Building Simulation, 8(6), 651–664. https://doi.org/10.1007/s12273-015-0245-4

Nossent, J., Elsen, P., & Bauwens, W. (2011). Sobol’ sensitivity analysis of a complex environmental model. Environmental Modelling and Software, 26(12), 1515–1525. https://doi.org/10.1016/j.envsoft.2011.08.010

Page, T., Smith, P., Beven, K., Pianosi, F., Sarrazin, F., Almeida, S., Holcombe, L., Freer, J., Chappell, N., & Wagener, T. (2023). Technical note: The CREDIBLE Uncertainty Estimation (CURE) toolbox: facilitating the communication of epistemic uncertainty. Hydrology and Earth System Sciences, 27(13), 2523–2534. https://doi.org/10.5194/hess-27-2523-2023

Paleari, L., & Confalonieri, R. (2016). Sensitivity analysis of a sensitivity analysis: We are likely overlooking the impact of distributional assumptions. Ecological Modelling. https://doi.org/10.1016/j.ecolmodel.2016.09.008

Paleari, L., Movedi, E., Zoli, M., Burato, A., Cecconi, I., Errahouly, J., Pecollo, E., Sorvillo, C., & Confalonieri, R. (2021). Sensitivity analysis using Morris: Just screening or an effective ranking method? Ecological Modelling, 455, 109648. https://doi.org/10.1016/j.ecolmodel.2021.109648

Pappenberger, F., Beven, K. J., Ratto, M., & Matgen, P. (2008). Multi-method global sensitivity analysis of flood inundation models. Advances in Water Resources, 31(1), 1–14. https://doi.org/10.1016/j.advwatres.2007.04.009

Peeters, L. J. M., Pagendam, D. E., Crosbie, R. S., Rachakonda, P. K., Dawes, W. R., Gao, L., Marvanek, S. P., Zhang, Y. Q., & McVicar, T. R. (2018). Determining the initial spatial extent of an environmental impact assessment with a probabilistic screening methodology. Environmental Modelling and Software, 109, 353–367. https://doi.org/10.1016/j.envsoft.2018.08.020

Peng, X., Adamowski, J., Inam, A., Alizadeh, M. R., & Albano, R. (2020). Development of a behaviour-pattern based global sensitivity analysis procedure for coupled socioeconomic and environmental models. Journal of Hydrology, 585, 124745. https://doi.org/10.1016/j.jhydrol.2020.124745

Pianosi, F., Beven, K., Freer, J., Hall, J. W., Rougier, J., Stephenson, D. B., & Wagener, T. (2016). Sensitivity analysis of environmental models: A systematic review with practical workflow. Environmental Modelling and Software, 79, 214–232. https://doi.org/10.1016/j.envsoft.2016.02.008

Pianosi, F., Sarrazin, F., & Wagener, T. (2015). A Matlab toolbox for Global Sensitivity Analysis. Environmental Modelling & Software, 70, 80–85. https://doi.org/10.1016/j.envsoft.2015.04.009

Pianosi, F., Sarrazin, F., & Wagener, T. (2020). How successfully is open-source research software adopted? Results and implications of surveying the users of a sensitivity analysis toolbox. Environmental Modelling and Software, 124. https://doi.org/10.1016/j.envsoft.2019.104579

Qian, G., & Mahdi, A. (2020). Sensitivity analysis methods in the biomedical sciences. Mathematical Biosciences, 323, 108306. https://doi.org/10.1016/j.mbs.2020.108306

Ravalico, J. K., Dandy, G. C., & Maier, H. R. (2010). Management Option Rank Equivalence (MORE) - A new method of sensitivity analysis for decision-making. Environmental Modelling and Software, 25(2), 171–181. https://doi.org/10.1016/j.envsoft.2009.06.012

Ravasi, R. A., Paleari, L., Vesely, F. M., Movedi, E., Thoelke, W., & Confalonieri, R. (2020). Ideotype definition to adapt legumes to climate change: A case study for field pea in Northern Italy. Agricultural and Forest Meteorology, 291. https://doi.org/10.1016/j.agrformet.2020.108081

Razavi, S., & Gupta, H. V. (2016a). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423–439. https://doi.org/10.1002/2015WR017558

Razavi, S., & Gupta, H. V. (2016b). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application. Water Resources Research, 52(1), 440–455. https://doi.org/10.1002/2015WR017559

Razavi, S., & Gupta, H. V. (2019). A multi-method Generalized Global Sensitivity Matrix approach to accounting for the dynamical nature of earth and environmental systems models. Environmental Modelling and Software, 114, 1–11. https://doi.org/10.1016/j.envsoft.2018.12.002

Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., Tarantola, S., Guillaume, J. H. A., Jakeman, J., Gupta, H., Melillo, N., Rabitti, G., Chabridon, V., Duan, Q., Sun, X., … Maier, H. R. (2021). The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support. Environmental Modelling and Software, 137. https://doi.org/10.1016/j.envsoft.2020.104954

Razavi, S., Sheikholeslami, R., Gupta, H. V., & Haghnegahdar, A. (2019). VARS-TOOL: A toolbox for comprehensive, efficient, and robust sensitivity and uncertainty analysis. Environmental Modelling and Software, 112, 95–107. https://doi.org/10.1016/j.envsoft.2018.10.005

Reinhart, B. D., Frankenberger, J. R., Hay, C. H., Bowling, L. C., & Hancock, B. G. (2020). Development and Sensitivity Analysis of an Online Tool for Evaluating Drainage Water Recycling Decisions. Transactions of the ASABE, 63(6), 1991–2002. https://doi.org/10.13031/trans.13900

Reusser, D. E., Buytaert, W., & Zehe, E. (2011). Temporal dynamics of model parameter sensitivity for computationally expensive models with the Fourier amplitude sensitivity test. Water Resources Research, 47(7). https://doi.org/10.1029/2010WR009947

Robles, A., Ruano, M. V., Ribes, J., Seco, A., & Ferrer, J. (2014a). Global sensitivity analysis of a filtration model for submerged anaerobic membrane bioreactors (AnMBR). Bioresource Technology, 158, 365–373. https://doi.org/10.1016/j.biortech.2014.02.087

Robles, A., Ruano, M. V., Ribes, J., Seco, A., & Ferrer, J. (2014b). Model-based automatic tuning of a filtration control system for submerged anaerobic membrane bioreactors (AnMBR). Journal of Membrane Science, 465, 14–26. https://doi.org/10.1016/j.memsci.2014.04.012

Ruano, M. V., Ribes, J., Seco, A., & Ferrer, J. (2012). An improved sampling strategy based on trajectory design for application of the Morris method to systems with many input factors. Environmental Modelling and Software, 37, 103–109. https://doi.org/10.1016/j.envsoft.2012.03.008

Şalap-Ayça, S., Jankowski, P., Clarke, K. C., & Nara, A. (2021). Is less more? Experimenting with visual stacking of coincident maps for spatial global sensitivity analysis in urban land-use change modeling. Environmental Modelling and Software, 145. https://doi.org/10.1016/j.envsoft.2021.105181

Saltelli, A., Chan, K., & Scott, E. (2000). Sensitivity Analysis: Wiley Series in Probability and Statistics. Wiley.

Saltelli, Andrea, & Annoni, P. (2010). How to avoid a perfunctory sensitivity analysis. Environmental Modelling and Software, 25(12), 1508–1517. https://doi.org/10.1016/j.envsoft.2010.04.012

Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., & Tarantola, S. (2010). Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Computer Physics Communications, 181(2), 259–270. https://doi.org/10.1016/j.cpc.2009.09.018

Saltelli, A., & Bolado, R. (1998). An alternative way to compute Fourier amplitude sensitivity test (FAST). Computational Statistics and Data Analysis, 26(4), 445–460. https://doi.org/10.1016/S0167-9473(97)00043-1

Saltelli, A., Jakeman, A., Razavi, S., & Wu, Q. (2021). Sensitivity analysis: A discipline coming of age. Environmental Modelling & Software, 146, 105226. https://doi.org/10.1016/j.envsoft.2021.105226

Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007). Global Sensitivity Analysis. The Primer. In International Statistical Review. John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470725184

Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004). Sensitivity analysis in practice: a guide to assessing scientific models.

Salviano, L. O., Gasparin, E. E., Mattos, V. C. N., Barbizan, B., Saltara, F., de Mello, P. E. B., Dezan, D. J., & Yanagihara, J. I. (2021). Sensitivity analysis and optimization of a CO2 centrifugal compressor impeller with a vaneless diffuser. Structural and Multidisciplinary Optimization, 64(3), 1607–1627. https://doi.org/10.1007/s00158-021-02914-2

Sarrazin, F., Hartmann, A., Pianosi, F., Rosolem, R., & Wagener, T. (2018). V2Karst V1.1: A parsimonious large-scale integrated vegetation-recharge model to simulate the impact of climate and land cover change in karst regions. In Geoscientific Model Development (Vol. 11, Issue 12). https://doi.org/10.5194/gmd-11-4933-2018

Sarrazin, F., Pianosi, F., & Wagener, T. (2016). Global Sensitivity Analysis of environmental models: Convergence and validation. Environmental Modelling and Software, 79, 135–152. https://doi.org/10.1016/j.envsoft.2016.02.005

Savage, I. R. (1956). Contributions to the Theory of Rank Order Statistics-the Two-Sample Case. The Annals of Mathematical Statistics, 27(3), 590–615. https://www.jstor.org/stable/2237370

Seo, S. B., Duchnowski, E. M., O’Neal, M., Motta, A. T., Passelaigue, F., Kang, S., Pastore, G., Manera, A., Petrov, V., Huang, P. H., & Brown, N. R. (2021). Sensitivity analysis of BISON model for characterization of impact of experimental parameters on hydrogen migration and redistribution in zirconium-based alloys. Journal of Nuclear Materials, 550, 152941. https://doi.org/10.1016/j.jnucmat.2021.152941

Sheikholeslami, R., Gharari, S., Papalexiou, S. M., & Clark, M. P. (2021). VISCOUS: A Variance-Based Sensitivity Analysis Using Copulas for Efficient Identification of Dominant Hydrological Processes. Water Resources Research, 57(7), 1–24. https://doi.org/10.1029/2020WR028435

Sheikholeslami, R., & Razavi, S. (2017). Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models. Environmental Modelling and Software, 93, 109–126. https://doi.org/10.1016/j.envsoft.2017.03.010

Sheikholeslami, R., Razavi, S., Gupta, H. V., Becker, W., & Haghnegahdar, A. (2019a). Global sensitivity analysis for high-dimensional problems: How to objectively group factors and measure robustness and convergence while reducing computational cost. Environmental Modelling and Software, 111, 282–299. https://doi.org/10.1016/j.envsoft.2018.09.002

Sheikholeslami, R., Razavi, S., & Haghnegahdar, A. (2019b). What should we do when a model crashes? Recommendations for global sensitivity analysis of Earth and environmental systems models. Geoscientific Model Development, 12(10), 4275–4296. https://doi.org/10.5194/gmd-12-4275-2019

Sheikholeslami, R., Yassin, F., Lindenschmidt, K. E., & Razavi, S. (2017). Improved understanding of river ice processes using global sensitivity analysis approaches. Journal of Hydrologic Engineering, 22(11), 04017048. https://doi.org/10.1061/(ASCE)HE.1943-5584.0001574

Shin, M. J., Guillaume, J. H. A., Croke, B. F. W., & Jakeman, A. J. (2013). Addressing ten questions about conceptual rainfall-runoff models with global sensitivity analyses in R. Journal of Hydrology, 503, 135–152. https://doi.org/10.1016/j.jhydrol.2013.08.047

Shin, M. J., Guillaume, J. H. A., Croke, B. F. W., & Jakeman, A. J. (2015). A review of foundational methods for checking the structural identifiability of models: Results for rainfall-runoff. Journal of Hydrology, 520, 1–16. https://doi.org/10.1016/j.jhydrol.2014.11.040

Sieber, A., & Uhlenbrook, S. (2005). Sensitivity analyses of a distributed catchment model to verify the model structure. Journal of Hydrology, 310(1–4), 216–235. https://doi.org/10.1016/j.jhydrol.2005.01.004

Silvestro, P. C., Pignatti, S., Yang, H., Yang, G., Pascucci, S., Castaldi, F., & Casa, R. (2017). Sensitivity analysis of the Aquacrop and SAFYE crop models for the assessment of water limited winter wheat yield in regional scale applications. PLoS ONE, 12(11), 1–30. https://doi.org/10.1371/journal.pone.0187485

Sobol, M. (1993). Sensitivity Estimates for Nonlinear Mathematical Models. Mathematical Modeling and Computational Experiment, 1(4), 407–414. https://doi.org/1061-7590/93/04407-008$9.00

Song, X. M., Kong, F. Z., Zhan, C. S., Han, J. W., & Zhang, X. H. (2013a). Parameter identification and global sensitivity analysis of Xin’anjiang model using meta-modeling approach. Water Science and Engineering, 6(1), 1–17. https://doi.org/10.3882/j.issn.1674-2370.2013.01.001

Song, X., Bryan, B. A., Almeida, A. C., Paul, K. I., Zhao, G., & Ren, Y. (2013b). Time-dependent sensitivity of a process-based ecological model. Ecological Modelling, 265, 114–123. https://doi.org/10.1016/j.ecolmodel.2013.06.013

Song, X., Bryan, B. A., Paul, K. I., & Zhao, G. (2012). Variance-based sensitivity analysis of a forest growth model. Ecological Modelling, 247, 135–143. https://doi.org/10.1016/j.ecolmodel.2012.08.005

Song, X., Zhang, J., Zhan, C., Xuan, Y., Ye, M., & Xu, C. (2015). Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications. Journal of Hydrology, 523(225), 739–757. https://doi.org/10.1016/j.jhydrol.2015.02.013

Specka, X., Nendel, C., & Wieland, R. (2019). Temporal sensitivity analysis of the MONICA model: Application of two global approaches to analyze the dynamics of parameter sensitivity. Agriculture (Switzerland), 9(2), 37. https://doi.org/10.3390/agriculture9020037

Sreedevi, S., Eldho, T. I., Madhusoodhanan, C. G., & Jayasankar, T. (2019). Multiobjective sensitivity analysis and model parameterization approach for coupled streamflow and groundwater table depth simulations using SHETRAN in a wet humid tropical catchment. Journal of Hydrology, 579, 124217. https://doi.org/10.1016/j.jhydrol.2019.124217

Sudheer, K. P., Lakshmi, G., & Chaubey, I. (2011). Application of a pseudo simulator to evaluate the sensitivity of parameters in complex watershed models. Environmental Modelling and Software, 26(2), 135–143. https://doi.org/10.1016/j.envsoft.2010.07.007

Sun, X. (2021). SAConvergenceAnalysis. GitHub. https://github.com/Xifus/SAConvergenceAnalysis

Sun, X., Croke, B., Jakeman, A., & Roberts, S. (2022). Benchmarking Active Subspace methods of global sensitivity analysis against variance-based Sobol’ and Morris methods with established test functions. Environmental Modelling & Software, 149, 105310. https://doi.org/10.1016/j.envsoft.2022.105310

Sun, X., Croke, B., Roberts, S., & Jakeman, A. (2021). Comparing methods of randomizing Sobol′ sequences for improving uncertainty of metrics in variance-based global sensitivity estimation. Reliability Engineering & System Safety, 210, 107499. https://doi.org/10.1016/j.ress.2021.107499

Tan, J., Cui, Y., & Luo, Y. (2017). Assessment of uncertainty and sensitivity analyses for ORYZA model under different ranges of parameter variation. European Journal of Agronomy, 91, 54–62. https://doi.org/10.1016/j.eja.2017.09.001

Tang, Y., Reed, P., Wagener, T., & Van Werkhoven, K. (2007). Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation. Hydrology and Earth System Sciences, 11(2), 793–817. https://doi.org/10.5194/hess-11-793-2007

Tarantola, S., Becker, W., & Zeitz, D. (2012). A comparison of two sampling methods for global sensitivity analysis. Computer Physics Communications, 183(5), 1061–1072. https://doi.org/10.1016/j.cpc.2011.12.015

Tarantola, S., Ferretti, F., Lo Piano, S., Kozlova, M., Lachi, A., Rosati, R., Puy, A., Roy, P., Vannucci, G., Kuc-Czarnecka, M., & Saltelli, A. (2024). An annotated timeline of sensitivity analysis. Environmental Modelling & Software, 174, 105977. https://doi.org/10.1016/j.envsoft.2024.105977

Touhami, H. B., Lardy, R., Barra, V., & Bellocchi, G. (2013). Screening parameters in the Pasture Simulation model using the Morris method. Ecological Modelling, 266(1), 42–57. https://doi.org/10.1016/j.ecolmodel.2013.07.005

Touzani, S., & Busby, D. (2014). Screening Method Using the Derivative-based Global Sensitivity Indices with Application to Reservoir Simulator. Oil & Gas Science and Technology – Revue d’IFP Energies Nouvelles, 69(4), 619–632. https://doi.org/10.2516/ogst/2013195

Uliana, E. M., da Silva, D. D., Moreira, M. C., & Pereira, D. dos R. (2019). Global sensitivity analysis methods applied to hydrologic modeling with the SAC-SMA model. Engenharia Agricola, 39(1), 65–74. https://doi.org/10.1590/1809-4430-Eng.Agric.v39n1p65-74/2019

Upreti, D., Pignatti, S., Pascucci, S., Tolomio, M., Li, Z., Huang, W., & Casa, R. (2020). A Comparison of Moment-Independent and Variance-Based Global Sensitivity Analysis Approaches for Wheat Yield Estimation with the Aquacrop-OS Model. Agronomy, 10(4), 607. https://doi.org/10.3390/agronomy10040607

Van Werkhoven, K., Wagener, T., Reed, P., & Tang, Y. (2008). Characterization of watershed model behavior across a hydroclimatic gradient. Water Resources Research, 44(1), 1–16. https://doi.org/10.1029/2007WR006271

Vanrolleghem, P. A., Mannina, G., Cosenza, A., & Neumann, M. B. (2015). Global sensitivity analysis for urban water quality modelling: Terminology, convergence and comparison of different methods. Journal of Hydrology, 522, 339–352. https://doi.org/10.1016/j.jhydrol.2014.12.056

Vigna, S. (2015). A weighted correlation index for rankings with ties. WWW 2015 - Proceedings of the 24th International Conference on World Wide Web, 1166–1176. https://doi.org/10.1145/2736277.2741088

Wagener, T., & Pianosi, F. (2019). What has Global Sensitivity Analysis ever done for us? A systematic review to support scientific advancement and to inform policy-making in earth system modelling. Earth-Science Reviews, 194, 1–18. https://doi.org/10.1016/j.earscirev.2019.04.006

Wagener, T., Van Werkhoven, K., Reed, P., & Tang, Y. (2009). Multiobjective sensitivity analysis to understand the information content in streamflow observations for distributed watershed modeling. Water Resources Research, 45(2), 1–5. https://doi.org/10.1029/2008WR007347

Wang, A., & Solomatine, D. P. (2019). Practical experience of sensitivity analysis: Comparing six methods, on three hydrological models, with three performance criteria. Water, 11(5), 1062, 1–26. https://doi.org/10.3390/w11051062

Wang, C., Duan, Q., Gong, W., Ye, A., Di, Z., & Miao, C. (2014). An evaluation of adaptive surrogate modeling based optimization with two benchmark problems. Environmental Modelling and Software, 60, 167–179. https://doi.org/10.1016/j.envsoft.2014.05.026

Wang, H., Gong, W., Duan, Q., & Di, Z. (2020). Evaluation of parameter interaction effect of hydrological models using the sparse polynomial chaos (SPC) method. Environmental Modelling and Software, 125. https://doi.org/10.1016/j.envsoft.2019.104612

Wang, J., Li, X., Lu, L., & Fang, F. (2013). Parameter sensitivity analysis of crop growth models based on the extended Fourier Amplitude Sensitivity Test method. Environmental Modelling and Software, 48, 171–182. https://doi.org/10.1016/j.envsoft.2013.06.007

Wang, Q., Guillaume, J. H. A., Jakeman, J. D., Yang, T., Iwanaga, T., Croke, B., & Jakeman, A. J. (2022). Assessing the predictive impact of factor fixing with an adaptive uncertainty-based approach. Environmental Modelling and Software, 148, 105290. https://doi.org/10.1016/j.envsoft.2021.105290

Wang, S., Flipo, N., & Romary, T. (2018). Time-dependent global sensitivity analysis of the C-RIVE biogeochemical model in contrasted hydrological and trophic contexts. Water Research, 144, 341–355. https://doi.org/10.1016/j.watres.2018.07.033

Wate, P., Iglesias, M., Coors, V., & Robinson, D. (2020). Framework for emulation and uncertainty quantification of a stochastic building performance simulator. Applied Energy, 258, 113759. https://doi.org/10.1016/j.apenergy.2019.113759

Wu, Z., Ma, B., Wang, H., Hu, C., Lv, H., & Zhang, X. (2021). Identification of Sensitive Parameters of Urban Flood Model Based on Artificial Neural Network. Water Resources Management, 35(7), 2115–2128. https://doi.org/10.1007/s11269-021-02825-3

Yang, J. (2011). Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis. Environmental Modelling and Software, 26(4), 444–457. https://doi.org/10.1016/j.envsoft.2010.10.007

Yang, J., Jakeman, A., Fang, G., & Chen, X. (2018). Uncertainty analysis of a semi-distributed hydrologic model based on a Gaussian Process emulator. Environmental Modelling and Software, 101, 289–300. https://doi.org/10.1016/j.envsoft.2017.11.037

Yang, J., Liu, Y., Yang, W., & Chen, Y. (2012). Multi-Objective Sensitivity Analysis of a Fully Distributed Hydrologic Model WetSpa. Water Resources Management, 26(1), 109–128. https://doi.org/10.1007/s11269-011-9908-9

Zhan, C., Song, X., Xia, J., & Tong, C. (2013). An efficient integrated approach for global sensitivity analysis of hydrological model parameters. Environmental Modelling and Software, 41, 39–52. https://doi.org/10.1016/j.envsoft.2012.10.009

Zhan, Y., & Zhang, M. (2013). Application of a combined sensitivity analysis approach on a pesticide environmental risk indicator. Environmental Modelling and Software, 49, 129–140. https://doi.org/10.1016/j.envsoft.2013.08.005

Zhang, C., Chu, J., & Fu, G. (2013). Sobol’’s sensitivity analysis for a distributed hydrological model of Yichun River Basin, China. Journal of Hydrology, 480, 58–68. https://doi.org/10.1016/j.jhydrol.2012.12.005

Zhao, G., Bryan, B. A., & Song, X. (2014). Sensitivity and uncertainty analysis of the APSIM-wheat model: Interactions between cultivar, environmental, and management parameters. Ecological Modelling, 279, 1–11. https://doi.org/10.1016/j.ecolmodel.2014.02.003

Zhao, Y., Nan, Z., Yu, W., & Zhang, L. (2019). Calibrating a hydrological model by stratifying frozen ground types and seasons in a cold alpine basin. Water, 11(5), 985. https://doi.org/10.3390/w11050985

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2024 Xifu Sun, Anthony J. Jakeman, Barry F.W. Croke, Stephen G. Roberts, John D. Jakeman