Toward SALib 2.0: Advancing the accessibility and interpretability of global sensitivity analyses
Article Full Text (PDF)


sensitivity analysis
community of practice
software accessibility

How to Cite

Iwanaga, T., Usher, W., & Herman, J. (2022). Toward SALib 2.0: Advancing the accessibility and interpretability of global sensitivity analyses. Socio-Environmental Systems Modelling, 4, 18155.


Sensitivity analysis is now considered a standard practice in environmental modeling. Several open-source libraries, such as the Sensitivity Analysis Library (SALib), have been published in the recent past aimed at simplifying the application of sensitivity analyses. Still, there remain issues in software usability and accessibility, as well as a lack of guidance in the interpretation of sensitivity analysis results. This paper describes the changes made and planned to SALib to advance the ease with which modelers may conduct sensitivity analysis and interpret results. We further offer our perspectives from the past 7 years of maintaining SALib for the consideration of those aspiring to launch their own software for sensitivity analysis, develop methodology, or those otherwise interested in becoming involved in a project like SALib. These include the value of a community of practice to foster best practices for sensitivity analysis, the potential for collaboration across different software (for sensitivity analysis) platforms, and the need to specifically support the software development that underpins computational science.

Article Full Text (PDF)


Aznar-Siguan, G., & Bresch, D. N. (2019). CLIMADA v1: A global weather and climate risk assessment platform. Geoscientific Model Development, 12(7), 3085–3097.

Baroni, G., & Francke, T. (2020). An effective strategy for combining variance- and distribution-based global sensitivity analysis. Environmental Modelling & Software, 134, 104851.

Borgonovo, E. (2007). A new uncertainty importance measure. Reliability Engineering & System Safety, 92(6), 771–784.

Bresch, D. N., & Aznar-Siguan, G. (2021). CLIMADA v1.4.1: Towards a globally consistent adaptation options appraisal tool. Geoscientific Model Development, 14(1), 351–363.

Campolongo, F., Cariboni, J., & Saltelli, A. (2007). An effective screening design for sensitivity analysis of large models. Environmental Modelling & Software, 22(10), 1509–1518.

Conda-Forge Community. (2015). The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem.

Cukier, R. I., Fortuin, C. M., Shuler, K. E., Petschek, A. G., & Schaibly, J. H. (1973). Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory. The Journal of Chemical Physics, 59(8), 3873–3878.

Cuntz, M., & Mai, J. (2020). pyeee: Parameter screening using Morris’ method or its extension of Efficient/Sequential Elementary Effects (2.0) [Computer software]. Zenodo.

Cuntz, M., Mai, J., Zink, M., Thober, S., Kumar, R., Schäfer, D., Schrön, M., Craven, J., Rakovec, O., Spieler, D., Prykhodko, V., Dalmasso, G., Musuuza, J., Langenberg, B., Attinger, S., & Samaniego, L. (2015). Computationally inexpensive identification of noninformative model parameters by sequential screening. Water Resources Research, 51(8), 6417–6441.

Douglas-Smith, D., Iwanaga, T., Croke, B. F. W., & Jakeman, A. J. (2020). Certain trends in uncertainty and sensitivity analysis: An overview of software tools and techniques. Environmental Modelling & Software, 124, 104588.

Downey, A. (2017). Modeling and Simulation in Python. Green Tea Press. (Original work published 2016)

Ferretti, F., Saltelli, A., & Tarantola, S. (2016). Trends in sensitivity analysis practice in the last decade. Science of The Total Environment, 568, 666–670.

Fonseca, J., Thomas, D., Mok, R., Mosteiro-Romero, M., Happle, G., Rogenhofer, L., Jack-Hawthorne, Fazel Khayatian, Zhongming Shi, Riegelbauer, E., Ong, B. L., Orenkiwi, H, T., Paulneitzel, Sulzer, M., Molony, R., Elesawy, A., JOSE ANTONIO BELLO ACOSTA, Bosova, A., … Strusoftsawen. (2021). architecture-building-systems/CityEnergyAnalyst: CityEnergyAnalyst v3.22.0 (v3.22.0) [Computer software]. Zenodo.

Foramitti, J. (2021). JoelForamitti/agentpy [Python]. (Original work published 2020)

Foster, I. (1995). Designing and building parallel programs: Concepts and tools for parallel software engineering. Addison-Wesley.

Hadjimichael, A., Gold, D., Hadka, D., & Reed, P. (2020). Rhodium: Python Library for Many-Objective Robust Decision Making and Exploratory Modeling. Journal of Open Research Software, 8(1), 12.

Hannay, J. E., MacLeod, C., Singer, J., Langtangen, H. P., Pfahl, D., & Wilson, G. (2009). How do scientists develop and use scientific software? 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering, 1–8.

Herlihy, M., & Shavit, N. (2012). The Art of Multiprocessor Programming. Elsevier Science.

Herman, J., & Usher, W. (2017, January 10). SALib: An open-source Python library for Sensitivity Analysis. The Journal of Open Source Software.

Hermans, F. (2021). The Programmer’s Brain: What every programmer needs to know about cognition. Manning Publications.

Iwanaga, T. (2021). ConnectedSystems/SALib-impact: V0.5. Zenodo.

Iwanaga, T., Wang, H.-H., Hamilton, S. H., Grimm, V., Koralewski, T. E., Salado, A., Elsawah, S., Razavi, S., Yang, J., Glynn, P., Badham, J., Voinov, A., Chen, M., Grant, W. E., Peterson, T. R., Frank, K., Shenk, G., Barton, C. M., Jakeman, A. J., & Little, J. C. (2021). Socio-technical scales in socio-environmental modeling: Managing a system-of-systems modeling approach. Environmental Modelling & Software, 104885.

Kelly, D. F. (2007). A Software Chasm: Software Engineering and Scientific Computing. IEEE Software, 24(6), 120–119.

Kwakkel, J. H. (2017). The Exploratory Modeling Workbench: An open source toolkit for exploratory modeling, scenario discovery, and (multi-objective) robust decision making. Environmental Modelling & Software, 96, 239–250.

Li, G., Rabitz, H., Yelvington, P. E., Oluwole, O. O., Bacon, F., Kolb, C. E., & Schoendorf, J. (2010). Global Sensitivity Analysis for Systems with Independent and/or Correlated Inputs. The Journal of Physical Chemistry A, 114(19), 6022–6032.

Little, J. C., Hester, E. T., Elsawah, S., Filz, G. M., Sandu, A., Carey, C. C., Iwanaga, T., & Jakeman, A. J. (2019). A tiered, system-of-systems modeling framework for resolving complex socio-environmental policy issues. Environmental Modelling & Software, 112, 82–94.

Marelli, S., & Sudret, B. (2014). UQLab: A Framework for Uncertainty Quantification in Matlab. Vulnerability, Uncertainty, and Risk, 2554–2563.

McKerns, M. M., Strand, L., Sullivan, T., Fang, A., & Aivazis, M. A. G. (2012). Building a Framework for Predictive Science. ArXiv:1202.1056 [Cs].

Morris, M. D. (1991). Factorial Sampling Plans for Preliminary Computational Experiments. Technometrics, 33(2), 161–174.

Niet, T., Shivakumar, A., Gardumi, F., Usher, W., Williams, E., & Howells, M. (2021). Developing a community of practice around an open source energy modelling tool. Energy Strategy Reviews, 35, 100650.

Niyazov, Y., Vogel, C., Price, R., Lund, B., Judd, D., Akil, A., Mortonson, M., Schwartzman, J., & Shron, M. (2016). Open Access Meets Discoverability: Citations to Articles Posted to PLOS ONE, 11(2), e0148257.

Noacco, V., Sarrazin, F., Pianosi, F., & Wagener, T. (2019). Matlab/R workflows to assess critical choices in Global Sensitivity Analysis using the SAFE toolbox. MethodsX, 6, 2258–2280.

Paleari, L., & Confalonieri, R. (2016). Sensitivity analysis of a sensitivity analysis: We are likely overlooking the impact of distributional assumptions. Ecological Modelling, 340, 57–63.

Pianosi, F., Beven, K., Freer, J., Hall, J. W., Rougier, J., Stephenson, D. B., & Wagener, T. (2016). Sensitivity analysis of environmental models: A systematic review with practical workflow. Environmental Modelling & Software, 79, 214–232.

Pianosi, F., Sarrazin, F., & Wagener, T. (2020). How successfully is open-source research software adopted? Results and implications of surveying the users of a sensitivity analysis toolbox. Environmental Modelling & Software, 124, 104579.

Pianosi, F., & Wagener, T. (2015). A simple and efficient method for global sensitivity analysis based on cumulative distribution functions. Environmental Modelling & Software, 67, 1–11.

Pianosi, F., & Wagener, T. (2018). Distribution-based sensitivity analysis from a generic input-output sample. Environmental Modelling & Software, 108, 197–207.

Plischke, E. (2010). An effective algorithm for computing global sensitivity indices (EASI). Reliability Engineering & System Safety, 95(4), 354–360.

Plischke, E., Borgonovo, E., & Smith, C. L. (2013). Global sensitivity measures from given data. European Journal of Operational Research, 226(3), 536–550.

Puy, A., Piano, S. L., Saltelli, A., & Levin, S. A. (2021). sensobol: An R package to compute variance-based sensitivity indices. ArXiv Preprint ArXiv:2101.10103.

Rabitz, H., Aliş, Ö. F., Shorter, J., & Shim, K. (1999). Efficient input—Output model representations. Computer Physics Communications, 117(1), 11–20.

Razavi, S., & Gupta, H. V. (2015). What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models. Water Resources Research, 51(5), 3070–3092.

Razavi, S., Jakeman, A. J., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., Tarantola, S., Guillaume, J. H. A., Jakeman, J., Gupta, H., Melillo, N., Rabitti, G., Chabridon, V., Duan, Q., Sun, X., … Maier, H. R. (2021). The Future of Sensitivity Analysis: An Essential Discipline for Systems Modeling and Policy Support. Environmental Modelling & Software, 137, 104954.

Razavi, S., Sheikholeslami, R., Gupta, H. V., & Haghnegahdar, A. (2019). VARS-TOOL: A toolbox for comprehensive, efficient, and robust sensitivity and uncertainty analysis. Environmental Modelling & Software, 112, 95–107.

Ruano, M. V., Ribes, J., Seco, A., & Ferrer, J. (2012). An improved sampling strategy based on trajectory design for application of the Morris method to systems with many input factors. Environmental Modelling & Software, 37, 103–109.

Saltelli, A. (2002). Making best use of model evaluations to compute sensitivity indices. Computer Physics Communications, 145, 280–297.

Saltelli, A., Aleksankina, K., Becker, W., Fennell, P., Ferretti, F., Holst, N., Li, S., & Wu, Q. (2019). Why So Many Published Sensitivity Analyses Are False: A Systematic Review of Sensitivity Analysis Practices. Environmental Modelling & Software, 114, 29–39.

Saltelli, A., & Annoni, P. (2010). How to avoid a perfunctory sensitivity analysis. Environmental Modelling and Software, 25(12), 1508–1517.

Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., & Tarantola, S. (2010). Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Computer Physics Communications, 181(2), 259–270.

Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2008). Global Sensitivity Analysis: The Primer. Wiley.

Saltelli, A., Tarantola, S., & Chan, K. P.-S. (1999). A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output. Technometrics, 41(1), 39–56.

Sheikholeslami, R., Gharari, S., Papalexiou, S. M., & Clark, M. P. (2021). VISCOUS: A Variance-Based Sensitivity Analysis Using Copulas for Efficient Identification of Dominant Hydrological Processes. Water Resources Research, 57(7), e2020WR028435.

Sobol′, I. M. (2001). Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Mathematics and Computers in Simulation, 55(1–3), 271–280.

Sobol’, I. M., & Kucherenko, S. (2010). Derivative based global sensitivity measures. Procedia - Social and Behavioral Sciences, 2(6), 7745–7746.

Steiner, M., Bourinet, J.-M., & Lahmer, T. (2019). An adaptive sampling method for global sensitivity analysis based on least-squares support vector regression. Reliability Engineering & System Safety, 183, 323–340.

Sweller, J. (1988). Cognitive Load During Problem Solving: Effects on Learning. Cognitive Science, 12(2), 257–285.

Tarantola, S., Gatelli, D., & Mara, T. A. (2006). Random balance designs for the estimation of first order global sensitivity indices. Reliability Engineering & System Safety, 91(6), 717–727.

Teplitskiy, M., Lu, G., & Duede, E. (2017). Amplifying the impact of open access: Wikipedia and the diffusion of science. Journal of the Association for Information Science and Technology, 68(9), 2116–2127.

Tissot, J.-Y., & Prieur, C. (2012). Bias correction for the estimation of sensitivity indices based on random balance designs. Reliability Engineering & System Safety, 107, 205–213.

Wagener, T., & Pianosi, F. (2019). What has Global Sensitivity Analysis ever done for us? A systematic review to support scientific advancement and to inform policy-making in earth system modelling. Earth-Science Reviews, 194, 1–18.

Wilson, G. (2006). Software Carpentry: Getting Scientists to Write Better Code by Making Them More Productive. Computing in Science Engineering, 8(6), 66–69.

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2022 Takuya Iwanaga, Will Usher, Jon Herman