Supplementary Files
Keywords
exploratory modeling
water management
groundwater
integration
How to Cite
Abstract
Exploratory analysis, while useful in assessing the implications of model assumptions under large uncertainty, is considered at best a semi-structured activity. There is no algorithmic way for performing exploratory analysis and the existing canonical techniques have their own limitations. To overcome this, we advocate a bricolage-style exploratory scenario analysis, which can be crafted by pragmatically and strategically combining different methods and practices. Our argument is illustrated using a case study in integrated water management in the Murray-Darling Basin, Australia. Scenario ensembles are generated to investigate potential policy innovations, climate and crop market conditions, as well as the effects of uncertainties in model components and parameters. Visualizations, regression trees and marginal effect analyses are exploited to make sense of the ensemble of scenarios. The analysis includes identifying patterns within a scenario ensemble, by visualizing initial hypotheses that are informed by prior knowledge, as well as by visualizing new hypotheses based on identified influential variables. Context-specific relationships are explored by analyzing which values of drivers and management options influence outcomes. Synthesis is achieved by identifying context-specific solutions to consider as part of policy design. The process of analysis is cast as a process of finding patterns and formulating questions within the ensemble of scenarios that merit further examination, allowing end-users to make the decision as to what underlying assumptions should be accepted, and whether uncertainties have been sufficiently explored. This approach is especially advantageous when the precise intentions of management are still subject to deliberations. By describing the reasoning and steps behind a bricolage-style exploratory analysis, we hope to raise awareness of the value of sharing this kind of (common but not often documented) analysis process, and motivate further work to improve sharing of know-how about bricolage in practice.
References
Badham, J., Elsawah, S., Guillaume, J.H.A., Hamilton, S.H., Hunt, R.J., Jakeman, A.J., Pierce, S.A., Snow, V.O., Babbar-Sebens, M., Fu, B., Gober, P., Hill, M.C., Iwanaga, T., Loucks, D.P., Merritt, W.S., Peckham, S.D., Richmond, A.K., Zare, F., Ames, D., & Bammer. G. (2019). Effective modeling for Integrated Water Resource Management: a guide to contextual practices by phases and steps and future opportunities. Environmental Modelling & Software, 116, 40-56. doi.org/10.1016/j.envsoft.2019.02.013.
Bankes, S. (1993). Exploratory Modeling for Policy Analysis. Operations Research, 41(3), 435-449. doi:10.1287/opre.41.3.435
Bankes, S., Walker, W. E., & Kwakkel, J. H. (2013). Exploratory modeling and analysis. In Encyclopedia of operations research and management science (pp. 532-537): Springer.
Bankes, S. C. (2002). Tools and techniques for developing policies for complex and uncertain systems. Proceedings of the National Academy of Sciences, 99(suppl 3), 7263-7266. doi:10.1073/pnas.092081399
Beck, M. B. (1987). Water quality modeling: a review of the analysis of uncertainty. Water Resources Research, 23(8), 1393-1442. doi:10.1029/WR023i008p01393
Beh, E. H. Y., Maier, H. R., & Dandy, G. C. (2015). Adaptive, multiobjective optimal sequencing approach for urban water supply augmentation under deep uncertainty. Water Resources Research, 51(3), 1529-1551. doi:10.1002/2014WR016254
Ben-Haim, Y. (2006). Info-Gap Decision Theory (Second Edition). Oxford: Academic Press.
Bennett, N. D., Croke, B. F. W., Guariso, G., Guillaume, J. H. A., Hamilton, S. H., Jakeman, A. J.,Marsili-Libelli, S., Newham, L.T.H., Norton, J.P., Perrin, C., Pierce, S.A., Robson, B., Seppelt, R., Voinov, A.A., Fath, B., & Andreassian, V. (2013). Characterising performance of environmental models. Environmental Modelling & Software, 40, 1-20. doi:10.1016/j.envsoft.2012.09.011
Blakers, R., Croke, B., & Jakeman, A. (2011). The influence of model simplicity on uncertainty in the context of surface–groundwater modelling and integrated assessment. Paper presented at the 19th International Congress on Modelling and Simulation, Perth, Australia.
Brown, C., Ghile, Y., Laverty, M., & Li, K. (2012). Decision scaling: Linking bottom-up vulnerability analysis with climate projections in the water sector. Water Resources Research, 48(9), n/a-n/a. doi:10.1029/2011WR011212
Bryant, B. P. (2014). sdtoolkit: Scenario Discovery Tools to Support Robust Decision Making (v2.33-1). Retrieved from cran.r-project.org/web/packages/sdtoolkit/index.html
Bryant, B. P., & Lempert, R. J. (2010). Thinking inside the box: A participatory, computer-assisted approach to scenario discovery. Technological Forecasting and Social Change, 77(1), 34-49. doi:10.1016/j.techfore.2009.08.002
Buurman, J., & Babovic, V. (2016). Adaptation Pathways and Real Options Analysis: An approach to deep uncertainty in climate change adaptation policies. Policy and Society, 35(2), 137-150. doi:10.1016/j.polsoc.2016.05.002
Chiew, F. H. S., Zhou, S. L., & McMahon, T. A. (2003). Use of seasonal streamflow forecasts in water resources management. Journal of Hydrology, 270(1–2), 135-144. doi:10.1016/S0022-1694(02)00292-5
Croke, B., Ticehurst, J. L., Letcher, R., Norton, J., Newham, L., & Jakeman, A. (2007). Integrated assessment of water resources: Australian experiences. Water Resources Management, 21(1), 351-373. doi:10.1007/s11269-006-9057-8
Croke, B. F., & Jakeman, A. J. (2004). A catchment moisture deficit module for the IHACRES rainfall-runoff model. Environmental Modelling & Software, 19(1), 1-5. doi:10.1016/j.envsoft.2003.09.001
D’Angelo, G., & Marzolla, M. (2014). New trends in parallel and distributed simulation: From many-cores to Cloud Computing. Simulation Modelling Practice and Theory, 49, 320-335. doi:10.1016/j.simpat.2014.06.007
Daggupati, P., Pai, N., Ale, S., Douglas-Mankin, K. R., Zeckoski, R. W., Jeong, J., . . . Youssef, M. A. (2015). A recommended calibration and validation strategy for hydrologic and water quality models. Transactions of the ASABE, 58(6), 1705-1719. doi:10.13031/trans.58.10712
De'Ath, G., & Fabricius, K. E. (2000). Classification and regression trees: a powerful yet simple technique for ecological data analysis. Ecology, 81, 3178-3192. doi:10.1890/0012-9658(2000)081[3178:CARTAP]2.0.CO
Fayyad, U. M., Wierse, A., & Grinstein, G. G. (2002). Information visualization in data mining and knowledge discovery: Morgan Kaufmann.
Fu, B., Dyer, F., Kravchenko, A., Dyack, B., Merritt, W., & Scarpa, R. (2017). A note on communicating environmental change for non-market valuation. Ecological Indicators, 72, 165-172. doi:10.1016/j.ecolind.2016.08.018
Fu, B., & Guillaume, J. (2018). A Bricolage Style Exploratory Scenario Analysis to Manage Uncertainty - Analysis Code and Data (Version v1.0). Retrieved from http://doi.org/10.5281/zenodo.1465093
Fu, B., & Guillaume, J. H. (2014). Assessing certainty and uncertainty in riparian habitat suitability models by identifying parameters with extreme outputs. Environmental Modelling & Software, 60, 277-289. doi:10.1016/j.envsoft.2014.06.015
Fu, B., Guillaume, J. H. A., & Jakeman, A. J. (2015). An iterative method for discovering feasible management interventions and targets conjointly using uncertainty visualizations. Environmental Modelling & Software, 71, 159-173. doi:10.1016/j.envsoft.2015.05.017
Gersonius, B., Ashley, R., Jeuken, A., Pathinara, A., & Zevenbergen, C. (2015). Accounting for uncertainty and flexibility in flood risk management: comparing R eal‐I n‐O ptions optimisation and A daptation T ipping P oints. Journal of Flood Risk Management, 8(2), 135-144.
Gil, Y., David, C. H., Demir, I., Essawy, B. T., Fulweiler, R. W., Goodall, J. L., . . . Yu, X. (2016). Toward the Geoscience Paper of the Future: Best practices for documenting and sharing research from data to software to provenance. Earth and Space Science, 3(10), 388-415. doi:doi:10.1002/2015EA000136
Giuliani, M., & Castelletti, A. (2016). Is robustness really robust? How different definitions of robustness impact decision-making under climate change. Climatic Change, 135(3-4), 409-424. doi:10.1007/s10584-015-1586-9
Giuliani, M., Herman, J., Castelletti, A., & Reed, P. (2014). Many‐objective reservoir policy identification and refinement to reduce policy inertia and myopia in water management. Water Resources Research, 50(4), 3355-3377. doi:10.1002/2013WR014700
Guillaume, J. H. A., Arshad, M., Jakeman, A. J., Jalava, M., & Kummu, M. (2016). Robust discrimination between uncertain management alternatives by iterative reflection on crossover point scenarios: Principles, design and implementations. Environmental Modelling & Software, 83, 326-343. doi:10.1016/j.envsoft.2016.04.005
Guillaume, J. H. A., Kummu, M., Räsänen, T. A., & Jakeman, A. J. (2015). Prediction under uncertainty as a boundary problem: A general formulation using Iterative Closed Question Modelling. Environmental Modelling & Software, 70, 97-112. doi:10.1016/j.envsoft.2015.04.004
Guzman, J. A., Shirmohammadi, A., Sadeghi, A. M., Wang, X., Chu, M. L., Jha, M. K., . . . Hernandez, J. E. (2015). Uncertainty considerations in calibration and validation of hydrologic and water quality models. Transactions of the ASABE, 58(6), 1745-1762. doi:10.13031/trans.58.10710
Haasnoot, M., van Deursen, W. P. A., Guillaume, J. H. A., Kwakkel, J. H., van Beek, E., & Middelkoop, H. (2014). Fit for purpose? Building and evaluating a fast, integrated model for exploring water policy pathways. Environmental Modelling & Software, 60, 99-120. doi:10.1016/j.envsoft.2014.05.020
Hadka, D. (2016). High Performance Computing with the MOEA Framework and Ignite: CreateSpace Independent Publishing Platform.
Hadka, D. (2019). Project Platypus - A Collection of Libraries for Optimization, Data Analysis, and Decision Making. Retrieved from https://github.com/Project-Platypus
Hadka, D., Herman, J., Reed, P., & Keller, K. (2015). An open source framework for many-objective robust decision making. Environmental Modelling & Software, 74, 114-129. doi:10.1016/j.envsoft.2015.07.014
Hall, J. W., Lempert, R. J., Keller, K., Hackbarth, A., Mijere, C., & McInerney, D. J. (2012). Robust Climate Policies Under Uncertainty: A Comparison of Robust Decision Making and Info-Gap Methods. Risk Analysis, 32(10), 1657-1672. doi:10.1111/j.1539-6924.2012.01802.x
Hamarat, C., Kwakkel, J. H., & Pruyt, E. (2013). Adaptive robust design under deep uncertainty. Technological Forecasting and Social Change, 80(3), 408-418. doi:10.1016/j.techfore.2012.10.004
Herman, J., Reed, P., Zeff, H., & Characklis, G. (2015). How Should Robustness Be Defined for Water Systems Planning under Change? Journal of Water Resources Planning and Management, 04015012. doi:10.1061/(ASCE)WR.1943-5452.0000509
Hodges, J. S. (1991). Six (or so) things you can do with a bad model. Operations Research, 39(3), 355-365. doi:10.1287/opre.39.3.355
Hodges, J. S., & Dewar, J. A. (1992). Is it you or your model talking?: A framework for model validation: Rand Santa Monica, CA.
Hothorn, T., Hornik, K., & Zeileis, A. (2006). Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical statistics, 15(3), 651-674. doi:10.1198/106186006X133933
Hu, Y., Garcia-Cabrejo, O., Cai, X., Valocchi, A. J., & DuPont, B. (2015). Global sensitivity analysis for large-scale socio-hydrological models using Hadoop. Environmental Modelling & Software, 73, 231-243. doi:10.1016/j.envsoft.2015.08.015
Ivkovic, K., Croke, B., Letcher, R., & Evans, W. (2005). The development of a simple model to investigate the impact of groundwater extraction on river flows in the Namoi catchment, NSW, Australia. Paper presented at the Proceedings of New Zealand Hydrological Society-IAH-NSSSS Conference 2005.
Jakeman, A., & Hornberger, G. (1993). How much complexity is warranted in a rainfall‐runoff model? Water Resources Research, 29(8), 2637-2649. doi:10.1029/93WR00877
Jakeman, A., Littlewood, I., & Whitehead, P. (1990). Computation of the instantaneous unit hydrograph and identifiable component flows with application to two small upland catchments. Journal of Hydrology, 117(1-4), 275-300. doi:10.1016/0022-1694(90)90097-H
Jaxa-Rozen, M., & Kwakkel, J. H. (2018). PyNetLogo: Linking NetLogo with Python. Journal of Artificial Societies and Social Simulation, 21(2), 4. doi:10.18564/jasss.3668
Kasprzyk, J., Guillaume, J., Kollat, J., & Danilo, C. (2014, 2014). Hypothesis Testing for Management: Evolving and Answering Closed Questions Using Multiobjective Visualization. Paper presented at the International Congress on Environmental Modelling and Software, San Diego, California, USA.
Kasprzyk, J. R., Nataraj, S., Reed, P. M., & Lempert, R. J. (2013). Many objective robust decision making for complex environmental systems undergoing change. Environmental Modelling & Software, 42, 55-71. doi:10.1016/j.envsoft.2012.12.007
Kehrer, J., & Hauser, H. (2013). Visualization and Visual Analysis of Multifaceted Scientific Data: A Survey. IEEE Transactions on Visualization and Computer Graphics, 19(3), 495-513. doi:10.1109/TVCG.2012.110
Kincheloe, J. L. (2001). Describing the Bricolage: Conceptualizing a New Rigor in Qualitative Research. Qualitative inquiry, 7(6), 679-692. doi:10.1177/107780040100700601
Kincheloe, J. L. (2005). On to the next level: Continuing the conceptualization of the bricolage. Qualitative inquiry, 11(3), 323-350. doi:10.1177/1077800405275056
Kwadijk, J. C., Haasnoot, M., Mulder, J. P., Hoogvliet, M., Jeuken, A., van der Krogt, R. A., . . . van Waveren, H. (2010). Using adaptation tipping points to prepare for climate change and sea level rise: a case study in the Netherlands. Wiley Interdisciplinary Reviews: Climate Change, 1(5), 729-740. doi:10.1002/wcc.64
Kwakkel, J. H. (2017). The Exploratory Modeling Workbench: An open source toolkit for exploratory modeling, scenario discovery, and (multi-objective) robust decision making. Environmental Modelling & Software, 96, 239-250. doi:10.1016/j.envsoft.2017.06.054
Kwakkel, J. H., Eker, S., & Pruyt, E. (2016). How Robust is a Robust Policy? Comparing Alternative Robustness Metrics for Robust Decision-Making. In M. Doumpos, C. Zopounidis, & E. Grigoroudis (Eds.), Robustness Analysis in Decision Aiding, Optimization, and Analytics (pp. 221-237). Cham: Springer International Publishing.
Kwakkel, J. H., & Haasnoot, M. (2019). Supporting DMDU: A taxonomy of approaches and tools. In Decision Making under Deep Uncertainty (pp. 355-374): Springer.
Kwakkel, J. H., Haasnoot, M., & Walker, W. E. (2015). Developing dynamic adaptive policy pathways: a computer-assisted approach for developing adaptive strategies for a deeply uncertain world. Climatic Change, 132(3), 373-386. doi:10.1007/s10584-014-1210-4
Kwakkel, J. H., Haasnoot, M., & Walker, W. E. (2016). Comparing robust decision-making and dynamic adaptive policy pathways for model-based decision support under deep uncertainty. Environmental Modelling & Software, 86, 168-183. doi:10.1016/j.envsoft.2016.09.017
Kwakkel, J. H., & Pruyt, E. (2013). Exploratory Modeling and Analysis, an approach for model-based foresight under deep uncertainty. Technological Forecasting and Social Change, 80(3), 419-431. doi:10.1016/j.techfore.2012.10.005
Lahtinen, T. J., Guillaume, J. H., & Hämäläinen, R. P. (2017). Why pay attention to paths in the practice of environmental modelling? Environmental Modelling & Software, 92, 74-81. doi:10.1016/j.envsoft.2017.02.019
Landry, M., Banville, C., & Oral, M. (1996). Model legitimisation in operational research. European Journal of Operational Research, 92(3), 443-457. doi:10.1016/0377-2217(96)00003-3
Lempert, R. J. (2002). A new decision sciences for complex systems. Proceedings of the National Academy of Sciences, 99(suppl 3), 7309-7313. doi:10.1073/pnas.082081699
Lempert, R. J., Bryant, B. P., & Bankes, S. C. (2008). Comparing algorithms for scenario discovery. RAND, Santa Monica, CA.
Lempert, R. J., & Collins, M. T. (2007). Managing the risk of uncertain threshold responses: comparison of robust, optimum, and precautionary approaches. Risk Analysis, 27(4), 1009-1026. doi:10.1111/j.1539-6924.2007.00940.x
Lempert, R. J., Popper, S. W., & Bankes, S. C. (2003). Shaping the next one hundred years: new methods for quantitative, long-term policy analysis (0833034855). Retrieved from Santa Monica, California, USA:
Letcher, R., Jakeman, A., & Croke, B. (2004). Model development for integrated assessment of water allocation options. Water Resources Research, 40(5). doi:10.1029/2003WR002933
Maier, H. R., Guillaume, J. H. A., van Delden, H., Riddell, G. A., Haasnoot, M., & Kwakkel, J. H. (2016). An uncertain future, deep uncertainty, scenarios, robustness and adaptation: How do they fit together? Environmental Modelling & Software, 81, 154-164. doi:10.1016/j.envsoft.2016.03.014
Matrosov, E. S., Padula, S., & Harou, J. J. (2013). Selecting portfolios of water supply and demand management strategies under uncertainty—contrasting economic optimisation and ‘robust decision making’approaches. Water Resources Management, 27(4), 1123-1148. doi:10.1007/s11269-012-0118-x
McPhail, C., Maier, H., Kwakkel, J., Giuliani, M., Castelletti, A., & Westra, S. (2018). Robustness metrics: How are they calculated, when should they be used and why do they give different results? Earth's Future, 6(2), 169-191. doi:10.1002/2017EF000649
Moallemi, E. A., Elsawah, S., & Ryan, M. J. (2018). Model-based multi-objective decision making under deep uncertainty from a multi-method design lens. Simulation Modelling Practice and Theory, 84, 232-250. doi:10.1016/j.simpat.2018.02.009
Moriasi, D. N., Arnold, J. G., Van Liew, M. W., Bingner, R. L., Harmel, R. D., & Veith, T. L. (2007). Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Transactions of the ASABE, 50(3), 885-900. doi:10.13031/2013.23153
Moriasi, D. N., Zeckoski, R. W., Arnold, J. G., Baffaut, C., Malone, R. W., Daggupati, P., . . . Wilson, B. N. (2015). Hydrologic and water quality models: Key calibration and validation topics. Transactions of the ASABE, 58(6), 1609-1618. doi:10.13031/trans.58.11075
Murray–Darling Basin Authority. (2010). Guide to the proposed Basin Plan. Retrieved from Canberra: https://www.mdba.gov.au/publications/archived-information/basin-plan-archives/guide-proposed-basin-plan
Murray Irrigation Limited. (2010). Response to the Guide to the Proposed Murray-Darling Basin Plan. Retrieved from http://www.murrayirrigation.com.au/media/2503/Submission%20to%20the%20MDBA%20re%20Guide%20to%20the%20Proposed%20Murray-Darling%20Basin%20Plan.pdf
Norton, J. (2015). An introduction to sensitivity assessment of simulation models. Environmental Modelling & Software, 69, 166-174. doi:10.1016/j.envsoft.2015.03.020
Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences. Science, 263(5147), 641-646. doi:10.1126/science.263.5147.641
Pappenberger, F., & Beven, K. J. (2006). Ignorance is bliss: Or seven reasons not to use uncertainty analysis. Water Resources Research, 42(5), n/a-n/a. doi:10.1029/2005WR004820
Pollino, C., Lester, R., Podger, G., Black, D., & Overton, I. (2011). Analysis of South Australia’s environmental water and water quality requirements and their delivery under the Guide to the proposed Basin Plan. Retrieved from Adelaide:
Refsgaard, J. C., & Knudsen, J. (1996). Operational Validation and Intercomparison of Different Types of Hydrological Models. Water Resources Research, 32(7), 2189-2202. doi:10.1029/96WR00896
Refsgaard, J. C., van der Sluijs, J. P., Højberg, A. L., & Vanrolleghem, P. A. (2007). Uncertainty in the environmental modelling process–a framework and guidance. Environmental Modelling & Software, 22(11), 1543-1556. doi:10.1016/j.envsoft.2007.02.004
Reichert, P., & Borsuk, M. E. (2005). Does high forecast uncertainty preclude effective decision support? Environmental Modelling & Software, 20(8), 991-1001. doi:10.1016/j.envsoft.2004.10.005
Roach, T., Kapelan, Z., & Ledbetter, R. (2015). Comparison of info-gap and robust optimisation methods for integrated water resource management under severe uncertainty. Procedia Engineering, 119, 874-883. doi:10.1016/j.proeng.2015.08.955
Roach, T., Kapelan, Z., Ledbetter, R., & Ledbetter, M. (2016). Comparison of robust optimization and info-gap methods for water resource management under deep uncertainty. Journal of Water Resources Planning and Management, 142(9), 04016028. doi:10.1061/(ASCE)WR.1943-5452.0000660
Roberts, J., & Marston, F. (2011). Water Regime of Wetland and Floodplain Plants: a Source Book for the Murray-Darling Basin. Canberra: National Water Commission.
Rogers, K., & Ralph, T. (2010). Floodplain Wetland Biota in the Murray-Darling Basin. Collingwood: CSIRO Publishing.
Saltelli, A., Chan, K., & Scott, E. M. (2000). Sensitivity analysis (Vol. 1): Wiley New York.
Saltelli, A., Tarantola, S., & Campolongo, F. (2000). Sensitivity analysis as an ingredient of modeling. Statistical Science, 15(4), 377-395.
Schoemaker, P. J. (1995). Scenario planning: a tool for strategic thinking. Sloan management review, 36(2), 25.
Strobl, C., Boulesteix, A.-L., Zeileis, A., & Hothorn, T. (2007). Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 8(1), 25. doi:10.1186/1471-2105-8-25
Ticehurst, J. L., & Curtis, A. L. (2015). Can existing practices expected to lead to improved on-farm water use efficiency enable irrigators to effectively respond to reduced water entitlements in the Murray–Darling Basin? Journal of Hydrology, 528, 613-620. doi:10.1016/j.jhydrol.2015.06.055
Tsoukiàs, A. (2008). From decision theory to decision aiding methodology. European Journal of Operational Research, 187(1), 138-161.
Tufte, E. R. (2001). The visual display of quantitative information (Vol. 2): Graphics press Cheshire, CT.
Vaishnavi, V. K., & Kuechler, W. (2007). Using Patterns to Illuminate Research Practice. In Design Science Research Methods and Patterns (pp. 57-73): Auerbach Publications.
van Asselt, M., & Rotmans, J. (1996). Uncertainty in perspective. Global Environmental Change, 6(2), 121-157. doi:10.1016/0959-3780(96)00015-5
Walker, W., Haasnoot, M., & Kwakkel, J. (2013). Adapt or Perish: A Review of Planning Approaches for Adaptation under Deep Uncertainty. Sustainability, 5(3), 955. doi:10.3390/su5030955
Weimer-Jehle, W. (2006). Cross-impact balances: A system-theoretical approach to cross-impact analysis. Technological Forecasting and Social Change, 73(4), 334-361. doi:10.1016/j.techfore.2005.06.005
Wickham, H. (2009). ggplot2: Elegant Graphics for Data Analysis: Springer-Verlag New York,.
Williams, B. K. (2011). Passive and active adaptive management: Approaches and an example. Journal of Environmental Management, 92(5), 1371-1378. doi:10.1016/j.jenvman.2010.10.039
Yee, J. S. R. (2010). Methodological Innovation in Practice-Based Design Doctorates. Journal of Research Practice, 6(2), 15-15.
Zhang, M., Yue, P., Wu, Z., Ziebelin, D., Wu, H., & Zhang, C. (2017). Model provenance tracking and inference for integrated environmental modelling. Environmental Modelling & Software, 96, 95-105. doi:https://doi.org/10.1016/j.envsoft.2017.06.051
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) 2020 Baihua Fu, Joseph H.A. Guillaume, Anthony J Jakeman, Michael J. Asher