Towards normalizing good practice across the whole modeling cycle: its instrumentation and future research topics
Article Full Text (PDF)

Keywords

good modelling practice
modelling lifecycle
modelling choices
fit for purpose

How to Cite

Jakeman, A. J., Elsawah, S., Wang, H.-H., Hamilton, S. H., Melsen, L., & Grimm, V. (2024). Towards normalizing good practice across the whole modeling cycle: its instrumentation and future research topics. Socio-Environmental Systems Modelling, 6, 18755. https://doi.org/10.18174/sesmo.18755

Abstract

Choices made in modeling matter and demand more explication since they determine how much we can trust modeling insights and predictions within their social, political and ethical contexts. Good Modeling Practice (GMP) is a key research area for strengthening and maturing the modeling field and community, through identifying, formulating and sharing knowledge about the craft of modeling. This craft represents the knowledge that modelers learn in practice about how they get things done, and how they adapt their practices to new situations. This Joint Special Issue is motivated by the importance of sharing good modeling practices from a whole modeling lifecycle viewpoint. We attempt to add conceptual clarity to this research area by defining the plethora of concepts and decision points used to characterize the choices to be made throughout the modeling process, and by synthesizing some of the existing efforts on GMP. We characterize a broad list of articles in the literature on GMP and identify a list of essential topics demanding more attention. This list is only a preliminary one as we anticipate that a more comprehensive list of knowledge gaps will be unearthed from the submissions to the Joint Special Issue collection on GMP, of which this is an introduction. We also propose a vision for GMP and suggest instrumental ways that good practice can become not just well-known but normal practice. This instrumentation focuses on journal standards, collective commitment and culture especially by research community societies, early career awards for advancing GMP, and legal requirements or accreditation. A vital instrument in all this is the design and development of a modeling curriculum that distills core requisite knowledge about modeling, as well as proven-to-work routines and practices that can be scaled up in different contexts.

Article Full Text (PDF)

References

Addor, N. & Melsen, L.A., 2019. Legacy, rather than adequacy, drives the selection of hydrological models. Water Resources Research, 55, 378-390.

Alexander, P., Prestele, R., Verburg, P.H., Arneth, A., Baranzelli, C., Batista e Silva, F., Brown, C., Butler, A., Calvin, K., Dendoncker, N., Doelman, J.C., Dunford, R., Engström, K., Eitelberg, D., Fujimori, S., Harrison, P.A., Hasegawa, T., Havlik, P., Holzhauer, S., Humpenöder, F., Jacobs-Crisioni, C., Jain, A.K., Krisztin, T., Kyle, P., Lavalle, C., Lenton, T., Liu, J., Meiyappan, P., Popp, A., Powell, T., Sands, R.D., Schaldach, R., Stehfest, E., Steinbuks, J., Tabeau, A., van Meijl, H., Wise, M.A., & Rounsevell, M.D.A. (2017). Assessing uncertainties in land cover projections. Global Change Biology, 23(2), 767-781.

Argent, R.M., Sojda, R.S., Guipponi, C., McIntosh, B., Voinov, A.A., & Maier, H.R. (2016) Best practices for conceptual modeling in environmental planning and management. Environmental Modeling and Software, 80, 113-121.

Babel, L., Vinck, D. & Karssenberg, D., 2019. Decision-making in model construction: Unveiling habits. Environmental Modeling & Software, 120, 104490.

Badham, J., Elsawah, S., Guillaume, J.H.A., Hamilton, S.H., Hunt, R.J., Jakeman, A.J., Pierce, S.A., Snow, V.O., Babbar-Sebens, M., Fu, B., Gober, P., Hill, M.C., Iwanaga, T., Loucks, D.P., Merritt, W.S., Peckham, S.D., Richmond, A.K., Zare, F., Ames, D., & Bammer, G. (2019) Effective modeling for Integrated Water Resource Management: A guide to contextual practices by phases and steps and future opportunities. Environmental Modelling & Software 116, 40-56.

Bahlburg, D., Thorpe, S. E., Meyer, B., Berger, U., & Murphy, E. J. (2023). An intercomparison of models predicting growth of Antarctic krill (Euphausia superba): The importance of recognizing model specificity. PLoS One, 18(7), e0286036.

Banitz, T., Schlüter, M., Lindkvist, E., Radosavljevic, S., Johansson, L. G., Ylikoski, P., Martínez-Peña, R., & Grimm, V. (2022). Model-derived causal explanations are inherently constrained by hidden assumptions and context: The example of Baltic cod dynamics. Environmental Modeling & Software, 156, 105489.

Barnett, B., Townley, L.R., Post, V., Evans, R.E., Hunt, R.J., Peeters, L., Richardson, S., Werner, A.D., Knapton, A. & Boronkay, A. (2012) Australian groundwater modelling guidelines, Waterlines report, National Water Commission, Canberra.

Barton, C.M., Ames, D., Chen, M., Frank, K., Jagers, H.R.A., Lee, A., Reis, S. and Swantek, L. (2022) Making modeling and software FAIR. Environmental Modeling and Software, 156, 105496.

Bennett, N.D., Croke, B.F.W., Guariso, G., Guillaume, J.H.A., Hamilton, S.H., Jakeman, A.J., Marsilli-Libelli, S., Newham, L.T.H., Norton, J.P., Perrin, C., Pierce, S.A., Robson, B., Seppelt, R., Voinov, A.A., Fath, B.D., & Andreassian, V. (2013) Characterising Performance of Environmental Models. Environmental Modeling and Software, 40, 1-20.

Beven, K. (2018) Environmental modeling: an uncertain future? CRC press.

Black, D.C., Wallbrink, P.J., & Jordan, P.W. (2014) Towards best practice implementation and application of models for analysis of water resource management scenarios. Environmental Modeling and Software, 52, 136-148

Boorman, D., Williams, R., Hutchins, M., Penning, E.. Groot, S. & Icke, J. (2007) A model selection protocol to support the use of models for water management. Hydrology and Earth System Sciences, 11, 634-646.

Cash, D.W, & Belloy, P.G. (2020) Salience, credibility and legitimacy in a rapidly shifting world of knowledge and action. Sustainability, 12(18), 7376.

Chen, S.H., & Pollino, C.A. (2012) Good practice in Bayesian network modeling. Environmental Modeling and Software, 37, 134-145.

Crout, N., Kokkonen, T., Jakeman, A.J., Norton, J.P., Newham, L.T.H., Anderson, R., Assaf, H., Croke, B.F.W., Gaber, N., Gibbons, J. & Holzworth, D. (2008) Good modeling practice. In: Jakeman, A.J.J., Voinov, A.A., Rizzoli, A.E., & Chen, S.H. (eds.) Developments in Integrated Environmental Assessment, 3, 15-31.

Di Fiore, M., Kuc-Czarnecka, M., Lo Piano, S., Puy, A. & Saltelli, A. (2023) Mind the framings: matching purpose and context. In: Saltelli, A., & Di Fiore, M. (Eds.) The politics of modeling: Numbers between science and policy. Oxford University Press, 37-50.

Elsawah, S., Filatova, T., Jakeman, A.J., Kettner, A.J., Zellner, M.L., Athanasiadis, I.N., Hamilton, S.H., Axtell, R.L., Brown, D.G., Gilligan, J.M., Janssen, M.A., Robinson, D.T., Rozenberg, J., Ullah, I.I.T., & Lade, S.J. (2020) Eight grand challenges in socio-environmental systems modelling. Socio-Environmental Systems Modelling 2, 16226.

Elsawah, S., Pierce, S., Hamilton, S.H., van Delden, H., Haase, D., Elmahdi, A. & Jakeman, A.J. (2017) An overview of the system dynamics process for integrated modeling of socio-ecological systems: Lessons on good modeling practice from five case studies. Environmental Modeling and Software 93, 127-145.

Gaber, N., Foley, G., Pascual, P., Stiber, N., Sunderland, E., Cope, B., Nold, A., & Saleem, Z. (2009) Guidance on the Development, Evaluation, and Application of Environmental Models. US Environmental Protection Agency, EPA/100/K-09/003. https://nepis.epa.gov/Exe/ZyPDF.cgi?Dockey=P1003E4R.PDF

Glaser, D., & Bridges, T.S. (2007) Separating the wheat from the chaff: the effective use of mathematical models as decision tools. Integrated Environmental Assessment and Management, 3(3), 442-449.

Glynn, P.D., Voinov, A.A., Shapiro, C.D., & White, P.A. (2017) From data to decisions: Processing information, biases, and beliefs for improved management of natural resources and environments. Earth's Future, 5(4), 356-378.

Grimm, V. (2023) Ecology needs to overcome siloed modeling. Trends in Ecology and Evolution, 38, 1122-1124.

Grimm, V., & Berger, U. (2016). Robustness analysis: Deconstructing computational models for ecological theory and applications. Ecological Modeling, 326, 162-167.

Grimm, V., & Railsback, S. F. (2005). Individual-based modeling and ecology. Princeton University Press, Princeton N.J.

Grimm, V., Augusiak, J., Focks, A., Frank, B.M., Gabsi, F., Johnston, A.S.A., Liu, C., Martin, B.T., Meli, M., Radchuk, V., Thorbek, P., & Railsback, S.F. (2014). Towards better modeling and decision support: Documenting model development, testing, and analysis using TRACE. Ecological Modeling, 280, 129-139.

Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S., Huse, G., Huth, A., Jepsen, J.U., Jørgensen, C., Mooij, W.M., Müller, B., Pe'er, G., Piou, C., Railsback, S.F., Robbins, A. M., Robbins, M.M., Rossmanith, E., Rüger, N., Strand, E., Souissi, S., Stillman, R. A., Vabø, R., Visser, U., & DeAngelis, D.L. (2006) A standard protocol for describing individual-based and agent-based models. Ecological Modeling 198, 115–126.

Grimm, V., Berger, U., DeAngelis, D. L., Polhill, J. G., Giske, J., & Railsback, S. F. (2010). The ODD protocol: a review and first update. Ecological Modeling, 221(23), 2760-2768.

Grimm, V., Railsback, S.F., Vincenot, C.E., Berger, U., Gallagher, C., DeAngelis, D.L., Edmonds, B., Ge, J., Giske, J., Groeneveld, J., Johnston, A.S.A., Milles, A., Nabe-Nielsen, J., Polhill, J.G., Radchuk, V., Rohwäder, M.-S., Stillman, R.A., Thiele, J.C., & Ayllón, D. (2020) The ODD Protocol for Describing Agent-Based and Other Simulation Models: A Second Update to Improve Clarity, Replication, and Structural Realism. Journal of Artificial Societies and Social Simulation, 23(2), 7.

Hamilton, S.H., ElSawah, S., Guillaume, J.H.A., & Jakeman, A.J. (2015) Integrated assessment and modeling: a review and synthesis of salient dimensions. Environmental Modeling and Software, 64, 215-229.

Hamilton, S.H., Fu, B., Guillaume, J.H.A., Badham, J., Elsawah, S., Gober, P., & Zare, F., 2019. A framework for characterising and evaluating the effectiveness of environmental modeling. Environmental Modeling and Software, 118, 83–98.

Hamilton, S.H., Pollino, C.A., Stratford, D.S., Fu, B., Jakeman, A.J. (2022) Fit-for-purpose environmental modeling: Targeting the intersection of usability, reliability and feasibility. Environmental Modelling and Software, 148, 105278.

Harmel, R.D., Smith, P.K., Migliaccio, K.W., Chaubey, I., Douglas-Mankin, K.R., Benham, B., Shukla, S., Muñoz-Carpena, R., & Robson, B.J. (2014) Evaluating, interpreting, and communicating performance of hydrologic/water quality models considering intended use: A review and recommendations. Environmental Modelling and Software, 57, 40-51.

Henckens, G., & Engel, W. (2017). Benchmark inundatiemodellen: modelfunctionaliteiten en testbank berekeningen. Report no. 34. Amersfoort: STOWA.

HM Treasury (2015). The Aqua Book: guidance on producing quality analysis for government. HM Government, London, UK, https://www.gov.uk/government/publications/the-aqua-book-guidance-on-producing-quality-analysis-for-government (accessed July 10, 2024).

Holländer, H.M., Blume, T., Bormann, H., Buytaert, W., Chirico, G.B., Exbrayat, J.F., Gustafsson, D., Hölzel, H., Kraft, P., Stamm, C., & Stoll, S. (2009). Comparative predictions of discharge from an artificial catchment (Chicken Creek) using sparse data. Hydrology and Earth System Sciences, 13(11), 2069-2094.

Hulsbeek, J.J.W., Kruit, J., Roeleveld, P.J. and Van Loosdrecht, M.C.M., (2002). A practical protocol for dynamic modeling of activated sludge systems. Water Science and Technology, 45(6), pp.127-136.

Iwanaga, T., Wang, H.-H., Hamilton, S.H., Grimm, V., Koralewski, T.E., Salado, A., Elsawah, S., Razavi, S., Yang, J., Glynn, P. Badham, J., Voinov, A., Chen, M., Grant, W.E., Petersen, T.T., Frank, K., Shenk, G., Barton, C.M., Jakeman, A.J., & Little, J.C. (2021a). Socio-technical scales in socio-environmental modeling: Managing a system-of-systems modeling approach. Environmental Modeling and Software, 135, 104885.

Iwanaga, T., Wang, H.H., Koralewski, T.E., Grant, W.E., Jakeman, A.J., & Little, J.C. (2021b) Toward a complete interdisciplinary treatment of scale: Reflexive lessons from socioenvironmental systems modeling. Elementa: Science of the Anthropocene, 9(1), 00182.

Jacobs, B., Tobi, H. and Hengeveld, G.M., 2024. Linking error measures to model questions. Ecological Modelling, 487, p.110562.

Jakeman, A.J., Letcher, R.A., & Norton, J.P. (2006) Ten iterative steps in development and evaluation of environmental models. Environmental Modeling and Software, 21, 602-614.

Jones, H. F., Özkundakci, D., Hunt, S., Giles, H., & Jenkins, B. (2020). Bridging the gap: A strategic framework for implementing best practice guidelines in environmental modeling. Environmental Science & Policy, 114, 533-541.

Kelly, R.A., Jakeman, A.J., Barreteau, O., Borsuk, M.E., ElSawah, S., Hamilton, S.H., Henriksen, H.J., Kuikka, S., Maier, H.R., Rizzoli, A.E., van Delden, H., & Voinov, A.A. (2013). Selecting among five common modelling approaches for integrated environmental assessment and management. Environmental Modelling and Software 47, 159-181.

Klein, A., Unverzagt, K., Alba, R., Donges, J., Hertz, T., Krueger, T., Lindkvist, E., Martin, R., Niewöhner, J., Prawitz, H., Schlüter, M., Schwarz, L., & Wijermans, N. (2024) From situated knowledges to situated modelling: a relational framework for simulation modelling. Ecosystems and People, 20(1), 2361706.

Lahtinen, T.J., Guillaume, J.H.A., & Hämäläinen, R.P. (2017). Why pay attention to paths in the practice of environmental modeling? Environmental Modeling and Software, 92, 74–81.

Lempert, R. J., & Turner, S. (2021). Engaging multiple worldviews with quantitative decision support: A robust decision‐making demonstration using the lake model. Risk Analysis, 41(6), 845-865.

Macpherson, E., Cuppari, R.I., Kagawa‐Viviani, A., Brause, H., Brewer, W.A., Grant, W.E., Herman‐Mercer, N., Livneh, B., Neupane, K.R., Petach, T., Peters, C.N., Wang, H.-H., Pahl-Wostl, C., & Wheater, H. (2024). Setting a pluralist agenda for water governance: Why power and scale matter. Wiley Interdisciplinary Reviews: Water, 11(5), e1734.

Maier, H.R., Guillaume, J.H., van Delden, H., Riddell, G.A., Haasnoot, M., & Kwakkel, J.H. (2016). An uncertain future, deep uncertainty, scenarios, robustness and adaptation: How do they fit together? Environmental Modeling & Software, 81, 154-164.

McIntosh, B.S., Ascough II, J.C., Twery, M., Chew, J., Elmahdi, A., Haase, D., Harou, J.J., Hepting, D., Cuddy, S., Jakeman, A.J., Chen, S., Kassahun, A., Lautenbach, S., Matthews, K., Merritt, W., Quinn, N.W.T., Rodriquez-Roda, I., Sieber, S., Stavenga, M., Sulis, A., Ticehurst, J., Volk, M., Wrobel, M., van Delden, H., El-Sawah, S., Rizzoli, A.E., & Voinov, A.A., (2011). Environmental decision support systems (EDSS) development - challenges and best practices. Environmental Modeling and Software, 26 (12), 1389–1402.

Melsen, L.A. (2022) It takes a village to run a model—the social practices of hydrological modeling. Water Resources Research, 58(2), e2021WR030600.

Melsen, L. A., Vos, J., & Boelens, R. (2018). What is the role of the model in socio-hydrology? Discussion of “Prediction in a socio-hydrological world”. Hydrological Sciences Journal, 63(9), 1435-1443.

Packett, E., Grigg, N.J., Wu, J., Cuddy, S.M., Wallbrink, P.J., & Jakeman, A.J. (2020) Mainstreaming gender into water management modelling processes. Environmental Modelling and Software, 127, 104683.

Pianosi, F., Beven, K., Freer, J., Hall, J.W., Rougier, J., Stephenson, D.B., & Wagener, T. (2016) Sensitivity analysis of environmental models: A systematic review with practical workflow. Environmental Modeling and Software, 79, 214-232.

Planque, B., Aarflot, J.M., Buttay, L., Carroll, J., Fransner, F., Hansen, C., Husson, B., Langangen, Ø., Lindstrøm, U., Pedersen, T., & Primicerio, R. (2022). A standard protocol for describing the evaluation of ecological models. Ecological Modeling, 471, 110059.

Refsgaard, J.C., Henriksen, H.J., Harrar, W.G., Scholten, H., & Kassahun, A. (2005). Quality assurance in model based water management–review of existing practice and outline of new approaches. Environmental Modeling & Software, 20, 1201-1215.

Refsgaard, J.C., van der Sluijs, J.P., Hojberg, A.L., & Vanrolleghem, P.A. (2007) Uncertainty in the environmental modeling process – a framework and guidance. Environmental Modeling and Software, 22, 1543-1556.

Rietveld, L.C., Van der Helm, A.W.C., Van Schagen, K.M., & Van der Aa, L.T.J. (2010) Good modeling practice in drinking water treatment, applied to Weesperkarspel plant of Waternet. Environmental Modeling and Software, 25(5), 661-669.

Robinson, S. (2008a) Conceptual modeling for simulation Part I: definition and requirements. Journal of the Operational Research Society, 59(3), 278-290.

Robinson, S. (2008b) Conceptual modeling for simulation Part II: a framework for conceptual modeling. Journal of the Operational Research Society, 59(3), 291-304.

Rose, K.A., Sable, S., DeAngelis, D.L., Yurek, S., Trexler, J.C., Graf, W., & Reed, D.J. (2015). Proposed best modeling practices for assessing the effects of ecosystem restoration on fish. Ecological Modeling, 300, 12-29.

Rosello, C., Elsawah, S., Guillaume, J., & Jakeman, A. (2022). A Century of Evolution of Modeling for River Basin Planning to the Next Generation of Models, Methods, and Concepts. Oxford Research Encyclopedia of Environmental Science. https://doi.org/10.1093/acrefore/9780199389414.013.624

Saltelli, A., & Di Fiore, M. (eds.) (2023). The politics of modeling: Numbers between science and policy. Oxford University Press.

Saltelli, A., Bammer, G., Bruno, I., Charters, E., Di Fiore, M., Didier, E., Nelson Espeland, W., Kay, J., Lo Piano, S., Mayo, D., & Pielke Jr, R., (2020b). Five ways to ensure that models serve society: a manifesto. Nature, 582(7813), 482-484.

Saltelli, A., Benini, L., Funtowicz, S., Giampietro, M., Kaiser, M., Reinert, E., & van der Sluijs, J.P., (2020a). The technique is never neutral. How methodological choices condition the generation of narratives for sustainability. Environmental Science & Policy, 106, 87-98.

Sanz, D., Vos, J., Rambags, F., Hoogesteger, J., Cassiraga, E., & Gómez-Alday, J.J. (2019). The social construction and consequences of groundwater modeling: insight from the Mancha Oriental aquifer, Spain. International Journal of Water Resources Development, 35(5), 808-829.

Schmolke, A., Thorbek, P., DeAngelis, D.L., & Grimm, V. (2010). Ecological models supporting environmental decision making: a strategy for the future. Trends in Ecology & Evolution, 25(8), 479-486.

Schuwirth, N., Borgwardt, F., Domisch, S., Friedrichs, M., Kattwinkel, M., Kneis, D., Kuemmerlen, M., Langhans, S.D., Martínez-López, J. and Vermeiren, P., 2019. How to make ecological models useful for environmental management. Ecological Modelling, 411, p.108784.

Scoones, I., & Stirling, A. (2020). Uncertainty and the politics of transformation. In: Scoones, I. & Stirling, A. (eds.) The politics of uncertainty: Challenges of transformation. Routledge, pp.1-30.

ter Horst, R., Alba, R., Vos, J., Rusca, M., Godinez-Madrigal, J., Babel, L.V., Veldwisch, G.J., Venot, J.P., Bonté, B., Walker, D.W., & Krueger, T. (2023). Making a case for power-sensitive water modeling: a literature review. Hydrology and Earth System Sciences Discussions. https://doi.org/10.5194/hess-2023-164.

van Vliet, J., Bregt, A.K., Brown, D.G., van Delden, H., Heckbert, S., & Verburg, P.H. (2016) A review of current calibration and validation practices in land-change modeling. Environmental Modeling and Software, 82, 174-182.

van Waveren, R.H., Groot, S., Scholten, H., Van Geer, F., Wösten, H., Koeze, R., & Noort, J. (1999) Good Modelling Practice Handbook (1999) STOWA, Utrecht, RWS-RIZA, Lelystad, The Netherlands, 185 pp., http://waterland.net/riza/aquest/projecten/modellen/gmph.shtml.

Vanwindekens, F.M., Stilmant, D., & Baret, P.V. (2013). Development of a broadened cognitive mapping approach for analysing systems of practices in social–ecological systems. Ecological Modelling, 250, 352-362.

Voinov, A., & Bousquet, F. (2010). Modeling with stakeholders. Environmental Modelling and Software, 25(11), 1268–1281.

Voinov, A., Seppelt, R., Reis, S., Nabel, J.E.M.S., & Shokravi, S. (2014) Values in socio-environmental modeling: Persuasion for action or excuse for inaction. Environmental Modeling and Software, 53, 207-212.

Walker, W.E., Harremoës, P., Rotmans, J., Van Der Sluijs, J.P., Van Asselt, M.B., Janssen, P., & Krayer von Krauss, M.P. (2003). Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5-17.

Waltemath, D., Adams, R., Beard, D. A., Bergmann, F. T., Bhalla, U. S., Britten, R., Chelliah, V., Cooling, M.T., Cooper, J., Crampin, E.J., Garny, A., Hoops, S., Hucka, M., Hunter, P., Klipp, E., Laibe, C., Miller, A.K., Moraru, I., Nickerson, D., Nielsen, P., Nikolski, M., Sahle, S., Sauro, H.M., Schmidt, H., Snoep, J.L., Tolle, D., Wolkenhauer, O., & Le Novère, N. (2011). Minimum information about a simulation experiment (MIASE). PLoS Computational Biology, 7(4), e1001122.

Wang, H.H., & Grant, W.E. (2021). Reflections of two systems ecologists on modeling coupled human and natural (socio-ecological, socio-environmental) systems. Ecological Modelling, 440, 109403.

Wang, H.-H., Van Voorn, G., Grant, W.E., Zare, F., Giupponi, C., Steinmann, P., Müller, B., Elsawah, S., Van Delden, H., Athanasiadis, I.N., Sun, Z., Jager, W., Little, J.C., & Jakeman, A.J. (2023). Scale decisions and good practices in socio-environmental systems modeling: guidance and documentation during problem scoping and model formulation. Socio-Environmental Systems Modeling, 5, 18563.

Wilkinson, M.D., Dumontier, M., Aalbersberg, I.J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L.B., Bourne, P.E., Bouwman, J., Brookes, A.J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C.T., Finkers, R., Gonzalez-Beltran, A., Gray, A.J.G., Groth, P., Goble, C., Grethe, J.S., Heringa, J., ’t Hoen, P.A.C., Hooft, R., Kuhn, T., Kok, R., Kok, J., Lusher, S.J., Martone, M.E., Mons, A., Packer, A.L., Persson, B., Rocca-Serra, P., Roos, M., van Schaik, R., Sansone, S.-A., Schultes, E., Sengstag, T., Slater, T., Strawn, G., Swertz, M. A., Thompson, M., van der Lei, J., van Mulligen, E., Velterop, J., Waagmeester, A., Wittenburg, P., Wolstencroft, K., Zhao, J., & Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018.

Zare, F., Guillaume, J.H.A., ElSawah, S., Croke, B., Fu, B., Iwanaga, T., Merritt, M., Partington, D., Ticehurst, J., & Jakeman, A.J. (2021) A formative and self-reflective approach to monitoring and evaluation of interdisciplinary team research: An integrated water resource modeling application in Australia. Journal of Hydrology, 596, 126070.

Zare, F., Guillaume, J.H.A., Jakeman, A.J., & Torabi, O. (2020) Reflective communication to improve problem-solving pathways: Key issues illustrated for an integrated environmental modeling case study. Environmental Modelling and Software, 126, 104645.

Zare, F., Jakeman, A.J., Elsawah, S., & Guillaume, J.H. (2024). Bridging practice and science in socio-environmental systems research and modeling: A design science approach. Ecological Modelling, 492, 110719.

Zurell, D., Franklin, J., König, C., Bouchet, P. J., Dormann, C. F., Elith, J., Fandos, G., Feng, X., Guillera-Arroita, G., Guisan, A., Lahoz-Monfort, J.J., Leitão, P.J., Park, D.S., Peterson, A.T., Rapacciuolo, G., Schmatz, D.R., Schröder, B., Serra-Diaz, J.M., Thuiller, W., Yates, K.L., Zimmermann, N.E., & Merow, C. (2020). A standard protocol for reporting species distribution models. Ecography, 43(9), 1261-1277.

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2024 Anthony J. Jakeman, Sondoss Elsawah, Hsiao-Hsuan Wang, Serena H. Hamilton, Lieke Melsen, Volker Grimm