Supplementary Files
Keywords
hydrodynamic modelling
modeller's perspective
interviews
standardisation
How to Cite
Abstract
Use of models in decision making, for example in water management, requires confidence in the model and its outputs. Since choices in model setup affect model output, this confidence is affected by the modellers’ professional judgement. Computer programmers can use their expertise in coding to standardise some of the tasks associated with computational modelling. Therefore, centralized automation has the potential to ensure quality of modelling decisions. Since it is the modeller that makes the choices in the model set-up, it is important to understand how modellers perceive automation. To explore their perspectives, we conducted fourteen interviews with modellers at water authorities and consulting companies in the Netherlands. The transcripts were analysed through deductive and inductive content analysis. Our study reveals that automated modelling processes can improve efficiency, transparency and consistency, but only if certain requirements are met, such as good documentation, clear ownership, adequate maintenance, and frequent evaluation. Therefore, managing the risks and benefits of automation requires balancing the power between modellers and programmers.
References
Addor, N., Melsen, L., 2019. Legacy, rather than adequacy, drives the selection of hydrological models. Water Resources Research 55, 378–390. doi:10.1029/2018WR022958.
Amorim, L.D., Cai, J., 2015. Modelling recurrent events: a tutorial for analysis in epidemiology. International Journal of Epidemiology 44, 324–333. doi:10.1093/ije/dyu222.
ATLAS.ti Scientific Software Development GmbH, 2022. ATLAS.ti Version 9.1.7.0, Retrieved from https://atlasti.com.
Babel, L., Vinck, D., Karssenberg, D., 2019. Decision-making in model construction: unveiling habits. Environmental Modelling & Software doi:10.1016/j.envsoft.2019.07.015.
Bacchelli, A., Bird, C., 2013. Expectations, outcomes, and challenges of modern code review, in: 2013 35th International Conference on Software Engineering (ICSE), IEEE. pp. 712–721.
Balci, O., 2001. A methodology for certification of modeling and simulation applications. ACM Transactions on Modeling and Computer Simulation (TOMACS) 11, 352–377. doi:10.1145/508366.508369.
Barnett, B., Townley, L., Post, V., Evans, R., Hunt, R., Peeters, L., Richardson, S., Werner, A., Knapton, A., Boronkay, A., 2012. Australian Groundwater Modelling Guidelines.
Bartram, D., Bayliss, R., 1984. Automated testing: Past, present and future. Journal of Occupational Psychology 57, 221–237. doi:10.1111/j.2044-8325.1984.tb00164.x.
Beeston, L., Blewitt, R., Bulmer, S., Wilson, J., 2021. Traffic modelling guidelines v4. Transport for London: London, UK.
Beven, K.J., 2000. Uniqueness of place and process representations in hydrological modelling. Hydrology & Earth System Sciences 4, 203–213. doi:10.5194/hess-4-203-2000.
Bremer, L.L., Hamel, P., Ponette-González, A.G., Pompeu, P.V., Saad, S.I., Brauman, K.A., 2020. Who are we measuring and modeling for? supporting multilevel decision-making in watershed management. Water Resources Research 56, e2019WR026011. doi:10.1029/2019WR026011.
Calder, M., Craig, C., Culley, D., de Cani, R., Donnelly, C.A., Douglas, R., Edmonds, B., Gascoigne, J., Gilbert, N., Hargrove, C., Hinds, D., Lane, D.C., Mitchell, D., Pavey, G., Robertson, D. Rosewell, B., Sherwin, S., Walport, M., Wilson, A., 2018. Computational modelling for decision-making: where, why, what, who and how. Royal Society Open Science 5, 172096. doi:10.1098/rsos.172096.
Carlisle, M., Gimbar, C., Jenkins, J.G., 2023. Auditor-client interactions—an exploration of power dynamics during audit evidence collection. Auditing: A Journal of Practice & Theory 42, 27–51. doi:10.2308/AJPT-2020-130.
Carver, L., Turoff, M., 2007. Human-computer interaction: the human and computer as a team in emergency management information systems. Communications of the ACM 50, 33–38. doi:10.1145/1226736.1226761.
Cockerill, K., Glynn, P., Chabay, I., Farooque, M., Hämäläinen, R., Miyamoto, B., McKay, P., 2019. Records of engagement and decision making for environmental and socio-ecological challenges. EURO Journal on Decision Processes 7, 243–265. doi:10.1007/s40070-019-00104-6.
Crout, N., Kokkonen, T., Jakeman, A., Norton, J., Newham, L., Anderson, R., Assaf, H., Croke, B., Gaber, N., Gibbons, J., Holzworth, D., Mysiak, J., Reichl, J., Seppelt, R., Wagener, T., Whitfied, P., 2008. Chapter two good modelling practice. Developments in Integrated Environmental Assessment 3, 15–31. doi:10.1016/S1574-101X(08)00602-9.
Davies, P., 2012. The state of evidence-based policy evaluation and its role in policy formation. National Institute Economic Review 219, R41–R52. doi:10.1177/002795011221900105.
Deltares, 2023a. D-HYDRO Suite 1D2D. https://www.deltares.nl/software-en-data/producten/d-hydro-suite-1d2d, Last accessed July 6 2023.
Deltares, 2023b. HYDROLIB. https://deltares.github.io/HYDROLIB/, Last accessed July 6 2023.
Deltares, 2023c. HYDROLIB github. https://github.com/Deltares/HYDROLIB, Last accessed July 6 2023.
Deltares, 2023d. Sobek Suite. https://www.deltares.nl/en/software/sobek/, Last accessed July 6 2023.
Deser, C., Phillips, A., Bourdette, V., Teng, H., 2012. Uncertainty in climate change projections: the role of internal variability. Climate Dynamics 38, 527–546. doi:10.1007/s00382-010-0977-x.
Eilander, D., Boisgontier, H., e. Bouaziz, L.J., Buitink, J., Couasnon, A., Dalmijn, B., Hegnauer, M., de Jong, T., Loos, S., Marth, I., van Verseveld, W., 2023. HydroMT: Automated and reproducible model building and analysis. Journal of Open Source Software 8, 4897. doi:10.21105/joss.04897.
Eyring, V., Cox, P.M., Flato, G.M., Gleckler, P.J., Abramowitz, G., Caldwell, P., Collins, W.D., Gier, B.K., Hall, A.D., Hoffman, F.M., Hurtt, G.C., Jahn, A., Jones, C.D., Klein, S.A., Krasting, J.P., Kwiatkowski, L., Lorenz, R., Maloney, E., Meehl, G.A., Pendergrass, A.G., Pincus, R., Ruane, A.C., Russel, J.L. Sanderson, B.M., Santer, B.D., Sherwood, S.C., Simpson, I.R., Stouffer, R.J., Williamson, M.S., 2019. Taking climate model evaluation to the next level. Nature Climate Change, 1. doi:10.1038/s41558-018-0355-y.
Fan, S.L., Lee, C.Y., Chong, H.Y., Skibniewski, M.J., 2018. A critical review of legal issues and solutions associated with building information modelling. Technological & Economic Development of Economy 24, 2098–2130. doi:10.3846/tede.2018.5695.
Garner, M., Hamilton, S., 2011. Principles of epidemiological modelling. Revue Scientifique et Technique-OIE 30, 407.
Glynn, P.D., Voinov, A.A., Shapiro, C.D., White, P.A., 2017. From data to decisions: Processing information, biases, and beliefs for improved management of natural resources and environments. Earth’s Future 5, 356–378. doi:10.1002/2016EF000487.
Government of the Netherlands, SA. Water management in The Netherlands. https://www.government.nl/topics/water-management/water-management-in-the-netherlands, Last accessed on July 27 2023.
Gundersen, O.E., 2021. The fundamental principles of reproducibility. Philosophical Transactions of the Royal Society A 379, 20200210. doi:10.1098/rsta.2020.0210.
Hamilton, S.H., Pollino, C.A., Stratford, D.S., Fu, B., Jakeman, A.J., 2022. Fit-for-purpose environmental modeling: Targeting the intersection of usability, reliability and feasibility. Environmental Modelling & Software 148, 105278. doi:10.1016/j.envsoft.2021.105278.
Hennink, M.M., Kaiser, B.N., Marconi, V.C., 2017. Code saturation versus meaning saturation: how many interviews are enough? Qualitative Health Research 27, 591–608. doi:10.1177/1049732316665344.
Howard, R., Björk, B.C., 2008. Building information modelling–experts’ views on standardization and industry deployment. Advanced Engineering Informatics 22, 271–280. doi:10.1016/j.aei.2007.03.001.
Jakeman, A.J., Letcher, R.A., Norton, J.P., 2006. Ten iterative steps in development and evaluation of environmental models. Environmental Modelling & Software 21, 602–614. doi:10.1016/j. envsoft.2006.01.004.
Kajko-Mattsson, M., 2005. A survey of documentation practice within corrective maintenance. Empirical Software Engineering 10, 31–55. doi:10.1023/B:LIDA.0000048322.42751.ca.
Knutti, R., Masson, D., Gettelman, A., 2013. Climate model genealogy: Generation CMIP5 and how we got there. Geophysical Research Letters 40, 1194–1199. doi:10.1002/grl.50256.
Lahtinen, T.J., Guillaume, J.H., Hämäläinen, R.P., 2017. Why pay attention to paths in the practice of environmental modelling? Environmental Modelling & Software 92, 74–81.
Latour, B., 1990. Technology is society made durable. The Sociological Review 38, 103–131. doi:10.1111/j.1467-954X.1990.tb03350.x.
Lewis, E., Birkinshaw, S., Kilsby, C., Fowler, H.J., 2018. Development of a system for automated setup of a physically-based, spatially-distributed hydrological model for catchments in Great Britain. Environmental Modelling & Software 108, 102–110. doi:10.1016/j.envsoft.2018.07.006.
Lim, T.C., Glynn, P.D., Shenk, G.W., Bitterman, P., Guillaume, J.H., Little, J.C., Webster, D., 2023. Recognizing political influences in participatory social-ecological systems modeling. Socio-Environmental Systems Modelling 5, 18509–18509. doi:10.18174/sesmo.18509.
Limerick, H., Coyle, D., Moore, J.W., 2014. The experience of agency in human-computer interactions: a review. Frontiers in Human Neuroscience 8, 643. doi:10.3389/fnhum.2014.00643.
Mai, J., Shen, H., Tolson, B.A., Gaborit, E´., Arsenault, R., Craig, J.R., Fortin, V., Fry, L.M., Gauch, M., Klotz, D., Kratzert, F., O’Brien, N., Princz, D.G., Rasiya Koya, S., Tirthankar, R., Seglenieks, F., Shrestha, N.K., Temgoua, A.G.T, Vionnet, V., Waddell, J.W., 2022. The Great Lakes runoff intercomparison project phase 4: the Great Lakes (GRIP-GL). Hydrology and Earth System Sciences 26, 3537–3572. doi:10.5194/hess-26-3537-2022.
McCracken, S., Salterio, S.E., Gibbins, M., 2008. Auditor–client management relationships and roles in negotiating financial reporting. Accounting, organizations and society 33, 362–383. doi:10.1016/j.aos.2007.09.002.
McFarlane, D.C., Latorella, K.A., 2002. The scope and importance of human interruption in human-computer interaction design. Human-Computer Interaction 17, 1–61. doi:10.1207/S15327051HCI1701_1.
Melsen, L., 2022. It takes a village to run a model—the social practices of hydrological modeling. Water Resources Research 58, e2021WR030600. doi:10.1029/2021WR030600.
Melsen, L.A., Vos, J., Boelens, R., 2018. What is the role of the model in socio-hydrology? Discussion of “Prediction in a socio-hydrological world”. Hydrological Sciences Journal 63, 1435–1443. doi:10.1080/02626667.2018.1499025.
Mer, F., Baethgen, W., Vervoort, R.W., 2020. Building trust in swat model scenarios through a multi-institutional approach in Uruguay. Socio-Environmental Systems Modelling 2, 17892–17892. doi:10.18174/sesmo.2020a17892.
Mir, M., Casadesús, M., 2011. Standardised innovation management systems: A case study of the Spanish standard une 166002: 2006. Innovar 21, 171–188.
Moallemi, E.A., Zare, F., Reed, P.M., Elsawah, S., Ryan, M.J., Bryan, B.A., 2020. Structuring and evaluating decision support processes to enhance the robustness of complex human–natural systems. Environmental Modelling & Software 123, 104551. doi:10.1016/j.envsoft.2019.104551.
Müller, B., Balbi, S., Buchmann, C.M., De Sousa, L., Dressler, G., Groeneveld, J., Klassert, C.J., Le, Q.B., Millington, J.D., Nolzen, H., Parker, D.C., Polhill, J.G., Schlüter, M., Schulze, J., Schwarz, N., Sun, Z., Taillandier, P., Weise, H., 2014. Standardised and transparent model descriptions for agent-based models: Current status and prospects. Environmental Modelling & Software 55, 156–163. doi:10.1016/j.envsoft.2014.01.029.
Nederlands Hydrologisch Instrumentarium, SA. HyDAMO Data Model. https://nhi.nu/ontwikkelingen/hydamo/, Last accessed on July 30 2023.
Pagano, T.C., Pappenberger, F., Wood, A.W., Ramos, M.H., Persson, A., Anderson, B., 2016. Automation and human expertise in operational river forecasting. Wiley Interdisciplinary Reviews: Water 3, 692–705. doi:10.1002/wat2.1163.
Parasuraman, R., Sheridan, T.B., Wickens, C.D., 2000. A model for types and levels of human interaction with automation. IEEE Transactions on systems, man, and cybernetics-Part A: Systems and Humans 30, 286–297. doi:10.1109/3468.844354.
Parnas, D.L., 2010. Precise documentation: The key to better software, in: The Future of Software Engineering. Springer, pp. 125–148. doi:10.1007/978-3-642-15187-3_8.
Pascarella, L., Spadini, D., Palomba, F., Bruntink, M., Bacchelli, A., 2018. Information needs in contemporary code review. Proceedings of the ACM on human-computer interaction 2, 1–27. doi:10.1145/3274404.
Polhill, J.G., Edmonds, B., 2007. Open access for social simulation. Journal of Artificial Societies and Social Simulation 10, 10–10. URL: https://www.jasss.org/10/3/10.html.
Refsgaard, J.C., van der Sluijs, J.P., Højberg, A.L., Vanrolleghem, P.A., 2007. Uncertainty in the environmental modelling process–a framework and guidance. Environmental Modelling & Software 22, 1543–1556. doi:10.1016/j.envsoft.2007.02.004.
Sampaio, P., Saraiva, P., Guimarães Rodrigues, A., 2011. Iso 9001 certification forecasting models. International Journal of Quality & Reliability Management 28, 5–26. doi:10.1108/02656711111097526.
Schmolke, A., Thorbek, P., DeAngelis, D.L., Grimm, V., 2010. Ecological models supporting environmental decision making: a strategy for the future. Trends in Ecology & Evolution 25, 479–486. doi:10.1016/j.tree.2010.05.001.
Schwarz, N., Dressler, G., Frank, K., Jager, W., Janssen, M.A., Müller, B., Schlüter, M., Wijermans, N., Groeneveld, J., 2020. Formalising theories of human decision-making for agent-based modelling of social-ecological systems: practical lessons learned and ways forward. Available at SSRN 4152673 doi:10.18174/sesmo.2020a16340.
Sharples, J.J., Cary, G.J., Fox-Hughes, P., Mooney, S., Evans, J.P., Fletcher, M.S., Fromm, M., Grierson, P.F., McRae, R., Baker, P., 2016. Natural hazards in Australia: extreme bushfire. Climatic Change 139, 85–99.
Stelling, G.S., Duinmeijer, S.A., 2003. A staggered conservative scheme for every froude number in rapidly varied shallow water flows. International Journal for Numerical Methods in Fluids 43, 1329–1354. doi:10.1002/fld.537.
van den Berg, J., van Lieshout, M., 2022. De wraak van de Zuiderzee: renovatie Afsluitdijk kost honderden miljoenen extra door blunder Rijkswaterstaat. Published in the Volkskrant online on May 20 2022, https://www.volkskrant.nl/nieuws-achtergrond/de-wraak-van-de-zuiderzee-renovatie-afsluitdijk-kost-honderden-miljoenen-extra-door-blunder-rijkswaterstaatba836a75/, Last accessed on July 27 2023.
van Waveren, R., Groot, S., Scholten, H., van Geer, F., Wüsten, H., Koeze, R., Noort, J., 1999. Vloeiend modelleren in het waterbeheer: Handboek Good Modelling Practice. Technical Report 99-05/99.036, STOWA/RIZA Wageningen
Verbeek, P.P., 2008. Morality in design: Design ethics and the morality of technological artifacts. In: Philosophy and design: From engineering to architecture (Editors: Vermaas, P.E., Kroes, P., Light, A., Moore, S.A.), 91–103. doi:10.1007/978-1-4020-6591-0_7.
Voinov, A., Jenni, K., Gray, S., Kolagani, N., Glynn, P.D., Bommel, P., Prell, C., Zellner, M., Paolisso, M., Jordan, R., et al., 2018. Tools and methods in participatory modeling: Selecting the right tool for the job. Environmental Modelling & Software 109, 232–255. doi:10.1016/j.envsoft.2018.08.028.
Vrontis, D., 2003. Integrating adaptation and standardisation in international marketing: the AdaptStand Modelling process. Journal of Marketing Management 19, 283–305. doi:10.1080/0267257X.2003.9728212.
Vukovic, R., 2022. From conducting to inspiring: A qualitative research on servant leadership in the corporate service of rijkswaterstaat.
Wang, H.H., Van Voorn, G., Grant, W.E., Zare, F., Giupponi, C., Steinmann, P., Müller, B., Elsawah, S., Van Delden, H., Athanasiadis, I.N., et al., 2023. Scale decisions and good practices in socio-environmental systems modelling: guidance and documentation during problem scoping and model formulation. Socio-Environmental Systems Modelling 5, 18563. doi:10.18174/sesmo.18563.
Watson, R.T., 2005. Turning science into policy: challenges and experiences from the science–policy interface. Philosophical Transactions of the Royal Society B: Biological Sciences 360, 471–477. doi:10.1098/rstb.2004.1601.
Wears, R.L., 2015. Standardisation and its discontents. Cognition, Technology & Work 17, 89–94. doi:10.1007/s10111-014-0299-6.
Werkowska, W., Márquez, A.L., Real, R., Acevedo, P., 2017. A practical overview of transferability in species distribution modeling. Environmental Reviews 25, 127–133. doi:10.1139/er-2016-0045.
Winkler, D., Hametner, R., Östreicher, T., Biffl, S., 2010. A framework for automated testing of automation systems, in: 2010 IEEE 15th Conference on Emerging Technologies Factory Automation (ETFA 2010), pp. 1–4. doi:10.1109/ETFA.2010.5641264.
Woods, D.D., Sarter, N.B., 2000. Learning from automation surprises and going sour accidents. In: N.B. Sarter & R. Amalberti (eds.) Cognitive engineering in the aviation domain. Lawrence Erlbaum Associates Publishers, 327–353.
Yates, K.L., Bouchet, P.J., Caley, M.J., Mengersen, K., Randin, C.F., Parnell, S., Fielding, A.H., Bamford, A.J., Ban, S., Barbosa, A.M., Dormann, C.F., Elith, J., Embling, C.B., Ervin, G.N., Fisher, R., Gould, S., Graf, R.F., Gregr, E.J., Halpin, P.N., Heikkinen, R.K., Heinänen, S., Jones, A.R., Krishnakumar, P.K., Laruia, V., Lozano-Montes, H., Mannocci, L., Mellin, C., Mesgaran, M.B., Moreno-Amat, E., Mormede, S., Novaczek, E., Oppel, S, Ortuño Crespo, G., Townsend Peterson, A., Rapacciuolo, G., Roberts, J.J., Ross, R.E., Scales, K.L., Schoeman, D., Snelgrove, P., Sundblad, G., Thuiller, W., Torres, L.G., Verbruggen, H., Wang, L., Wenger, S., Whittingham, M.J., Zharikov, Y., Zurell, D., Sequeira, A.M.M., 2018. Outstanding challenges in the transferability of ecological models. Trends in Ecology & Evolution 33, 790–802. doi:10.1016/j.tree.2018.08.001.
Zurell, D., Franklin, J., König, C., Bouchet, P.J., Dormann, C.F., Elith, J., Fandos, G., Feng, X., Guillera-Arroita, G., Guisan, A., Lahoz-Monfort, J.J., Leitão, P.J., Park, D.S., Townsend Peterson, A., Rapacciuolo, G., Schmatz, D.R., Schröder, B., Serra-Diaz, J.M., Thuiller, W., Yates, K.L., Zimmermann, N.E., Merow, C., 2020. A standard protocol for reporting species distribution models. Ecography 43, 1261–1277. doi:10.1111/ecog.04960.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) 2024 J.O.E. Remmers, A.J. Teuling, R.J. Dahm, A. van Dam, L.A. Melsen