Vol. 2, Research Papers
Vol. 2

Eight grand challenges in socio-environmental systems modeling

Research Papers
Published 2020-01-01
Sondoss Elsawah
University of New South Wales
Tatiana Filatova
University of Twente, The Netherlands
Anthony J. Jakeman
Australian National University, Australia
Albert J. Kettner
University of Colorado
Moira L. Zellner
University of Illinois at Chicago
Ioannis N. Athanasiadis
Wageningen University, Netherlands
Serena H. Hamilton
Edith Cowan University, Australia
Robert L. Axtell
George Mason University, USA
Daniel G. Brown
University of Washington, USA
Jonathan M. Gilligan
Vanderbilt University, USA
Marco A. Janssen
Arizona State University
Derek T. Robinson
University of Waterloo, Canada
Julie Rozenberg
World Bank, Washington DC, USA
Isaac I. T. Ullah
San Diego State University
Steve J. Lade
Stockholm University
Article Full Text (PDF)

Keywords

Socio-ecological
uncertainty
actionable science
systemic change
decision support

How to Cite

Elsawah, S., Filatova, T., Jakeman, A. J., Kettner, A. J., Zellner, M. L., Athanasiadis, I. N., Hamilton, S. H., Axtell, R. L., Brown, D. G., Gilligan, J. M., Janssen, M. A., Robinson, D. T., Rozenberg, J., Ullah, I. I. T., & Lade, S. J. (2020). Eight grand challenges in socio-environmental systems modeling. Socio-Environmental Systems Modelling, 2, 16226. https://doi.org/10.18174/sesmo.2020a16226
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Abstract

Modeling is essential to characterize and explore complex societal and environmental issues in systematic and collaborative ways. Socio-environmental systems (SES) modeling integrates knowledge and perspectives into conceptual and computational tools that explicitly recognize how human decisions affect the environment. Depending on the modeling purpose, many SES modelers also realize that involvement of stakeholders and experts is fundamental to support social learning and decision-making processes for achieving improved environmental and social outcomes. The contribution of this paper lies in identifying and formulating grand challenges that need to be overcome to accelerate the development and adaptation of SES modeling. Eight challenges are delineated: bridging epistemologies across disciplines; multi-dimensional uncertainty assessment and management; scales and scaling issues; combining qualitative and quantitative methods and data; furthering the adoption and impacts of SES modeling on policy; capturing structural changes; representing human dimensions in SES; and leveraging new data types and sources. These challenges limit our ability to effectively use SES modeling to provide the knowledge and information essential for supporting decision making. Whereas some of these challenges are not unique to SES modeling and may be pervasive in other scientific fields, they still act as barriers as well as research opportunities for the SES modeling community. For each challenge, we outline basic steps that can be taken to surmount the underpinning barriers. Thus, the paper identifies priority research areas in SES modeling, chiefly related to progressing modeling products, processes and practices.

Article Full Text (PDF)

References

Abar, S., Theodoropoulos, G. K., Lemarinier, P., & O’Hare, G. M. (2017). Agent based modelling and simulation tools: a review of the state-of-art software. Computer Science Review, 24, 13-33. https://doi.org/10.1016/j.cosrev.2017.03.001.

An, L. (2012) Modeling human decisions in coupled human and natural systems: Review of agent-based models. Ecological Modelling, 229, 25-36.

Anderies, J.M., Janssen, M.A., Bousquet, F., Cardenas, J.-C., Castillo, D., Lopez, M.-C., Tobias, R., Vollan, B., & Wutich, A. (2011). The challenge of understanding decisions in experimental studies of common pool resource governance. Ecological Economics, 70(9), 1571-1579.

Athanasiadis, I. N. (2017). How to Start an Environmental Software Project. In Environmental Software Systems. Computer Science for Environmental Protection: 12th IFIP WG 5.11 International Symposium, ISESS 2017, Zadar, Croatia, May 10-12, 2017, Proceedings 12 (pp. 395-407). Springer International Publishing.

Baker, M. (2016) Is there a reproducibility crisis? Nature, 533, 452–454.

Bankes, S., Lempert, R., & Popper, S. (2002) Making Computational Social Science Effective: Epistemology, Methodology, and Technology. Social Science Computer Review, 20(4), 377-388. https://doi.org/10.1177%2F089443902237317.

Barberá, P., & Rivero, G. (2015). Understanding the political representativeness of Twitter users. Social Science Computer Review, 33(6), 712-729.

Bayat, B., Crasta, N., Crespi, A., Pascoal, A.M., & Ijspeert, A. (2017) Environmental Monitoring Using Autonomous Vehicles: A Survey of Recent Searching Techniques. Current Opinion in Biotechnology, 45, 76–84. https://doi.org/10.1016/j.copbio.2017.01.009.

Beckage, B., Gross, L.J., Lacasse, K., Carr, E., Metcalf, S.S., Winter, J.M., Howe, P.D., Fefferman, N., Franck, T., Zia, A., Kinzig, A., & Hoffman, F.M. (2018) Linking models of human behaviour and climate alters projected climate change. Nature Climate Change, 8 (1), 79-84. https://doi.org/10.1038/s41558-017-0031-7.

Belete, G. F., Voinov, A., & Laniak, G. F. (2017a). An overview of the model integration process: From pre-integration assessment to testing. Environmental modelling & software, 87, 49-63.

Belete, G.F., Voinov, A., & Morales, J. (2017b) Designing the Distributed Model Integration Framework – DMIF. Environmental Modelling & Software, 94, 112–126. https://doi.org/10.1016/j.envsoft.2017.04.003.

Belete, G, F., Voinov, A., Arto, I., Dhavala, K., Bulavskaya, T., Niamir, L., Moghayer, S. & Filatova, T. (2019) Exploring Low-Carbon Futures: A Web Service Approach to Linking Diverse Climate-Energy-Economy Models. Energies 12 (15), 2880. https://doi.org/10.3390/en12152880.

Bell, A., Calvo-Hernandez, C., & Oppenheimer, M. (2019). Migration, Intensification, and Diversification as Adaptive Strategies. Socio-Environmental Systems Modelling, 1. https://doi.org/10.18174/sesmo.2019a16102.

Bell, A.R., Robinson, D.T., Malik, A., & Dewal, S. (2015) Modular ABM development for improved dissemination and training. Environmental Modelling & Software, 73, 189-200. https://doi.org/10.1016/j.envsoft.2015.07.016.

Bert, F.E., Rovere, S.L., Macal, C.M., North, M.J., & Podestra, G.P. (2014) Lessons from a comprehensive validation of an agent based-model: The experience of the Pampas Model of Argentinean agricultural systems. Ecological Modelling, 273, 284–298. https://doi.org/10.1016/j.ecolmodel.2013.11.024.

Beven, K. (2016) Facets of uncertainty: epistemic uncertainty, non-stationarity, likelihood, hypothesis testing, and communication. Hydrological Science Journal, 61, 1652-1665. https://doi.org/10.1080/02626667.2015.1031761.

Bitterman, P., & Bennett, D.A. (2016) Constructing stability landscapes to identify alternative states in coupled social-ecological agent-based models. Ecology and Society, 21(3). https://doi.org/10.5751/ES-08677-210321.

Blazquez, D., & Domenech, J. (2018). Big Data sources and methods for social and economic analyses. Technological Forecasting and Social Change, 130, 99-113.

Bonabeau, E. (2002). Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences 99 (Supplementary 3), 7280–7287. doi: 10.1073/pnas.082080899.

Bone, C., Johnson, B., Nielsen-Pincus, M., Sproles, E., & Bolte, J. (2013). A Temporal Variant-Invariant Validation Approach for Agent-based Models of Landscape Dynamics. Transactions in GIS, 18(2), 161-182. https://doi.org/10.1111/tgis.12016.

Bonzanigo, L., Brown, C., Harou, J.J., Hurford, A., Ray, P., & Karki, P. (2015). South Asia - Investment decision making in hydropower: decision tree case study of the upper Arun hydropower project and Koshi basin hydropower development in Nepal. Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/179901476791918856/South-Asia-Investment-decision-making-in-hydropower-decision-tree-case-study-of-the-upper-Arun-hydropower-project-and-Koshi-basin-hydropower-development-in-Nepal

Brandt, T., Bendler, J., & Neumann, D. (2017). Social media analytics and value creation in urban smart tourism ecosystems. Information & Management, 54(6), 703-713. https://doi.org/10.1016/j.im.2017.01.004.

Brooks, H. (1984). The Resolution of Technically Intensive Public Policy Disputes. Science, Technology, & Human Values, 9(1), 39–50. https://doi.org/10.1177/016224398400900106.

Brown, D.G., Page, S.E., Riolo, R., Zellner, M.L., & Rand, W. (2005). Path dependence and the validation of agent-based spatial models of land use. International Journal of Geographical Information Science, Special Issue on Land Use Dynamics, 19 (2): 153-174. https://doi.org/10.1080/13658810410001713399.

Brugnach, M., Pahl-Wostl, C., Lindenschmidt, K.E., Janssen, J.A.E.B., Filatova, T., Mouton, A., Holtz, G., van der Keur, P., & Gaber, N. N. (2008). Complexity and uncertainty: rethinking the modelling activity. In: Jakeman A. J., Voinov A. A., Rizzoli A. E., Chen S. H. (eds.), Environmental modelling, software and decision support: state of the art and new perspectives, Chapter 4, Amsterdam, Elsevier, pp.49-68.

Bungartz, H.J., and Griebel, M. (2004). Sparse grids. Acta Numerica, 13, 147-269. https://doi.org/10.1017/S0962492904000182.

Busch, J., Godoy, F., Turner, W.R., & Harvey, C.A. (2011). Biodiversity co-benefits of reducing emissions from deforestation under alternative reference levels and levels of finance. Conservation Letters, 4(2), pp. 101-115.

Castilla-Rho J.C., Rojas, R., Andersen, M.S, Holley, C., & Mariethoz, G. (2017). Social tipping points in global groundwater management. Nature Human Behaviour, 1, 640–649. https://doi.org/10.1038/s41562-017-0181-7.

Certomà, C., Corsini, F., & Rizzi, F. (2015). Crowdsourcing Urban Sustainability. Data, People and Technologies in Participatory Governance. Futures, 74, 93–106. https://doi.org/10.1016/j.futures.2014.11.006.

Chakladar, S. (2016). A model driven engineering framework for simulation experiment management. PhD dissertation, Auburn University.

Chang, Y.C., Hong, F.W., & Lee, M.T. (2008). A system dynamic based DSS for sustainable coral reef management in Kenting coastal zone, Taiwan. Ecological Modelling, 211(1-2), 153-168.

Charli-Joseph, L., Siqueiros-Garcia, J.M., Eakin, H., Manuel-Navarrete, D., & Shelton, R. (2018). Promoting agency for social-ecological transformation: A transformation-lab in the Xochimilco social-ecological system. Ecology and Society, 23(2), 46.

Cheong, S.-M., Brown, D. G., Kok, K., and Lopez‐Carr, D. (2011). Mixed methods in land change research: towards integration. Transactions of the Institute of British Geographers, 37(1), 8–12. https://doi.org/10.1111/j.1475-5661.2011.00482.x.

Chen, X., Elmes, G., Ye, X., & Chang, J. (2016). Implementing a real-time Twitter-based system for resource dispatch in disaster management. GeoJournal, 81(6) 863-873. https://doi.org/10.1007/s10708-016-9745-8.

Chin, A., Coelho, R., Nugent, R., Munakata, R., & Puig-Suari, J. (2008, September). CubeSat: the pico-satellite standard for research and education. In: AIAA Space 2008 Conference & Exposition (p. 7734).

Contreras, D., Guiot, J., Suarez, R., Kirman, A. (2018) Reaching the human scale: A spatial and temporal downscaling approach to the archaeological implications of paleoclimate data. Journal of Archaeological Science, 93, 54-67.

Coron, L. Andreassian, V., Perrin, C., Lerat, J., Vaze, J., Bourqui, M., & Hendrickx, F. (2012). Crash testing hydrological models in contrasted climate conditions: an experiment on 216 Australian catchments. Water Resources Research, 48(5), W05552. https://doi.org/10.1029/2011WR011721.

Coleman, J. (1998). Foundations of Social Theory. Harvard University Press.

Crow, M.M., & Dabars, W.B. (2015). Designing the New American University, John Hopkins University Press, Baltimore.

De Haan, Fjalar J., and Jan Rotmans. (2018). “A Proposed Theoretical Framework for Actors in Transformative Change.” Technological Forecasting and Social Change 128, 275–86. https://doi.org/10.1016/j.techfore.2017.12.017.

Dellink, R., Chateau, J., Lanzi, E., & Magné, B. (2017) Long-term economic growth projections in the Shared Socioeconomic Pathways. Global Environmental Change, 42, pp. 200-214.

Deletic, A., Dotto, C.B.S., McCarthy, D.T., Kleidorfer, M., Freni, G., Mannina, G., Uhl, M., Henrichs, M., Fletcher, T.D., Rauch, W., Bertrand-Krajewski, J.L., & Tait, S. (2012). Assessing uncertainties in urban drainage models. Physics and Chemistry of the Earth, 42-44: 3-10.

Di Baldassarre, G., Viglione, A., Carr, G., Kuil, L., Yan, K., Brandimarte, L., & Blöschl, G. (2015). Debates—Perspectives on socio‐hydrology: Capturing feedbacks between physical and social processes. Water Resources Research, 51(6), 4770–4781. https://doi.org/10/f3n3p5.

Di Baldassarre, G., Brandimarte, L., & Beven, K. (2016). The seventh facet of uncertainty: wrong assumptions, unknowns and surprises in the dynamics of human–water systems. Hydrological Sciences Journal, 61(9), 1748–1758. https://doi.org/10.1080/02626667.2015.1091460.

Drosatos, G., Efraimidis, P. S., Athanasiadis, I.N., D'Hondt, E., & Stevens, M. (2012). A privacy-preserving cloud computing system for creating participatory noise maps. In Computer Software and Applications Conference (COMPSAC), 2012 IEEE 36th Annual (pp. 581-586). IEEE. https://doi.org/10.1109/COMPSAC.2012.78.

Drosatos, G., Efraimidis, P. S., Athanasiadis, I. N., Stevens, M., & D’Hondt, E. (2014). Privacy-preserving computation of participatory noise maps in the cloud. Journal of Systems and Software, 92, 170-183. https://doi.org/10.1016/j.jss.2014.01.035.

Elbattah, M. (2018). Hybrid systems modelling aided by machine learning with applications in healthcare, Doctoral dissertation, NUI Galway.

Elsawah, S., Guillaume J.H.A., Filatova T., Rook J., & Jakeman, A.J. (2015). A methodology for eliciting, representing, and analysing stakeholder knowledge for decision making on complex socio-ecological systems: From cognitive maps to agent-based models. Journal of Environmental Management, 151, 500-516. https://doi.org/10.1016/j.jenvman.2014.11.028.

Epstein, J.M., & Axtell, R. (1996). Growing Artificial Societies: Social Science from the Bottom Up. Cambridge, MA: MIT Press.

Evans, T., Robinson, D.T., & Schmitt-Harsh, M. (2013). Limitations, challenges, and solutions to integrating carbon dynamics with land-use models. Brown, D.G., Robinson, D.T., French, N.H.F., and B.C. Reed (eds), in Land use and the carbon cycle: Advances in Integrated Science, Management, and Policy. Cambridge University Press. Pp. 178-208.

Fang, S., Xu, L.D., Zhu, Y., Ahati, J., Pei, H., Yan, J., & Liu, Z. (2014). An Integrated System for Regional Environmental Monitoring and Management Based on Internet of Things. IEEE Transactions on Industrial Informatics, 10 (2), 1596–1605. https://doi.org/10.1109/TII.2014.2302638.

Farmer, J.D., & Foley, D. (2009). The economy needs agent-based modelling. Nature, 460, 685-686. https://doi.org/10.1038/460685a.

Farmer, J.D., Hepburn, C., Mealy, P., and Teytelboym, A. (2015). A Third Wave in the Economics of Climate Change. Environmental and Resource Economics, 62(2): 329-357. https://doi.org/10.1007/s10640-015-9965-2.

Farrell, H. (2017). How Facebook Stymies Social Science. The Chronicle of Higher Education, December 19. https://www.chronicle.com/article/How-Facebook-Stymies-Social/242090.

Ferson, S., & Sentz, K. (2016). Epistemic Uncertainty in Agent-based Modeling. In S. Freitag, R. L. Muhanna, & R. L. Mullen (Eds.), Proceedings of the 7th International Workshop of Reliable Engineering Computing: Computing with Polymorphic Uncertain Data (pp. 65–82). Bochum, Germany. Retrieved from http://rec2016.rub.de/downloads/rec2016_proceedings.pdf.

Filatova T., Verburg, P.H., Parker, D.C., & Stannard, C.A. (2013). Spatial agent-based models for socio-ecological systems: challenges and prospects. Environmental Modelling & Software, 45, 1-7.

Filatova T, Polhill, J.G., & van Ewijk, S. (2016). Regime shifts in coupled socio-environmental systems: Review of modelling challenges and approaches. Environmental Modelling & Software, 75:333–347. https://doi.org/10.1016/j.envsoft.2015.04.003.

Fischhoff, B. (2006). Behaviorally Realistic Risk Management. In R. J. Daniels, D. F. Kettl, & H. Kunreuther (Eds.), On Risk and Disaster: Lessons from Hurricane Katrina (pp. 77–88). Philadelphia: University of Pennsylvania Press.

Fischhoff, B., Slovic, P., & Lichtenstein, S. (1982). Lay foibles and expert fables in judgments about risk. The American Statistician, 36(3), 240–255. https://doi.org/10/fq48fh.

Forni, M, & Lippi, M. (1997). Aggregation and the Microfoundations of Dynamic Macroeconomics. Oxford: Oxford University Press.

Fowler, H.J., Blenkinsop, S., & Tebaldi, C. (2007). Linking climate change modelling to impacts studies: recent advances in downscaling techniques for hydrological modelling. International Journal of Climatology, 27(12), 1547-1578. https://doi.org/10.1002/joc.1556.

Fu, B., Guillaume, J.H.A., & Jakeman, A.J. (2015). An iterative method for discovering feasible management interventions and targets conjointly using uncertainty visualizations. Environmental Modelling & Software, 71, 159-173. https://doi.org/10.1016/j.envsoft.2015.05.017.

Gibert, K., Horsburgh, J.S., Athanasiadis, I.N., & Holmes, G. (2018). Environmental Data Science, Environmental Modelling & Software, 106, 4-12.

Gilligan, J.M. (2018). Accounting for the human factor. Nature Climate Change, 8, 14-15. https://doi.org/10.1038/s41558-017-0038-0.

Gibson, C.C., Ostrom, E., & Ahn, T.K. (2000). The concept of scale and the human dimensions of global change: a survey. Ecological Economics, 32(2), 217-239. https://doi.org/10.1016/S0921-8009(99)00092-0.

Goring, S.J., Weathers, K.C., Dodds, W.K., Soranno, P.A., Sweet, L.C., Cheruvelil, K.S., Kominoski, J.S., Rüegg, J., Thorn, A.M., & Utz, R.M. (2014). Improving the culture of interdisciplinary collaboration in ecology by expanding measures of success. Frontiers in Ecology and the Environment, 12(1), 39–47. https://doi.org/10.1890/120370.

Grainger, S., Mao, F., & Buytaert, W. (2016). Environmental data visualisation for non-scientific contexts: Literature review and design framework. Environmental Modelling & Software, 85, 299-318. https://doi.org/10.1016/j.envsoft.2016.09.004

Gray S.A., Zanre E., & Gray S.R.J. (2014). Fuzzy Cognitive Maps as Representations of Mental Models and Group Beliefs. In: Papageorgiou E. (eds) Fuzzy Cognitive Maps for Applied Sciences and Engineering. Intelligent Systems Reference Library, vol 54. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39739-4_2.

Gray, S., Voinov., A., Paolisso, M., Jordan, R., BenDor, T., Bommel, P., Glynn, P., Hedelin, B., Hubacek, K., Introne, J., Kolagani, N., Laursen, B., Prell, C., Schmitt-Olabisi, L., Singer, A., Sterling, E., & Zellner, M. (2018). Purpose, Processes, Partnerships, and Products: 4Ps to advance Participatory Socio-Environmental Modeling. Ecological Applications, 28(1), 46-61.

Groves, D.G., and Lempert, R.J. (2007). A new analytic method for finding policy-relevant scenarios. Global Environmental Change, 17(1), 73-85. https://doi.org/10.1016/j.gloenvcha.2006.11.006.

Guillaume, J.H.A., Hunt, R.J., Comunian, A., Blakers, R.S., & Fu, B. (2016). Methods for Exploring Uncertainty in Groundwater Management Predictions. In: Jakeman A.J., Barreteau O., Hunt R.J., Rinaudo JD., Ross A. (eds) Integrated Groundwater Management. Springer, Cham. https://doi.org/10.1007/978-3-319-23576-9_28.

Guivarch, C., Lempert, R., & Trutnevyte, E. (2017). Scenario techniques for energy and environmental research: An overview of recent developments to broaden the capacity to deal with complexity and uncertainty. Environmental Modelling & Software, 97, 201–210. https://doi.org/10.1016/j.envsoft.2017.07.017.

Haasnoot, M., Van Deursen, W. P. A., Guillaume, J. H., Kwakkel, J. H., van Beek, E., & Middelkoop, H. (2014). Fit for purpose? Building and evaluating a fast, integrated model for exploring water policy pathways. Environmental Modelling & Software, 60, 99-120. https://doi.org/10.1016/j.envsoft.2014.05.020.

Hamilton, S.H., Elsawah, S., Guillaume, J.H., Jakeman, A.J., & Pierce, S.A. (2015). Integrated assessment and modelling: overview and synthesis of salient dimensions. Environmental Modelling & Software, 64, 215-229. https://doi.org/10.1016/j.envsoft.2014.12.005.

Happe, K. (2005). Agent-based modelling and sensitivity analysis by experimental design and metamodelling: an application to modelling regional structural change. In: XIth International Congress of the European Association of Agricultural Economists. European Association of Agricultrual Economists, The Future of Rural Europe in the Global Agri-Food System, Copenhagen, Denmark, August 24-27, 2005.

Harp, D.R., & Vesselinov, V.V. (2012). An agent-based approach to global uncertainty and sensitivity analysis. Computers & Geosciences, 40, 19–27. https://doi.org/10.1016/j.cageo.2011.06.025.

Heppenstall, A.J.J., Crooks, A.T., See, L.M., & Batty, M. (2012). Agent-Based Models of Geographical Systems. Springer: Berlin. pp 1-759

Hoch, C.J., Zellner, M.L., Milz, D.C., Radinsky, J., & Lyons, L. (2015). Seeing is not believing: cognitive bias and modelling in collaborative planning. Planning Theory and Practice, 16 (3), 319-335. https://doi.org/10.1080/14649357.2015.1045015.

Howe, P.D., & Leiserowitz, A. (2013). Who Remembers a Hot Summer or a Cold Winter? The Asymmetric Effect of Beliefs about Global Warming on Perceptions of Local Climate Conditions in the U.S., Global Environmental Change, 23 (6), 1488–1500. https://doi.org/10/f5phcx.

Ioannidis, J.P.A. (2013). Informed Consent, Big Data, and the Oxymoron of Research That Is Not Research. The American Journal of Bioethics, 13 (4), 40–42. https://doi.org/10.1080/15265161.2013.768864.

Jakeman, A.J., Barreteau, O., Hunt, R., Rinaudo, J.D., & Ross, A. (2016). Integrated Groundwater Management. Springer.

Jakeman A.J., Jakeman J.D. (2016). An Overview of Methods to Identify and Manage Uncertainty for Modelling Problems in the Water–Environment–Agriculture Cross-Sector. In: Anderssen R., Broadbridge P., Fukumoto Y., Kajiwara K., Simpson M., Turner I. (eds) Agriculture as a Metaphor for Creativity in All Human Endeavors. FMfI 2016. Mathematics for Industry, vol 28. Springer, Singapore, DOI https://doi.org/10.1007/978-981-10-7811-8_15.

Jakeman, A.J., Letcher, R.A., & Norton, J.P. (2006). Ten iterative steps in development and evaluation of environmental models. Environmental Modelling & Software, 21, 602-614. https://doi.org/10.1016/j.envsoft.2006.01.004.

Janssen, M.A. (2016). Impact of diverse behavioral theories on environmental management: Explorations with Daisyworld. In: 2016 Winter Simulation Conference (WSC) (pp. 1690–1701). https://doi.org/10.1109/WSC.2016.7822217.

Janssen, M.A. (2017). The Practice of Archiving Model Code of Agent-Based Models. Journal of Artificial Societies and Social Simulation, 20(1), 2. https://doi.org/10.18564/jasss.3317.

Janssen, P.H.M., Petersen, A.C., van der Sluijs, J.P., Risbey, J.S., & Ravetz, J.R. (2003). RIVM/MNP Guidance for Uncertainty Assessment and Communication: Quickscan Hints & Actions List. 90-6960-105-2, RIVM/MNP. Available from http://www.nusap.net.

Jasanoff, S. (2003). Technologies of humility: citizen participation in governing science. Minerva, 41(3), 223–244.

Jefferson, J.L., Gilbert, J.M., Constantine, P.G., & Maxwell, R.M. (2015). Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model. Computers & Geosciences, 83, 127-138. https://doi.org/10.1016/j.cageo.2015.07.001.

Jiang, Q., Bregt, A.K., & Kooistra, L. (2018). Formal and Informal Environmental Sensing Data and Integration Potential: Perceptions of Citizens and Experts. Science of The Total Environment, 619–620, 1133–42. https://doi.org/10.1016/j.scitotenv.2017.10.329.

Jick, T. D. (1979). Mixing Qualitative and Quantitative Methods: Triangulation in Action. Administrative Science Quarterly, 24(4), 602–611. https://doi.org/10/dd4b5d.

Jongman, B., Winsemius, H. C., Aerts, J. C., de Perez, E. C., van Aalst, M. K., Kron, W., & Ward, P. J. (2015). Declining vulnerability to river floods and the global benefits of adaptation. Proceedings of the National Academy of Sciences, 112(18), E2271-E2280.

Jordan, R., Gray, S., Zellner, M., Glynn, P., Voinov, A., Hedelin, B., Sterling, E., Leong, K., Schmitt Olabisi, L., Hubacek, K., Bommel, P., BenDor, T., Jetter, A., Laursen, B., Singer, A., Giabbanelli, P., Kolagani, N., Basco Carrera, L., & Jenni, K. (2018). 12 Questions for the participatory modeling community. Earth's Future, 6(8), 1046-1057.

Kaipio, J., and Somersalo, E. (2006). Statistical and computational inverse problems, Vol. 160. Springer Science & Business Media.

Kalra, N., Hallegatte, S., Lempert, R., Brown, C., Fozzard, A., Gill, S., & Shah, A. (2015). Agreeing on robust decisions: new processes for decision making under deep uncertainty. Policy Research working paper; no. WPS 6906. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/365031468338971343/Agreeing-on-robust-decisions-new-processes-for-decision-making-under-deep-uncertainty.

Kasprzyk, J.R., Nataraj, S., Reed, P.M., & Lempert, R.J. (2013). Many objective robust decision making for complex environmental systems undergoing change. Environmental Modelling & Software, 42, 55–71. https://doi.org/10/f4scqr.

Kelly, R.A., Jakeman, A.J., Barreteau, O., Borsuk, M.E., Elsawah, S., Hamilton, S.H., Henriksen, H.J., Kuikka, S., Maier, H.R., Rizzoli, A.E., van Delden, H., & Voinov, A., 2013. Selecting among five common modelling approaches for integrated environmental assessment and management. Environmental Modelling & Software, 47, 159-181. https://doi.org/10.1016/j.envsoft.2013.05.005.

Kettner, A.J., & Syvitski, J.P.M. (2016). Uncertainty and Sensitivity in Surface Dynamics Modeling. Computers & Geosciences, 90, 1-5. https://doi.org/10.1016/j.cageo.2016.03.003.

Kirman, A.P. (1992). Whom or what does the representative individual represent? Journal of Economic Perspectives, 6(2), 117-136. https://doi.org/10.1257/jep.6.2.117.

Kitchin, R. (2013). Big Data and Human Geography: Opportunities, Challenges and Risks. Dialogues in Human Geography, 3 (3): 262–67. https://doi.org/10.1177/2043820613513388.

Kline, J.D., White, E.M., Fischer, A.P., Steen-Adams, M.M., Charnley, S., Olsen, C.S., Spies, T.A., & Bailey, J.D. (2017). Integrating social science into empirical models of coupled human and natural systems. Ecology and Society, 22(3). http://www.jstor.org/stable/26270162.

Kretschmer, B. & Peterson, S. (2010). Integrating bioenergy into computable general equilibrium models - A survey. Energy Economics, 32(3), pp. 673-686.

Kryvasheyeu, Y., Chen, H., Obradovich, N., Moro, E., Van Hentenryck, P., Fowler, J., & Cebrian, M. (2016). Rapid Assessment of Disaster Damage Using Social Media Activity. Science Advances, 2 (3), e1500779. https://doi.org/10/gc5tfp.

Kwakkel, J.H., & Pruyt, E. (2013). Exploratory Modeling and Analysis, an approach for model-based foresight under deep uncertainty. Technological Forecasting and Social Change, 80(3), 419-431. https://doi.org/10.1016/j.techfore.2012.10.005.

Lade, S.J., Haider, L.J., Engström, G., & Schlüter, M. (2017). Resilience offers escape from trapped thinking on poverty alleviation. Science Advances, 3(5), e1603043. https://doi.org/10.1126/sciadv.1603043.

Lade, S.J. & Niiranen, S. (2017). Generalized modeling of empirical social‐ecological systems. Natural Resource Modeling, 30(3), e12129. https://doi.org/10.1111/nrm.12129.

Lahsen, M., & Nobre, C.A. (2007). Challenges of connecting international science and local level sustainability efforts: the case of the Large-Scale Biosphere–Atmosphere Experiment in Amazonia. Environmental Science & Policy, 10(1), 62-74. https://doi.org/10.1016/j.envsci.2006.10.005.

Lahtinen, T.J., Guillaume, J.H., & Hämäläinen, R.P. (2017). Why pay attention to paths in the practice of environmental modelling? Environmental Modelling & Software, 92, 74-81. https://doi.org/10.1016/j.envsoft.2017.02.019.

Lamberson P.J., & Page, S.E. (2012). Tipping Points. Santa Fe Institute Working Papers.

Lane, D.C. & Kopainsky, B. (2017). Editorial - Natural Resource Management: Contributions of System Dynamics to Research, Policy and Implementation. Systems Research and Behavioral Science, 34, 378-385.

Lattuca, L.R. (2001). Creating Interdisciplinarity: Interdisciplinary Research and Teaching Among College and University Faculty (1st ed). Nashville: Vanderbilt University Press.

Lawrence Livermore National Laboratory. (2012). CF Conventions and Metadata. http://cfconventions.org/.

Lee J.-S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, J.G., Sun, Z., & Parker, D.C. (2015). The Complexities of Agent-Based Modeling Output Analysis. Journal of Artificial Societies and Social Simulation, 18(4). https://doi.org/10.18564/jasss.2897.

Lee, D.B. Jr. (1973). Requiem for large-scale models. Journal of the American Institute of Planners, 39, pp. 163–178. https://doi.org/10.1080/01944367308977851.

Lempert, R.J. (2002). A new decision sciences for complex systems. Proceedings of the National Academy of Sciences, 99 (suppl 3) 7309-7313. https://doi.org/10.1073/pnas.082081699.

Lempert, R.J., Groves, D.G., Popper, S.W., & Bankes, S.C. (2006). A General, Analytic Method for Generating Robust Strategies and Narrative Scenarios. Management science, 52(4): 514-528. https://doi.org/10.1287/mnsc.1050.0472.

Levin, S.A. (1992). The Problem of Pattern and Scale in Ecology: The Robert H. MacArthur Award Lecture. Ecology, 73, 1943-1967. https://doi.org/10.2307/1941447.

Levin, S., Xepapadeas, T., Crepin, A.-S., Norber, J., de Zeeuw, A., Folke, C., Hughes, T., Arrow, K., Barrett, S., Daily, G., Ehrlich, P., Kautsky, N., Maler, K.-G., Polasky, S., Troell, M., Vincent, J.R., & Walker, B. (2013). Social-ecological systems as complex adaptive systems: modeling and policy implications. Environment and Development Economics, 18(2), 111-132. https://doi.org/10.1017/S1355770X12000460.

Ligmann-Zielinska, A., Kramer, D.B., Cheruvelil, K.S., & Soranno, P.A. (2014). Using Uncertainty and Sensitivity Analyses in Socioecological Agent-Based Models to Improve Their Analytical Performance and Policy Relevance. PLOS ONE, 9(10), e109779. https://doi.org/10.1371/journal.pone.0109779.

Lippe, M., Bithell, M., Gotts, N., Natalini, D., Barbrook-Johnson, P., Giupponi, C., Hallier, M. Hofstede, G.J., Le Page, C., Matthews, R.B., Schlüter, M., Smith, P., Teglio, A., Thellmann, K. (2019). Using agent-based modelling to simulate social-ecological systems across scales. GeoInformatica, 23(2), 269-298.

Lokers, R., Knapen, R., Janssen, S., van Randen, Y., & Jansen, J. (2016). Analysis of Big Data technologies for use in agro-environmental science. Environmental Modelling & Software, 84, 494-504. https://doi.org/10.1016/j.envsoft.2016.07.017.

Luvuno, L., Biggs, R., Stevens, N., & Esler, K. (2018). Woody encroachment as a social-ecological regime shift. Sustainability 10(7): 2221. https://doi.org/10.3390/su10072221.

Lum, K. (2017). Limitations of Mitigating Judicial Bias with Machine Learning. Nature Human Behaviour, 1 (7): 0141. https://doi.org/10.1038/s41562-017-0141.

Lum, K., & Isaac, W., 2016. To Predict and Serve? Significance 13 (5), 14–19. https://doi.org/10.1111/j.1740-9713.2016.00960.x.

MacMynowski, D.P. (2007). Pausing at the brink of interdisciplinarity: power and knowledge at the meeting of social and biophysical science. Ecology and Society, 12(1): 20. http://www.ecologyandsociety.org/vol12/iss1/art20.

Mai, J.-E. (2016). Big Data Privacy: The Datafication of Personal Information. The Information Society, 32 (3): 192–99. https://doi.org/10.1080/01972243.2016.1153010.

Maier, H.R., Guillaume, J.H.A., van Delden, H., Riddell, G.A., Haasnoot, M., & Kwakkel, J.H. (2016). An uncertain future, deep uncertainty, scenarios, robustness and adaptation: How do they fit together? Environmental Modelling & Software, 81, 154–164. https://doi.org/10.1016/j.envsoft.2016.03.014.

Maliapen, M. (2009). Clinical Genomics Data Use: Protecting Patients Privacy Rights. Studies in Ethics, Law, and Technology, 3 (1). https://doi.org/10.2202/1941-6008.1080.

Mallavarapu, A., Lyons, L., Shelley, T., Zellner, M., Minor, E., & Slattery, B. (2015). Developing Computational Methods to Measure and Track Learners’ Spatial Reasoning in an Open-Ended Simulation. Journal of Educational Data Mining, 7(2), 49-82. https://jedm.educationaldatamining.org/index.php/JEDM/article/view/JEDM122.

Manson, S.M., & Evans, T. (2007). Agent-based modeling of deforestation in southern Yucatán, Mexico, and reforestation in the Midwest United States. Proceedings of the National Academy of Sciences, 104(52), 20678–20683. https://doi.org/10.1073/pnas.0705802104.

Mattmann, C.A. (2013). Computing: A vision for data science. Nature, 493(7433), 473–475. https://doi.org/10.1038/493473a.

Matthews, R.B., Gilbert, N.G., Roach, A., Polhill, J.G., & Gotts, N.M. (2007). Agent-based land-use models: A review of applications. Landscape Ecology, 22(10), pp. 1447-1459.

McPhail, C., Maier, H.R., Kwakkel, J.H., Giuliani, M., Castelletti, A., & Westra, S. (2018). Robustness Metrics: How Are They Calculated, When Should They Be Used and Why Do They Give Different Results? Earth’s Future, 6, 169-191. https://doi.org/10.1002/2017EF000649.

Melgar, L.E.A., Lalith, M., Hori, M., Ichimura, T., & Tanaka, S. (2014). A Scalable Workbench for Large Urban Area Simulations, Comprised of Resources for Behavioural Models, Interactions and Dynamic Environments. In PRIMA 2014: Principles and Practice of Multi-Agent Systems, 166–81. Lecture Notes in Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-319-13191-7_14.

Merz, B., Vorogushyn, S., Lall, U., Viglione, A., & Blöschl, G. (2015). Charting unknown waters—On the role of surprise in flood risk assessment and management. Water Resources Research, 51(8), 6399–6416. https://doi.org/10.1002/2015WR017464.

Meyer, R.J., Daniels, R.J., Kettl, D.F., & Kunreuther, H. (2006). Why we Under-Prepare for Hazards. In On Risk and Disaster: Lessons from Hurricane Katrina (pp. 156–57). Philadelphia: University of Pennsylvania Press.

Midgley, G., Cavana, R.Y., Brocklesby, J., Foote, J.L., Wood, D.R.R., & Ahuriri-Driscoll, A. (2013). Towards a new framework for evaluating systemic problem structuring methods. European Journal of Operational Research, 229(1), 143–154. https://doi.org/10/f2z29n.

Miller, T.R.,Baird, T.D., Littlefield, C.M., Kofinas, G., Chapin III, F., & Redman, C.L. (2008). Epistemological pluralism: reorganizing interdisciplinary research. Ecology and Society, 13(2), 46. http://www.ecologyandsociety.org/vol13/iss2/art46.

Millington, J.D.A., & Wainwright, J. (2017). Mixed qualitative-simulation methods: Understanding geography through thick and thin. Progress in Human Geography, 41(1), 68–88. https://doi.org/10/f9r4k3.

Milz, D. (2015). Mismatched Scales, Mismatched Intentions: Regional Wastewater Planning on Cape Cod, Massachusetts, USA. PhD, University of Illinois at Chicago.

Milz, D., Zellner, M., Hoch, C., Radinsky, J., Pudlock, K., & Lyons, L. (2017). Reconsidering Scale: Using Geographic Information Systems to Support Spatial Planning Conversations. Planning Practice & Research, https://doi.org/10.1080/02697459.2017.1378979.

Mingers, J. (2001). Combining IS Research Methods: Towards a Pluralist Methodology. Information Systems Research, 12(3), 240–259. https://doi.org/10/dx628v.

Mingers, J. (2006). Philosphical foundations: critical realism. In: Mingers, J. Realising system thinking: knowledge and action in management science. Springer. pp. 11-31.

Moore, M.-L., Tjornbo, O., Enfors, E., Knapp, C., Hodbod, J., Baggio, J.A., Norström, A., Olsson, P., & Biggs, D. (2014). Studying the complexity of change: toward an analytical framework for understanding deliberate social-ecological transformations. Ecology and Society, 19(4), 54. http://dx.doi.org/10.5751/ES-06966-190454.

Morgan, M. G., Fischoff, B., Bostrom, A., & Atman, C. (2002). Risk communication: A mental models approach, Cambridge University Press, New York.

Muller C.L., Chapman L., Johnston S., Kidd C., Illingworth S., Foody G., Overeem A., & Leigh R.R. (2015). Crowdsourcing for Climate and Atmospheric Sciences: Current Status and Future Potential. International Journal of Climatology, 35 (11), 3185–3203. https://doi.org/10.1002/joc.4210.

Müller-Hansen, F., Schlüter, M., Mäs, M., Donges, J. F., Kolb, J. J., Thonicke, K., and Heitzig, J., 2017. Towards representing human behavior and decision making in Earth system models – an overview of techniques and approaches. Earth System Dynamics, 8, 977-1007. https://doi.org/10.5194/esd-8-977-2017.

National Research Council. (2006). Facing Hazards and Disasters: Understanding Human Dimensions. National Academies Press. Retrieved from http://www.nap.edu/catalog/11671/facing-hazards-and-disasters-understanding-human-dimensions.

National Research Council. (2007). Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. Washington, DC: National Academies Press. https://doi.org/10.17226/11699.

Nordhaus, W. (2018). Evolution of modeling of the economics of global warming: changes in the DICE model, 1992–2017. Climatic Change, 148(4), 623-640.

O’Neil, C. (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Reprint edition. Broadway Books.

O'Neill, B.C., Kriegler, E., Ebi, K.L., Kemp-Benedict, E., Riahi, K., Rothman, D.S., van Ruijven, B.J., van Vuuren, D.P., Birkmann, J., Kok, K., Levy, M., & Solecki, W. (2017). The roads ahead: Narratives for shared socioeconomic pathways describing world futures in the 21st century. Global Environmental Change, 42, 169-180

Paget, N., Bonté, B., Barreteau, O., Pigozzi, G., & Maurel, P. (2019). An in-silico analysis of information sharing systems for adaptable resources management: a case study of oyster farmers. Socio-Environmental Systems Modelling, 1. https://doi.org/10.18174/sesmo.2019a16166.

Pahl-Wostl, C., Arthington, A., Bogardi, J., Bunn, S.E., Hoff, H., Lebel, L., Nikitina, E., Palmer, M., Poff, L.N., Richards, K., Schluter, M., Schulze, R., St-Hilaire, A., Tharme, R., Tockner, K., & Tsegai, D. (2013). Environmental flows and water governance: managing sustainable water uses. Environmental Sustainability, 5, 341–351. https://doi.org/10.1016/j.cosust.2013.06.009.

Parker, A.M., Srinivasan, S.V., Lempert, R.J., & Berry, S.H. (2015). Evaluating simulation-derived scenarios for effective decision support. Technological Forecasting and Social Change, 91, 64–77. https://doi.org/10/gc3f69.

Peckham, S.D. (2014). The CSDMS Standard Names: Cross-Domain Naming Conventions for Describing Process Models, Data Sets and Their Associated Variables. Proceedings of the Seventh International Congress on Environmental Modeling and Software.

Pesenson, M.Z., Pesenson, I.Z., & McCollum, B. (2010) The Data Big Bang and the Expanding Digital Universe: High-Dimensional, Complex and Massive Data Sets in an Inflationary Epoch. Advances in Astronomy, vol. 2010, Article ID 350891. https://doi.org/10.1155/2010/350891.

Peterson, G.D., Carpenter, S.R., & Brock, W.A. (2003). Uncertainty and the management of multistate ecosystems: An apparently rational route to collapse. Ecology, 84(6), 1403-1411.

Pindyck, R.S. (2013). Climate Change Policy: What Do the Models Tell Us? Journal of Economic Literature, 51, 860-872. https://doi.org/10.1257/jel.51.3.860.

Platt, J.R. (1964). Strong Inference. Science, 146(3642), 347–353. https://doi.org/10.1126/science.146.3642.347.

Plummer, M.L. (2009) Assessing benefit transfer for the valuation of ecosystem services. Frontiers in Ecology and the Environment, 7(1), 38-45.

Poggio, L., Simonetti, E., & Gimona, A. (2018) Enhancing the WorldClim data set for national and regional applications. Science of the Total Environment, 625, 1628-1643.

Polhill, J.G., Sutherland, L.-A., & Gotts, N.M. (2009). Using Qualitative Evidence to Enhance an Agent-Based Modelling System for Studying Land Use Change. Journal of Artificial Societies and Social Simulation, 13(2), 10.

Pollino, C.A., White, A.K. & Hart, B.T. (2007). Examination of conflicts and improved strategies for the management of an endangered Eucalypt species using Bayesian networks. Ecological Modelling, 201, 37-59.

Poteete, A.R., Janssen, M.A., & Ostrom, E. (2010). Working Together: Collective Action, the Commons, and Multiple Methods in Practice. Princeton, N.J: Princeton University Press.

Quinn, J. D., Reed, P. M., Giuliani, M., & Castelletti, A. (2017). Rival framings: A framework for discovering how problem formulation uncertainties shape risk management trade‐offs in water resources systems. Water Resources Research, 53(8), 7208-7233.

Radinsky, J., Milz, D., Zellner, M., Pudlock, K., Witek, C., Hoch, C., & Lyons, L. (2017). How planners and stakeholders learn with visualization tools: using learning sciences methods to examine planning processes, Journal of Environmental Planning and Management, 60(7), 1296-1323, https://doi.org/10.1080/09640568.2016.1221795.

Railsback, S.F. & Grimm, V. (2011). Agent-based and Individual-based modeling: a practical introduction. Princeton: Princeton University Press, 352pp.

Rasmussen, C.E., & Williams, C.K.I (2006). Gaussian Processes for Machine Learning, the MIT Press, ISBN 026218253X. http://www.gaussianprocess.org/gpml.

Refsgaard, J.C., van der Sluijs, J.P., Hojberg, A.L., & Vanrolleghem, P.A. (2007). Uncertainty in the environmental modelling process – a framework and guidance. Environmental Modelling & Software, 22, 1543-1556. https://doi.org/10.1016/j.envsoft.2007.02.004.

Renard, B., Kavetski, D., Kuczera, G., Thyer, M., & Franks, S.W. (2010). Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors. Water Resources Research, 46, W05521. https://doi.org/10.1029/2009WR008328.

Robinson, D. T., Brown, D. G., Parker, D. C., Schreinemachers, P., Janssen, M. A., Huigen, M., Wittmer, H., Gotts, N., Promburom, P., Irwin, E., Berger, T., Gatzweiler, F., & Barnaud, C. (2007). Comparison of empirical methods for building agent-based models in land use science. Journal of Land Use Science, 2(1), 31–55. https://doi.org/10/bnvs8h.

Robinson, D.T., Di Vittorio, A., Alexander, P., Arneth, A., Barton, C.M., Brown, D.G., Kettner, A.J., Lemmen, C., O’Neill, B.C., Janssen, M., Pugh, T.A.M., Rabin, S.S., Rounsevell, M., Syvitski, J.P.M., Ullah, I., & Verburg, P.H. (2018). Modelling feedbacks between human and natural processes in the land system. Earth System Dynamics, 9, 1–47. https://doi.org/10.5194/esd-2017-68.

Rosenberg, M., Confessore, N., and Cadwalladr, C. (2018). How Trump Consultants Exploited the Facebook Data of Millions. The New York Times, March 17, 2018, sec. Politics. https://www.nytimes.com/2018/03/17/us/politics/cambridge-analytica-trump-campaign.html.

Rothstein, M.A. (2010). Is Deidentification Sufficient to Protect Health Privacy in Research? The American Journal of Bioethics, 10, 9, 3–11. https://doi.org/10.1080/15265161.2010.494215.

Rounsevell, M. et al. (2016) Linking Earth System Dynamics and Social System Modeling. Report of the Computational Human and Earth Systems Science (CHESS) group from the Workshop: ‘Linking Earth System Dynamics and Social System Modeling’, 23-25 May 2016, Boulder, Colorado, USA. Retrieved from: https://csdms.colorado.edu/wiki/CHESS on August 6th, 2018.

Sætra, H.S. (2017). Exploring the use of agent-based modelling in mixed methods research. Barataria. Revista Castellano-Manchega de Ciencias Sociales, 22, 15-31. https://doi.org/10.20932/barataria.v0i22.337.

Saltelli, A., Ratto, M., Tarantola, S., & Campolongo, F. (2006). Sensitivity analysis practices: Strategies for model-based inference. Reliability Engineering & System Safety, 91, 10-11, 1109-1125. https://doi.org/10.1016/j.ress.2005.11.014.

Samourkasidis, A., & Athanasiadis, I.N. (2016). A miniature data repository on a Raspberry Pi. Electronics, 6(1), 1. https://doi.org/10.3390/electronics6010001.

Sarewitz, D. (2004). How science makes environmental controversies worse. Environmental Science & Policy, 7(5), 385-403. https://doi.org/10.1016/j.envsci.2004.06.001.

Sayama, H., Pestov, I., Schmidt, J., Bush, B.J., Wong, C., Yamanoi, J., Gross, T. (2013). Modeling complex systems with adaptive networks. Computers & Mathematics with Applications, 65(10), 1645-1664.

Schade, S., Tsinaraki, C., & Roglia, E. (2017). Scientific data from and for the citizen. First Monday, 22(8). https://doi.org/10.5210/fm.v22i8.7842

Scheffer, M. (2009). Critical transitions in nature and society. Princeton University Press, Princeton.

Scheffer, M., & Carpenter, S.R. (2003). Catastrophic regime shifts in ecosystems: linking theory to observation. Trends in Ecology & Evolution, 18:648–656. https://doi.org/10.1016/j.tree.2003.09.002.

Schimel, D., K. Hibbard, D. Costa, P. Cox, & S. van der Leeuw (2015). Analysis, Integration and Modeling of the Earth System (AIMES): Advancing the post-disciplinary understanding of coupled human–environment dynamics in the Anthropocene. Anthropocene, 12, p 99–106.

Schmolke, A., Thorbek, P., DeAngelis, D.L., Grimm, V. (2010). Ecological modelling supporting environmental decision making: a strategy for the future. Trends in Ecology & Evolution 25 (8), 479-486.

Scholes, R.J., Reyers, B., Biggs, R., Spierenburg, M.J., & Duriappah, A. (2013). Multi-scale and cross-scale assessments of social–ecological systems and their ecosystem services. Current Opinion in Environmental Sustainability, 5(1), 16-25. https://doi.org/10.1016/j.cosust.2013.01.004.

Schulze, J., Müller, B., Groeneveld, J., & Grimm, V. (2017). Agent-based modelling of social-ecological systems: Achievements, challenges, and a way forward. Journal of Artificial Societies and Social Simulation, 20(2), 8.

Schlüter, M., Müller, B., & Frank, K. (2019). The potential of models and modeling for social-ecological systems research: the reference frame ModSES. Ecology and Society 24(1).

Schlüter, M., Hinkel, J. Bots, P. W. G. & Arlinghaus, R. (2014). Application of the SES framework for model-based analysis of the dynamics of social-ecological systems. Ecology and Society, 19(1), 36. https://doi.org/10.5751/ES-05782-190136.

Schlüter, M., Baeza, A., Dressler, G., Frank, K., Groeneveld, J., Jager, W., Janssen, M.A., McAllister, R.R.J., Muller, B., Orzch, K., Schwarz, N., & Wijermans, N. (2017). A framework for mapping and comparing behavioural theories in models of social-ecological systems. Ecological Economics, 131, 21–35. https://doi.org/10.1016/j.ecolecon.2016.08.008.

Sætra, H.S., 2017. Exploring the use of agent-based modelling in mixed methods research. Barataria. Revista Castellano-Manchega de Ciencias Sociales, 22, 15–31. https://doi.org/10.20932/barataria.v0i22.337.

Smajgl, A., & Barreteau, O. (2014). Empiricism and Agent-Based Modelling. In A. Smajgl & O. Barreteau (Eds.), Empirical Agent-Based Modelling - Challenges and Solutions (pp. 1–26). Springer New York. https://doi.org/10.1007/978-1-4614-6134-0_1.

Sterling, E., Zellner, M., Leong, K., Jenni, K., Gray, S., Jordan, R., Bendor, T., Jetter, A., Schmitt-Olabisi, L, Paolisso, M., Hubacek, K., Bommel, P., & Bammer, G. (2019). Try, try again: Lessons learned from success and failure in participatory modeling. Elementa, 7.

Stern, N., (2016). Economics: Current climate models are grossly misleading. Nature, 530, 407-409. https://doi/org/10.1038/530407a.

Stodden, V., Seiler, J., & Ma, Z. (2018). An empirical analysis of journal policy effectiveness for computational reproducibility, PNAS, 115, 2583-2589. https://doi.org/10.1073/pnas.1708290115.

Sudret, B. (2008). Global sensitivity analysis using polynomial chaos expansions. Reliability Engineering & System Safety. 93(7), 964-979. https://doi.org/10.1016/j.ress.2007.04.002.

Taleb, N. N. (2007). The black swan: The impact of the highly improbable. Random house.

Tenkanen, H., Di Minin, E., Heikinheimo, V., Hausmann, A., Herbst, M., Kajala, L., & Toivonen, T. (2017). Instagram, Flickr, or Twitter: Assessing the usability of social media data for visitor monitoring in protected areas. Scientific Reports, 7, 17615. https://doi.org/10.1038/s41598-017-18007-4.

Thakuriah, P., Tilahun, N., & Zellner, M. (2016). Big Data and Urban Informatics: Innovations and Challenges to Urban Planning and Knowledge Discovery. In Seeing Cities through Big Data: Research Methods and Applications in Urban Informatics, edited by P. Thakuriah, Tilahun, N. and Zellner, M., Springer.

Tolk, A. (2015). The next generation of modeling & simulation: integrating big data and deep learning. In Proceedings of the conference on summer computer simulation (pp. 1-8). Society for Computer Simulation International.

Tigchelaar, M., Battisti, D.S., Naylor, R.L., Ray, D.K. (2018). Future warming increases probability of globally synchronized maize production shocks. Proceedings of the National Academy of Sciences of the United States of America, 115(26), pp. 6644-6649.

Trenholm, R., Lantz, V., Martinez-Espineira, R., & Little, S. (2013). Cost-benefit analysis of riparian protection in an eastern Canadian watershed. Journal of Environmental Management, 116(15), 81-94. https://doi.org/10.1016/j.jenvman.2012.11.039.

Trutnevyte, E., Guivarch, C., Lempert, R., & Strachan, N. (2016). Reinvigorating the scenario technique to expand uncertainty consideration. Climatic Change, 135(3–4), 373–379. https://doi.org/10.1007/s10584-015-1585-x.

Ullah, I.I.T., Kuijt, I., & Freeman, J. (2015). Toward a theory of punctuated subsistence change. PNAS, 112, 9579–9584. https://doi.org/10.1073/pnas.1503628112.

Ulrich, W. (2013). Critical systems thinking. Encyclopedia of Operations Research and Management Science, 314-326. https://doi.org/10.1007/978-1-4419-1153-7.

UN Global Pulse, Social Media and Forced Displacement (2017a). Big Data Analytics & Machine-Learning, White Paper, UNHCR Innovation Service.

UN Global Pulse, Social Media and Forced Displacement (2017b). Social Media and Forced Displacement: Big Data Analytics & Machine-Learning, White Paper, UNHCR Innovation Service.

van der Sluijs, J.P., Craye, M., Funtowicz, S., Kloprogge, P., Ravetz, J., & Risbey, J. (2005). Combining Quantitative and Qualitative Measures of Uncertainty in Model‐Based Environmental Assessment: The NUSAP System. Risk Analysis, 25(2): 481-492. https://doi.org/10.1111/j.1539-6924.2005.00604.x.

van Ittersum, M.K., Cassman, K.G., Grassini, P., Wolf, J., Tittonell, P., & Hochman, Z. (2013). Yield gap analysis with local to global relevance—A review. Field Crops Research, 143, 4-17. https://doi.org/10.1016/j.fcr.2012.09.009.

van Nes, E.H., Arani, B.M.S., Staal, A., van der Bolt, B., Flores, B.M., Bathiany, S., & Scheffer, M. (2016). What Do You Mean, ‘Tipping Point’? Trends in Ecology & Evolution, 31, 902–904. https://doi.org/10.1016/j.tree.2016.09.011.

van Vliet, J., Hurkens, J., White, R., van Delden, H. (2012) An activity-based cellular automaton model to simulate land-use dynamics, Environment and Planning B: Planning and Design 39(2), pp. 198-212

van Zanten, B.T., van Berkel, D.B., Meentemeyer, R.K., Smith, J.W., Tieskens, K.F., & Verburg, P.H. (2016). Continental Scale Quantification of Landscape Values Using Social Media Data. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.1614158113.

Vaughan, D. (1996). The Challenger Launch Decision: Risky Technologies, Deviance, and Culture at NASA. Chicago, University of Chicago Press.

Verburg, P.H., Soepboer, W., Veldkamp, A., Limpiada, R., Espaldon, V., & Mastura, S.S.A. (2002). Modeling the spatial dynamics of regional land use: The CLUE-S model, Environmental Management, 30(3), pp. 391-405.

Verburg, P.H., Dearing, J.A., Dyke, J.G., van der Leeuw, S., Seitzinger, S., Steffen, W., & Syvitski, J.P.M. (2016). Methods and approaches to modelling the Anthropocene. Global Environmental Change, 39, 328-340. https://doi.org/10.1016/j.gloenvcha.2015.08.007.

Vereecken, H., R. Kasteel, J. Vanderborght, & Harter, T. (2007). Upscaling Hydraulic Properties and Soil Water Flow Processes in Heterogeneous Soils. Vadose Zone Journal, 6:1-28. https://doi.org/10.2136/vzj2006.0055.

Vermaat, J.E., Eppink, F., van den Bergh, J.C.J.M., Barendregt, a., & svan Belle, J. (2005). Aggregation and the matching of scales in spatial economics and landscape ecology: empirical evidence and prospects for integration. Ecological Economics, 52(2): 229-237. https://doi.org/10.1016/j.ecolecon.2004.06.027.

Villa, F., Balbi, S., Athanasiadis, I.N., & Caracciolo, C. (2017). Semantics for interoperability of distributed data and models: Foundations for better-connected information. F1000Research, 6, 686. https://doi.org/10.12688/f1000research.11638.1.

Voinov, A., & Bousquet, F. (2010). Modelling with stakeholders. Environmental Modelling & Software, 25(11), 1268-1281. https://doi.org/10.1016/j.envsoft.2010.03.007.

Voinov, A., Kolagani, N., McCall, M.K., Glynn, P.D., Kragt, M.E., Ostermann, F.O., Pierce, S.A., & Ramu, P. (2016). Modelling with stakeholders - Next generation. Environmental Modelling & Software, 77, 196-220.

Voinov, A., Jenni, K., Gray, S., Kolagani, N., Glynn, P., Bommel, P., Prell, C., Zellner, M., Paolisso, M., Jordan, R., Sterling, E., Schmitt Olabisi, L., Giabbanelli, P., Sun, Z., Le Page, C., Elsawah, S., BenDor, T., Hubacek, K., Laursen, B., Jetter, A., Basco Carrera, L., Singer, A., Young, L., & Brunacini, J. (2018). Tools and methods in participatory modeling: selecting the right tool for the job. Environmental Modelling & Software, 109, 232-250.

Voinov, A., & Shugart, H.H. (2013). 'Integronsters', integral and integrated modelling. Environmental Modelling & Software, 39, 149-158.

Vrugt, J.A., ter Braak, C.J.F., Clark, M.P., Hyman, J.M., & Robinson, B.A. (2008). Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation, Water Resources Research, 44, W00B09. https://doi.org/10.1029/2007WR006720.

Wagener, T. & Wheater, H.S. (2006). Parameter estimation and regionalization for continuous rainfall-runoff models including uncertainty. Journal of Hydrology, 320, 132-154.

Walker, B., Carpenter, S., Anderies, J., Abel, N., Cumming, G., Janssen, M., Lebel, L., Norberg, J., Peterson, G.D., & Pritchard, R. (2002). Resilience Management in Social-ecological Systems: a Working Hypothesis for a Participatory Approach. Conservation Ecology, 6(1): 14. https://www.jstor.org/stable/26271859.

Walker, W.E., Harremoës, P., Rotmans, J., Van der Sluijs, J.P., Van Asselt, M.B.A., Janssen, P., & Krayer von Krauss, M.P. (2003) Defining Uncertainty A Conceptual Basis for Uncertainty Management in Model-Based Decision Support, Integrated Assessment, 4(1), 5-17.

Walter, E., & Pronzato, L. (1997). Identification of Parametric Models from Experimental Data. Springer Verlag, Berlin.

Wang, T. (2013). Big Data Needs Thick Data. http://ethnographymatters.net/blog/2013/05/13/big-data-needs-thick-data/.

Wang, W., Rothschild, D., Goel, S., & Gelman, A. (2015). Forecasting Elections with Non-Representative Polls. International Journal of Forecasting, 31 (3), 980–91. https://doi.org/10.1016/j.ijforecast.2014.06.001.

Watkins, M. D., & Bazerman, M. H. (2003). Predictable surprises: The disasters you should have seen coming. Harvard business review, 81(3), 72-85.

Weeg, C., Schwartz, H.A., Hill, S., Merchant, R.M., Arango, C., & Ungar, L. (2015). Using Twitter to Measure Public Discussion of Diseases: A Case Study. JMIR Public Health and Surveillance, 1 (1): e6. https://doi.org/10.2196/publichealth.3953.

Wilensky, U., & Rand W. (2007). Making Models Match: Replicating an Agent-Based Model. Journal of Artificial Societies and Social Simulation, 10(4), 2. http://jasss.soc.surrey.ac.uk/10/4/2.html.

Willcock, S., Martínez-López, J., Hooftman, D.A.P., Bagstad, K.J., Balbi, S., Marzo, A., Prato, C., Sciandrello, S., Signorello, G., Voigt, B., Villa, F., Bullock, J.M., & Athanasiadis, I.N. (2018). Machine Learning for Ecosystem Services, Ecosystem Services, https://doi.org/10.1016/j.ecoser.2018.04.004.

Wilson, J., Low, B., Costanza, R., & Ostrom, E. (1999). Scale misperceptions and the spatial dynamics of a social–ecological system. Ecological Economics, 31, 243–257.

Winsemius, H.C., Van Beek, L.P.H., Jongman, B., Ward, P.J., & Bouwman, A. (2013). A framework for global river flood risk assessments, Hydrology and Earth System Sciences, 17, 1871-1892, https://doi.org/10.5194/hess-17-1871-2013.

Yang, J., Jakeman, A., Fang, G., & Chen, X. (2018). Uncertainty analysis of a semi-distributed hydrologic model based on a Gaussian Process emulator. Environmental Modelling & Software, 101, 289-300.

Zare, F., Elsawah, S., Iwanaga, T., Jakeman, A.J., & Pierce, S.A. (2017) Integrated water assessment and modelling: a bibliometric analysis of trends in the water resources sector. Journal of Hydrology, 552, 765-778.

Zellner, M.L. (2008). Embracing Complexity and Uncertainty: The Potential of Agent-Based Modeling for Environmental Planning and Policy. Planning Theory & Practice, 9 (4), 437-457. https://doi.org/10.1080/14649350802481470.

Zellner, M.L., & Campbell, S. (2015). Planning for Deep-Rooted Problems: What Can We Learn from Aligning Complex Systems and Wicked Problems? Planning Theory and Practice, 16 (4), 457-478. https://doi.org/10.1080/14649357.2015.1084360.

Zellner, M.L., Lyons, L., Hoch, C J., Weizeorick, J., Kunda, C., & Milz, D. (2012). Modeling, Learning and Planning Together: An Application of Participatory Agent-Based Modeling to Environmental Planning. URISA Journal, GIS in Spatial Planning Issue, 24 (1), 77-92.

Zellner, M., Lyons, L., Milz, D., Shelley, J., Hoch, C., Massey, D., & Radinsky, J., in press. Participatory Complex Systems Modeling for Environmental Planning: Opportunities and Barriers to Learning and Policy Innovation. In Porter, Zhao, Smitt Olabisi, and McNall (Eds.) Innovations in Collaborative Modeling, Michigan State University Press.

Zhang, X., Holt, J.B., Yun, S., Lu, H., Greenlund, K.J., & Croft, J.B. (2015). Validation of Multilevel Regression and Post stratification Methodology for Small Area Estimation of Health Indicators From the Behavioral Risk Factor Surveillance System. American Journal of Epidemiology, 182 (2), 127–37. https://doi.org/10.1093/aje/kwv002.