Transparent and comprehensive ecological modeling documentation
Schmolke A, Thorbek P, DeAngelis DL & Grimm V (2010) Ecological models supporting environmental decision making: a strategy for the future. Trends in Ecology & Evolution, 25: 479-486. DOI: 10.1016/j.tree.2010.05.001
Why TRACE?
Ecological models are becoming increasingly important in the context of chemical risk assessments in particular and environmental decision support in general. However, no general guidelines exist for their development and use. Such guidelines for good modeling practice would be essential for quality assurance of ecological models in the context of environmental decision support, and would provide a tool for regulatory agencies to assess the usefulness of models in specific contexts. We present the first step towards the implementation of a good modeling practice: a standard framework for the transparent and comprehensive documentation of ecological models and the underlying modeling process – the TRACE documentation.
From the modeling cycle to the TRACE documentation (from Schmolke et al. 2010)
Goals and benefits of TRACE documentations
• Standard framework for transparent and comprehensive ecological modeling (TRACE) documentation
• Documentation encompasses the whole modeling process: model development, testing and analysis, and application
• Template for a modeling notebook for day-to-day documentation
• Organization of the modeling process by modelers
• Facilitates assessment of model quality and suitability by decision makers
Role of TRACE in CREAM
The TRACE documentation framework can only become established as a standard if it is applied and refined by numerous projects. The CREAM project encompasses more than a dozen ecological modeling projects with the objective of application in chemical risk assessment. All modeling projects are conducted in close collaboration with decision makers from regulatory authorities and industry, and will apply the TRACE framework for documentation. Accordingly, CREAM currently functions as a test bed, and at present is producing a collection of examples for TRACE.
Document structure
I. Model development
- Problem formulation: Context in which the model will be used, and the type of audience addressed; specification of the question(s) that should be answered with the model; statement of the domain of applicability of the model, including the extent of acceptable extrapolations; assessment of the availability of knowledge and data; specification of necessary model outputs.
- Design and formulation: Description of the conceptual model; description and justification of the modeling approach used and of the complexity; entities and processes represented in the model; most important, the applied assumptions about the system.
- Model description: Detailed description of the actual model and how it has been implemented (programs, software platforms, scripts).
- Parameterization: List of all parameter values used in the model, the data sources, and how the parameter values were obtained or calculated; uncertainties associated with each parameter.
- Calibration: Documentation of the data sets used for calibration; which parameters were calibrated; what optimization method was used.
II. Model testing and analysis
- Verification: Assessment of whether the model is working according to its specifications; documentation of what tests have been conducted.
- Sensitivity analysis: Exploration of the model behavior for varying parameters; documentation of which parameter combinations have been tested; justification of used parameter ranges and combinations.
- Validation: Comparison of model or submodel outputs with empirical data that were not used for parameterization or calibration; documentation of data sources; what parts (submodels) have
been validated; what validation methods were applied.
III. Model application
- Results: Outputs that are used to inform decisions; description of simulation experiments (scenarios) conducted; statistics applied to analyze model outputs.
- Uncertainty analysis: Uncertainties in model outputs used for recommendations; description of variance, noise, and bias in empirical data; determination of stochasticity in the model; description of model uncertainty which can be assessed through application of different models or submodels; best- and worst-case scenarios.
- Recommendation: Description of how initial question(s) could be answered; summary of conclusions drawn from model; clarification of extrapolations used (in time and space).
The modeling notebook
A modeling notebook is a day-to-day documentation of modeling projects. In empirical research, lab notebooks are indispensable parts of research. This routine needs to be adopted by ecological modelers in order to make the progress of modeling projects tracebale to peers. The modeling notebook should be organized in the same way as the final TRACE documentation.
TRACE Guidance (download)
Upcoming
• Examples of TRACE documentations as produced by CREAM projects will be prepared for publication in due term (intended publication date: second half of 2011).
