Transparent and comprehensive model evaludation
Ecological models are becoming increasingly important in the context of chemical risk assessments in particular and environmental decision support in general. However, no general guidelines exist for their development and use. Such guidelines for good modeling practice would be essential for quality assurance of ecological models in the context of environmental decision support, and would provide a tool for regulatory agencies to assess the usefulness of models in specific contexts. We present the first step towards the implementation of a good modeling practice: a standard framework for the transparent and comprehensive documentation of ecological models and the underlying modeling process – the TRACE documentation.
Goals and benefits of TRACE documentations
• Standard framework for the documentation of transparent and comprehensive model evaludation (TRACE)
• Documentation encompasses the whole modeling process: model development, testing and analysis, and application
• Template for a modeling notebook for day-to-day documentation
• Organization of the modeling process by modelers
• Facilitates assessment of model quality and suitability by decision makers
A short history of TRACE
In a programmatic article (Schmolke et al. 2010), members of the CREAM consortium introduced a new concept for developing and establishing Good Modelling Practice (GoMP), called TRACE (TRansparent And Comprehensive Ecological modelling) documentation. Virtually all previous attempts to establish GoMP failed because they tried to impose certain protocols which required additional resources but provided no direct benefits to modellers. In contrast, TRACE is focussed on documenting, in a systematic and standardised way, what thoughtful and experienced modellers are doing as part of model development: formulating a model’s purpose, developing a conceptual model, implementing and testing the model as a set of equations or computer program, analysing and understanding the model, and answering the questions addressed with the model.
TRACE was then used and tested in several CREAM projects. As a result, the format of TRACE was revised (Grimm et al. 2014) by basing it on a new concept, developed in CREAM: model “evaludation”, which is a merger of model evaluation and validation (Augusiak et al. 2014). Accordingly, TRACE documents provide supporting information that a model was thoughtfully designed, correctly implemented, thoroughly tested, well understood, and appropriately used for its intended purpose. The revised TRACE was published together with a new, detailed template for TRACE documents, and with three example TRACE documents. TRACE now stands for “TRansparent And Comprehensive model Evaludation”.
TRACE was met with great interest, in particular by regulatory authorities, who need protocols to assess whether or not a model is suitable for being taken into account in risk assessments. A working group of the panel for Plant Protection Products and their Residues at EFSA (European Food Safety Authority) developed and published guidance for Good Modelling Practice; several members of the CREAM consortium were part of this working group, and the final guidance is to a large degree based on the ideas underlying TRACE.
Download TRACE-related papers, templates and examples: Zip file
Schmolke A, Thorbek P, DeAngelis DL & Grimm V (2010) Ecological models supporting environmental decision making: a strategy for the future. Trends in Ecology and Evolution 25: 479-486.
Grimm V, Augusiak J, Focks A, Frank BM, Gabsi F, Johnston ASA, Liu C, Martin BT, Meli M, Radchuk V, Thorbek P, Railsback SF (2014) Towards better modelling and decision support: documenting model development, testing, and analysis using TRACE. Ecological Modelling 280:129-139.
Augusiak J, Van den Brink PJ, Grimm V (2014) Merging validation and evaluation of ecological models to ‘evaludation’: a review of terminology and a practical approach. Ecological Modelling 280: 117-128.