What was CREAM?
Objectives of CREAM
Current regulatory risk assessment of chemicals in Europe is based on the ratio between expected exposure in the environment and toxicity endpoints, e.g. hazard quotient, PNEC or TER. However, toxicity is usually determined at the level of individuals under laboratory conditions for a small number of standard test species. It remains unclear how well hazard quotients predict risks to populations and ecosystems, which are important environmental protection goals of current regulations in Europe. Since empirical approaches are too limited to solve this problem, mechanistic effect models are needed to make chemical risk assessment more ecologically relevant. To make this possible, CREAM is designed to (1) develop guidance for Good Modelling Practice which makes modelling more transparent, consistent, comprehensive, and reliable; (2) test and demonstrate the added value of models in a number of case studies; (3) train more researchers in both modelling and regulatory risk assessment so they can be involved in future model-based risk assessments in academia, industry, and regulatory authorities.
Description of work of CREAM
Good Modelling Practice
In a programmatic article, members of the CREAM consortium introduced a new concept for developing and establishing Good Modelling Practice (GoMP), called TRACE (TRansparent And Comprehensive Ecological modelling) documentation. Virtually all previous attempts to establish GoMP failed because they tried to impose certain protocols which required additional resources but provided no direct benefits to modellers. In contrast, TRACE is focussed on documenting, in a systematic and standardised way, what thoughtful and experienced modellers are doing as part of model development: formulating a model’s purpose, developing a conceptual model, implementing and testing the model as a set of equations or computer program, analysing and understanding the model, and answering the questions addressed with the model. This article appeared in the highest-ranked ecological journal (“Trends in Ecology and Evolution”), which reflects the novelty and potential of TRACE.
TRACE was then used and tested in several CREAM projects. As a result, the format of TRACE was revised by basing it on a new concept, developed in CREAM: model “evaludation”, which is a merger of model evaluation and validation. Accordingly, TRACE documents provide supporting information that a model was thoughtfully designed, correctly implemented, thoroughly tested, well understood, and appropriately used for its intended purpose. The revised TRACE was published together with a new, detailed template for TRACE documents, and with three example TRACE documents.
TRACE was met with great interest, in particular by regulatory authorities, who need protocols to assess whether or not a model is suitable for being taken into account in risk assessments. A working group of the panel for Plant Protection Products and their Residues at EFSA (European Food Safety Authority) developed and published guidance for Good Modelling Practice; several members of the CREAM consortium were part of this working group, and the final guidance is to a large degree based on the ideas underlying TRACE.
With TRACE, CREAM laid the foundation for developing and establishing Good Modelling Practice, which will considerably facilitate the development of models which are not only robust and predictive, but also accepted as decision support tools by stakeholders.
In 11 CREAM projects, experiments were performed, but the main focus of most of the 23 CREAM projects was on developing models. One postdoc project (Risk) used methods from social science to explore the attitudes of different stakeholders with regard to the potential role of ecological models in chemical risk assessment.
The CREAM models cover a wide range of model types and questions. Eight projects used more than one model type, which was one of the declared aims of CREAM because different model types, for example matrix and individual-based models, have different potentials and limitations. Most models addressed population dynamics, but in seven projects the effects of toxicants at the level of individuals was explored (TK/TD models); such models are important for improving the representation of exposure and effects at the individual level, and for being used as generic submodels in population models.
As for the questions addressed, in virtually all projects (19) the effect of chemicals at the population level was explored. The link between exposure to a chemical and its effects were studied in more detail in 10 projects; seven projects tried to predict population recovery after pulsed exposures. Indirect effects, via ecological interactions, were included in three projects, and bioaccumulation of toxicants in two projects. The organisms studied cover aquatic and terrestrial invertebrates, including soil organisms, and vertebrates, including fish, mammals, and birds.
- Experiments that reveal the effects of ecological factors for individual performance and thus susceptibility to chemicals (projects Scales-2 and DEB-1).
- Establishment of the cultivation of new test species (Copepod).
- Identification of an ecologically meaningful test species for fish and development of a model for this species (Fish-2).
- Experiments and individual-level models that will help reduce animal testing using fish (Fish-1).
- Predicting population-level effects from individual-level data (Daphnia-2).
- A model that predicts the uptake of heavy metals by bats for the entire UK (Mammals-2).
- A model that links, for water courses at a regional scale, effect and exposure models and thereby allows linking risk assessment to landscape structure and land use (Scales-4).
CREAM fellows obtained training courses in ecological modelling, ecological risk assessment, computational and statistical tools, and complementary skills for scientists. CREAM fellows thus obtained a unique training, which after finishing their projects will considerably increase their chances to pursue successful careers in the field of model-based ecological risk assessment in academia, industry, and regulatory authorities.
Potential final impact and use of CREAM’s output
Pollution and increasing pesticide load in the environment may pose considerable risks not only to humans but also to ecosystems and the services they provide for human well-being, but these risks are hard to assess and hard to quantitatively relate to the economic benefits in terms of, e.g., plant protection and food production. Current risk assessment schemes have been largely successful in Europe and North America to prevent major incidents and reduce pollutions and pesticide loads to the environment, but there is still considerable concern that the risks for biodiversity and ecosystem services are not appropriately captured, or are underestimated or overestimated considering current changes in land use and climate. Especially, the ecological relevance of current risk assessment methods is being questioned.
Mechanistic effect modelling is the most promising approach to make risk assessments more ecologically relevant. However before CREAM, stakeholders were sceptical and models were not used or not accepted. CREAM initiated a change of this situation. The scientific and regulatory community involved with risk assessment of chemicals is now aware of the potential of mechanistic effects models; they learned how they are developed and tested and how they can be evaluated and validated. They have seen, in more than 60 peer-reviewed publications and more than 60 conference presentations, a large number of case studies, focussing on a wide range of organisms, systems, and questions; they are aware of TRACE and the resulting guidance for Good Modelling Practice released by EFSA. Industry, consulting firms, and regulatory authorities started hiring modellers, also from the CREAM fellows, and are developing or evaluating models for risk assessment. Regulators became less sceptical, and have already begun to take models into account in a weight-of-evidence approach.
Thus, CREAM made it possible for mechanistic effect models to be developed and used in future regulatory risk assessment of chemicals, in particular pesticides. It did so by successfully pursuing its three main objectives, developing guidance for Good Modelling Practice, developing example models in a suite of case studies, and training more than 20 early stage researchers in mechanistic effect modelling. Moreover, many of these models can also be used to compare different use patterns of the chemicals and compare the effects to the alternatives which would be used instead of the chemical in questions.
CREAM will have long-lasting effects, also via the network of 50-80 researchers that emerged over the four years of CREAM, and via the cohort of CREAM fellows that were trained in rigorous model development and testing and will disseminate this culture of rigorous modelling for decision support to their countries and future enterprises.