The Canadian Earth System Model version 5 (CanESM5) is a global
model developed to simulate historical climate change and variability, to
make centennial-scale projections of future climate, and to produce
initialized seasonal and decadal predictions. This paper describes the model
components and their coupling, as well as various aspects of model
development, including tuning, optimization, and a reproducibility strategy.
We also document the stability of the model using a long control simulation,
quantify the model's ability to reproduce large-scale features of the
historical climate, and evaluate the response of the model to external
forcing. CanESM5 is comprised of three-dimensional atmosphere (T63 spectral
resolution equivalent roughly to 2.8
A multitude of evidence shows that human influence is driving accelerating changes in the climate system, which are unprecedented in millennia (IPCC, 2013). As the impacts of climate change are increasingly being felt, so is the urgency to take action based on reliable scientific information (UNFCCC, 2015). To this end, the Canadian Centre for Climate Modelling and Analysis (CCCma) is engaged in an ongoing effort to improve modelling of the global Earth system, with the aim of enhancing our understanding of climate system function, variability, and historical changes, and making improved quantitative predictions and projections of future climate. The global coupled model, the Canadian Earth System Model (CanESM), forms the basis of the CCCma modelling system and shares components with the Canadian Regional Climate Model (CanRCM) for finer-scale modelling of the atmosphere (Scinocca et al., 2016), the Canadian Middle Atmosphere Model (CMAM) with atmospheric chemistry (Scinocca et al., 2008), and the Canadian Seasonal to Interseasonal Prediction System which is used for seasonal prediction and decadal forecasts (CanSIPS, Merryfield et al., 2013).
CanESM5 is the current version of CCCma's global model and has a pedigree extending back 40 years to the introduction of the first atmospheric general circulation model (GCM) developed at CCCma's predecessor, the Canadian Climate Centre (Boer and McFarlane, 1979; Boer et al., 1984; McFarlane et al., 1992). Successive versions of the model introduced a dynamic three-dimensional ocean in CGCM1 (Flato et al., 2000; Boer et al., 2000a, b), and later an interactive carbon cycle was included to form CanESM1 (Arora et al., 2009; Christian et al., 2010). The last major iteration of the model, CanESM2 (Arora et al., 2011), was used in the Coupled Model Intercomparison Project phase 5 (CMIP5) and continues to be employed for novel science applications such as generating large initial condition ensembles for detection and attribution (e.g. Kirchmeier-Young et al., 2017; Swart et al., 2018).
As detailed below, CanESM5 represents a major update to CanESM2. The leap
from version 2 to version 5 was a one-off correction made to reconcile our
internal model version labelling with the version label released to the
public. The update includes incremental improvements to the atmosphere, land
surface, and terrestrial ecosystem models. The major changes relative to
CanESM2 are the implementation of completely new models for the ocean,
sea ice, and marine ecosystems, and a new coupler. Model developers have a
choice in distributing increasing, but finite, computational resources
between improvements in model resolution, model complexity, and model
throughput (i.e. number of years simulated). The resolution of CanESM5 (T63
or
The aim of this paper is to provide a comprehensive reference that documents CanESM5. In the sections below, each of the model components is briefly described, and we also explain the approach used to develop, tune, and numerically optimize the model. Following that, we document the stability of the model in a long pre-industrial control simulation, and the model's ability to reproduce large-scale features of the climate system. Finally, we investigate the sensitivity of the model to external forcings.
In CanESM5, the atmosphere is represented by the Canadian Atmosphere Model (CanAM5), which incorporates the Canadian Land Surface Scheme (CLASS) and the Canadian Terrestrial Ecosystem Model (CTEM). The ocean is represented by a CCCma-customized version of the Nucleus for European Modelling of the Ocean model (NEMO), with ocean biogeochemistry represented by either the Canadian Model of Ocean Carbon (CMOC) in the standard model version labelled as CanESM5, or the Canadian Ocean Ecosystem model (CanOE) in versions labelled CanESM5-CanOE. The atmosphere and ocean components are coupled by means of the Canadian Coupler (CanCPL). Each of these components of CanESM5 are described further below.
Version 5 of the Canadian Atmospheric Model (CanAM5) employs a spectral
dynamical core with a hybrid sigma-pressure coordinate in the vertical. The
package of physical parameterizations used by CanAM5 is based on an updated
version of its predecessor, CanAM4 (von Salzen et al., 2013). The physics
package includes a prognostic cloud microphysics scheme governing water
vapour, cloud liquid water, and cloud ice; a statistical layer-cloud scheme;
and independent cloud-base mass-flux schemes for both deep and shallow
convection. Aerosols are parameterized using a prognostic scheme for bulk
concentrations of natural and anthropogenic aerosols, including sulfate,
black and organic carbon, sea salt, and mineral dust; parameterizations for
emissions, transport, gas-phase and aqueous-phase chemistry; and dry and wet
deposition account for interactions with simulated meteorology. CanAM5
employs a triangular truncation at total wavenumber 63 (T63) corresponding
to an approximate isotropic resolution of 2.8
Updates to the package of physical parameterizations in CanAM5 over those in CanAM4 are as follows. While the radiative transfer solution in CanAM5 is similar to that in CanAM4, the representation of optical properties was improved through changes to the parameterization of albedos for bare soil, snow, and ocean whitecaps; cloud optics for ice clouds and polluted liquid clouds; improved aerosol optical properties and absorption by the water vapour continuum at solar wavelengths. For aerosols, the parameterization for emissions of mineral dust and dimethyl sulfide (DMS) was improved, while the bulk stratiform cloud microphysical scheme was modified to include a parameterization of the second indirect effect.
Parameterizations of surface processes were improved through an upgrade of the Canadian Land Surface Scheme (CLASS) from version 2.7 to 3.6.2 as well as the inclusion of a parameterization for subgrid lakes. CanESM5 represents the first coupled model produced by the CCCma in which the atmosphere and ocean do not employ coincident horizontal computational grids. As a consequence, CanAM5 was modified to support a fractional land mask, by generalizing its underlying surface to support grid-box fractional tiles of land and water. This tiling technology was extended to include land surface components of ocean, sea ice, and subgrid-scale lakes. In this way, appropriate fluxes can be provided to each component. A full description CanAM5 and its relation to CanAM4 will be provided in a companion paper in this special issue (Cole et al., 2019).
The CLASS-CTEM modelling framework consists of the Canadian Land Surface
Scheme (CLASS) and the Canadian Terrestrial Ecosystem Model (CTEM) which
together form the land component of CanESM5. CLASS and CTEM simulate the
physical and biogeochemical land surface processes, respectively, and
together they calculate fluxes of energy, water,
CLASS is described in detail in Verseghy (1991, 2000) and Verseghy et al. (1993) and version 3.6.2 is used in CanESM5. It prognostically calculates the temperature for its soil layers, their liquid and frozen moisture contents, temperature of a single vegetation canopy layer if it is present as dictated by the specified land cover, and the snow water equivalent and temperature of a single snow layer if it is present. Three permeable soil layers are used with default thicknesses of 0.1, 0.25, and 3.75 m. The depth to bedrock is specified on the basis of the global dataset of Zobler (1986), which reduces the thicknesses of the permeable soil layers. CLASS performs energy and water balance calculations and all physical land surface processes for four plant functional types (PFTs) (needleleaf trees, broadleaf trees, crops, and grasses) and operates at the same subdaily time step as the rest of the atmospheric component.
CTEM models photosynthesis, autotrophic respiration from its three living
vegetation components (leaves, stem, and roots), and heterotrophic respiration
fluxes from its two dead carbon components (litter and soil carbon) and is
described in detail in Arora (2003) and Arora and Boer (2003, 2005).
CTEM's photosynthesis module operates within CLASS, at the same
time step as rest of the atmospheric component. CTEM provides CLASS with
dynamically simulated structural attributes of vegetation including leaf
area index (LAI), vegetation height, rooting depth and distribution, and
aboveground canopy mass, which change in response to changes in climate and
atmospheric
While the modelled structural vegetation attributes respond to changes in
climate and atmospheric
Both CanESM5 and its predecessor CanESM2 do not include nutrient
limitation of photosynthesis on land since the terrestrial nitrogen cycle is
not represented. However, both models include a representation of
terrestrial photosynthesis downregulation based on Arora et al. (2009), who
used results from plants grown in ambient and elevated
The calculation of wetland extent and methane emissions from wetlands is
described in detail in Arora et al. (2018). In brief, dynamic wetland extent
is based on the “flat” fraction in each grid cell with slopes less than
0.2 %. As the liquid soil moisture in the top soil layer increases above a
specified threshold, the wetland fraction increases linearly up to a maximum
value, equal to the flat fraction in a grid cell. The simulated
Surface runoff and baseflow simulated by CLASS are routed through river networks. Major river basins are discretized at the resolution of the model and river routing is performed at the model resolution using the variable velocity river-routing scheme presented in Arora and Boer (1999). The delay in routing is caused by the time taken by runoff to travel over land in an assumed rectangular river channel and a groundwater component to which baseflow contributes. Streamflow (i.e. the routed runoff) contributes freshwater to the ocean grid cell where the land fraction of a CanAM grid cell first drops below 0.5 along the river network as the river approaches the ocean.
In CanESM5, glacier coverage is specified and static. Grid cells are
specified as glaciers if the fraction of the grid cell covered by ice exceeds
40 %, based on the GLC2000 dataset (Bartholomé and Belward, 2005). The
combination of this threshold and the model resolution results in glacier-covered
cells predominantly representing the Antarctic and Greenland ice
sheets, with a few glacier cells in the Himalayas, northern Canada, and
Alaska. Snow can accumulate on glaciers, and any additional snow above the
threshold of 100 kg m
The ocean component is based on NEMO version 3.4.1 (Madec and the NEMO team, 2012). It
is configured on the tripolar ORCA1 C grid with 45
Two modifications have been introduced to NEMO's mesoscale and small-scale mixing physics. The first modification is motivated by the observational evidence suggesting that away from the tropics the eddy scale decreases less rapidly than does the Rossby radius (e.g. Chelton et al., 2011). This is taken into consideration in the formulation for the eddy mixing length scale, which is used to compute the mesoscale eddy transfer coefficient for the Gent and McWilliams (1990) scheme (for details, see Saenko et al., 2018). The second modification is motivated by the observationally based estimates suggesting that a fraction of the mesoscale eddy energy could get scattered into high-wavenumber internal waves, the breaking of which results in enhanced diapycnal mixing (e.g. Marshall and Naveira Garabato, 2008; Sheen et al., 2014). A simple way to represent this process in an ocean general circulation model was proposed in Saenko et al. (2012). Here, we employ an updated version of their scheme which accounts better for the eddy-induced diapycnal mixing observed in the deep Southern Ocean (e.g. Sheen et al., 2014).
CanESM5 uses the LIM2 sea-ice model (Fichefet and Morales Maqueda, 1997; Bouillon et al., 2009), which is run within the NEMO framework. Some details regarding the calculation of surface temperature over sea ice are described in the coupling section below.
Two different ocean biogeochemical models, of differing complexity and expense, were developed in the NEMO framework: CMOC and CanOE. Two coupled models versions will be submitted to CMIP6. The version labelled as CanESM5 uses CMOC and was used to run all the experiments that CCCma has committed to. The version labelled CanESM5-CanOE, described in another paper in this special issue (Christian et al., 2019), is identical to CanESM5, except that CMOC was replaced with CanOE, and this version has been used to run a subset of the CMIP6 experiments, including DECK and historical (see Sect. 3.4). Both biogeochemical models simulate ocean carbon chemistry and abiotic chemical processes such as oxygen solubility identically, in accordance with the Ocean Model Intercomparison Project Biogeochemistry (OMIP-BGC) protocol (Orr et al., 2017).
The Canadian Model of Ocean Carbon was developed for earlier versions of CanESM (Zahariev et al., 2008; Christian et al., 2010; Arora et al., 2011) and includes carbon chemistry and biology. The biological component is a simple nutrient–phytoplankton–zooplankton–detritus (NPZD) model, with fixed Redfield stoichiometry, and simple parameterizations of iron limitation, nitrogen fixation, and export flux of calcium carbonate. CMOC was migrated into the NEMO modelling system, and the following important modifications were made: (i) oxygen was added as a passive tracer with no feedback on biology; (ii) carbon chemistry routines were updated to conform to the OMIP-BGC protocol (Orr et al., 2017); (iii) additional passive tracers requested by OMIP were added, including natural and abiotic dissolved inorganic carbon (DIC) as well as the inert tracers (CFC11, CFC12, and SF6).
The Canadian Ocean Ecosystem Model (CanOE) is a new ocean biology model with
a greater degree of complexity than CMOC and explicitly represents some
processes that were highly parameterized in CMOC. CanOE has two size classes
for each of phytoplankton, zooplankton, and detritus, with variable elemental
(C
CanCPL is a new coupler developed to facilitate communication between CanAM and CanNEMO. CanCPL depends on Earth System Modeling Framework (ESMF) library routines for regridding, time advancement, and other miscellaneous infrastructure (Theurich et al., 2016; Collins et al., 2005; Hill et al., 2004). It was designed for the multiple programme multiple data (MPMD) execution mode, with communication between the model components and the coupler via the Message Passing Interface (MPI).
The fields passed between the model components are summarized in Tables A1–A4.
In general, CanNEMO passes instantaneous prognostic fields, which are
remapped by CanCPL and given to CanAM as lower boundary conditions. These
prognostic fields (sea-surface temperature, sea-ice concentration, and mass
of sea ice and snow) are held constant in CanAM over the course of the
coupling cycle. After integrating forward for a coupling cycle, CanAM passes
back fluxes, averaged over the coupling interval, which are remapped in
CanCPL and passed on to NEMO as surface boundary conditions. An exception is
the ocean surface
All regridding in CanCPL is done using the ESMF first-order conservative
regridding option (ESMF, 2018), ensuring that global integrals remain
constant for all quantities passed between component models (but see an
exception below). The remapping weights
Within the NEMO coupling interface, the “conservative” coupling option is employed. This option dictates that net fluxes are passed over the combined ocean–ice cell, and the fluxes over only the ice-covered fraction of the cell are also supplied, in principle allowing net conservation, even if the distribution of ice has changed given the unavoidable one coupling cycle lag encountered in parallel coupling mode. It was verified that the net heat fluxes passed from CanAM were identical to the net fluxes received by NEMO, to the level of machine precision. Conservation in the coupled model piControl run is discussed further in Sect. 4.
Sea-ice thermodynamics are computed in the LIM2 ice model, based on the
surface fluxes received from CanAM and the basal heat flux from the NEMO
liquid ocean. LIM2 provides the sea-ice concentration, snow and ice
thickness to CanAM, via the coupler. The surface flux calculation in CanAM5
requires the ground temperature at the snow–sea-ice interface, GT
After a significant number of CMIP6 production simulations were complete, it was determined that while conservative remapping was desirable for heat and water fluxes, it introduced issues in the wind-stress field passed from CanAM to CanNEMO. Specifically, since CanAM is nominally 3 times coarser than CanNEMO, conservative remapping resulted in constant wind-stress fields over several NEMO grid cells, followed by an abrupt change at the edge of the next CanAM cell. This blockiness in the wind stress results in a non-smooth first derivative, and the resulting peaked wind-stress curl results in unphysical features in, for example, the ocean vertical velocities. Changing regridding of only wind stresses to the more typical bilinear interpolation, instead of conservative remapping, largely alleviates this issue. Sensitivity tests indicate no major impact on gross climate change characteristics such as transient climate response or equilibrium climate sensitivity, or on general features of the surface climate. However, there is an impact on local ocean dynamics, which led to the decision to submit a “perturbed” physics member to CMIP6. Hence, simulations submitted to CMIP6 labelled as perturbed physics member 1 (“p1”) use conservative remapping for wind stress, while those labelled as “p2” use bilinear regridding (see Sect. 3.4). A comparison between p1 and p2 runs is provided in Appendix E.
CanESM5 represents radiative forcing from individual greenhouse gases
(GHGs). Aside from
Each of the CanESM5 component models (CanAM5, CLASS-CTEM, and CanNEMO) were initially developed independently, driven by observations in stand-alone configurations – CanAM5 in present-day (2003–2008) Atmospheric Model Intercomparison Project (AMIP) mode and CanNEMO in pre-industrial (PI) OMIP-like mode using CORE bulk formulae. In these configurations, free parameters were initially adjusted to reduce climatological biases assessed via a range of diagnostics. Further details of the CanAM5 tuning may be found in Cole et al. (2019). The component models were then brought together in a pre-industrial configuration (i.e. the piControl experiment), which was evaluated based on an array of diagnostics. Several thousand years of coupled simulation were run during the finalization of the model, and an approach was taken whereby AMIP simulations would be used to derive parameter adjustments in CanAM, which would then be applied to the coupled model.
Initial present-day configurations of CanAM5 that were tuned to give roughly
the observed top-of-atmosphere net radiative forcing (top-of-atmosphere forcing
This initial coupled-model cold bias was rectified by adjusting free
parameters in CanAM, CLASS, and LIM2 in order to achieve a piControl
simulation with a global-mean screen temperature of around 13.7
The consequence of the adjustments in CanAM5 was an increase in the present-day TOA forcing in AMIP mode from
The final adjustment was to the carbon uptake over land so as to better
match the observed value over the historical period, and achieved via the
parameter which controls the strength of the
CanESM5 is the first version of the model to be publicly released, and this code sharing has been facilitated by the adoption of a new version-control-based strategy for code management. Additional goals of this new system are to adopt industry standard software development practices, to improve development efficiency, and to make all CanESM5 CMIP6 simulations fully repeatable.
To maintain modularity, the code is organized such that each model component
has a dedicated git repository for the version control of its source code
(Table 1). A dedicated super-repository tracks each of the components as git
submodules. In this way, the super-repo. keeps track of which specific versions of each component combine together
to form a functional version of CanESM. A commit of the CanESM super-repo.,
which is representable by an eight-character truncated SHA1 checksum, hence
uniquely defines a version of the full CanESM source code. The model
development process follows an industry standard workflow (Table B1). New
model features are merged onto the
A dedicated ecosystem of software is used to configure, compile, run, and analyse CanESM simulations on Environment and Climate Change Canada (ECCC)'s high-performance computer (HPC) (Table B2). Several measures are taken to ensure modularity and repeatability. The source code for each run is recursively cloned from gitlab and is fully self contained. A strict checking routine ensures that any code changes are committed to the version control system and any run-specific configuration changes are captured in a dedicated configuration repository. A database records the SHA1 checksums of the particular model version and configuration used for every run, and these are included in CMIP6 NetCDF output for traceability. Input files for model initialization and forcing are also tracked for reproducibility (Table B1).
Code structure and repositories.
Schematic of CanESM5 optimization. See Sect. 3.3 and Appendix C for details.
Our strategy of version control, run isolation, strict checking, and logging ensures that simulations can be repeated in the future, and the same climate will be obtained (bit-identical reproducibility is a further step and is dependant on machine architecture and compilers). The implementation of a clear branching workflow, and the uptake of modern tools such as issue trackers, and the gitlab online code-hosting application, has improved both collaboration and management of the code. This new system also led to large, unexpected improvements in model performance for two major reasons. The first was democratization of the code – via the promotion of group ownership of the code. The second was the freedom to experiment across the full code base ensured by our isolated run setup (Table B2), which was not possible under the previous system of using a single installed library of code shared across many runs. The performance gains achieved are described in the following section.
The ECCC HPC system consists of the following components: a “backend” Cray XC40, with two 18-core Broadwell CPUs per node (for 36 cores per node), and roughly 800 nodes in total, connected to a multi-PB lustre file system used as scratch space. This machine is networked to a “frontend” Cray CS5000, with several PB of attached spinning disk. This whole compute arrangement is replicated in a separate hall for redundancy, effectively doubling the available resources. Finally, a large tape-storage system (HPNLS) is available for archiving model results.
The initial implementation of a CanESM5 precursor on this new HPC occurred around 1 November 2017. The original workflow roughly followed that used for CanESM2 CMIP5 simulations. All CanESM5 components (atmosphere CanAM, coupler CanCPL, and ocean CanNEMO) were originally running at 64-bit precision. The atmospheric component CanAM was running on two 36-core compute nodes, the coupler was running on a separate node, and the ocean component was running on three nodes, resulting in six nodes in total. The initial throughput on the system, without queue time, was around 4.6 years of simulation per wall-clock day (ypd), or alternatively 0.02 simulation years per core day, when normalizing by the number of cores used.
In parallel to the physical model development, significant effort was made
to improve the model throughput and eliminate a number of inefficiencies in
the older CMIP5 workflow (Fig. 1). The largest effort was devoted to
improving the efficiency of CanAM5, since this was identified as the major
bottleneck. A brief summary of the improvements is given in Table C1 and
Fig. 1. The most substantial and rewarding change was in converting the
64-bit CanAM component to 32-bit numerics. Since the remaining two
components, CanCPL and CanNEMO, are still running at the 64-bit precision,
the communication between CanAM and CanCPL required the promotion of a
number of variables from 32-bit precision to 64-bit and back. The 32-bit
CanAM implementation required a number of modifications to maintain the
numerical stability of the code. Calculations in some subroutines, most
notably in the radiation code, were promoted to the 64-bit accuracy.
Conservation of some tracers, in particular
In the final setup, the CanAM/CanCPL components are running on three shared compute nodes, and the ocean component CanNEMO is running on five nodes, resulting in eight nodes overall. The combined effect of the improvements listed in Table C1 resulted in more than tripling the original throughput to about 16 ypd (Fig. 1). Despite the increase in the total node count from six to eight, the efficiency of the model also improved roughly 3-fold, from 0.02 simulation years per core day of compute to about 0.06 years per core day. This final model configuration can complete a realization of the 165-year CMIP6 historical experiment in just over 10 d, compared to about 36 d, had no optimization been undertaken. At the time of writing, over 50 000 years of CMIP6 related simulation have been conducted with CanESM5, consuming about 1 million core days of compute time, resulting in about 8 PB of data archived to tape and over 100 TB of data publicly served on the Earth System Grid Federation (ESGF).
This section describes the major experiments and model variants of CanESM5 that are being conducted for the Coupled Model Intercomparison phase 6 (CMIP6), the first major science application of the model. Figure 2 shows the global-mean surface temperature for several of the key CMIP6 experiments, which can be used to infer important properties of the model, as discussed further in Sects. 4–6. Table 2 lists the variants of CanESM5 which are being submitted to CMIP6. These include the “p1” and “p2” perturbed physics members of CanESM5 (see Sect. 2.5) and a version of the model with a different ocean biogeochemistry model, CanESM5-CanOE.
Global average screen temperature in CanESM5 for the CMIP6 DECK experiments, as well as the historical and tier 1 Shared Socioeconomic Pathway (SSP) experiments (SSP5-85, SSP3-70, SSP2-45, and SSP1-26). Thick lines are the 11-year running means; thin lines are annual means.
Table D1 lists the 20 CMIP6-endorsed MIPs in which CanESM5 is participating
and which model variants are being run for each MIP. The volume of
simulation continues to grow and will likely exceed 60 000 years. This is
significantly more than the
Individual historical realizations (ensemble members) of CanESM5 were generated by launching historical runs at 50-year intervals off the piControl simulation. This is the same as the approach used to generate the five realizations of CanESM2, which were submitted to CMIP5. The 50-year separation was chosen to allow for differences in multi-decadal ocean variability between realizations. Below, we discuss the properties of the model, including illustrations of the internal variability generated spread across the historical ensemble. All results below are based on the CanESM5 p1 model variant.
Model variants.
The characteristics and stability of the CanESM5 pre-industrial control climate are evaluated using 1000 years of simulation from the CMIP6 piControl experiment, conducted under constant specified greenhouse gas concentrations and forcings for the year 1850 (Eyring et al., 2016). Ideally, a climate model and all its subcomponents would exhibit perfect conservation of tracer mass (e.g. water, carbon), energy, and momentum, and would be run for long enough to achieve equilibrium. In this case, we would expect to see, on a long-term average, zero net fluxes of heat, freshwater, and carbon at the interface between the atmosphere, ocean, and land surface, zero top-of-atmosphere net radiation, and constant long-term average temperatures or tracer mass within each component. In reality, however, models are not perfectly conservative due to the limitations of numerical representation (i.e. machine precision) as well as possible design flaws or bugs in the code, and models are generally not run to perfect equilibrium due to computational constraints. Despite imperfect conservation or spinup, models can still usefully be applied, as long as the drifts in the control run are small relative to the signal of interest, in our case historical anthropogenic climate. Below, we consider conservation and drift of heat, water, and carbon in CanESM5 (Fig. 3).
Stability of the CanESM5 piControl run showing global-mean
The CanESM5 pre-industrial control shows a stable TOA
net heat flux of 0.1 Wm
At the liquid ocean surface, a small net freshwater flux results in a freshening trend and sea-level rise of about 24 cm over 1000 years (Fig. 3e, f). This rate of drift is more than 20 times smaller than the signal of anthropogenic sea-level rise. The LIM2 ice model appears to be the source of non-conservation: the net freshwater flux provided from CanAM is very close to zero, about 6 times smaller than that noted above (24 cm per 1000 years). Snow and ice volume are stable, not exhibiting any long-term drift, yet they are subject to considerable decadal- and centennial-scale variability (Fig. 3g, h).
Atmosphere–land carbon fluxes average to zero, and carbon pools within CTEM
are stable (Fig. 3i, k). The net ocean carbon flux is fairly close to zero
but remains slightly negative on average at
In this section, we use the CMIP6 historical simulations (Eyring et al., 2016) of CanESM5 “p1”, focusing on climatologies computed over 1981 to 2010, unless otherwise noted.
Summary statistics quantifying the ability of CanESM to reproduce
large-scale climate features. Shown are the correlation coefficient (
The ability of CanESM5 to reproduce observed large-scale spatial patterns in
the climate system is quantified using global summary statistics computed
over the 1981 to 2010 mean climate (Fig. 4). Shown are the correlation
coefficient between CanESM5 and observations (
Climatologies of surface air temperature over 1981 to 2010 in
CanESM5
Climatologies of precipitation over 1981 to 2010 in
CanESM5
For most variables, normalized RMSE has decreased in CanESM5 relative to CanESM2, indicating an improvement in the ability of the new model to reproduce observed climate patterns over its predecessor. The largest improvements were seen for ocean biogeochemistry variables, while small increases in error were seen for 3-D distribution of zonal winds (ua), sea-surface temperatures (tos), the March distribution of sea ice in the Southern Hemisphere (siconc), and surface latent heat flux (hfls). In the following sections, individual realms are examined, with a closer look at regional details and biases.
Climatologies of sea-level pressure over 1981 to 2010 in CanESM5
Cloud fraction in CanESM5
CanESM5 reproduces the large-scale climatological features of surface air temperatures (Fig. 5), precipitation (Fig. 6), and sea-level pressure (Fig. 7), though significant regional biases exist. CanESM5 is significantly colder than observed over sea-ice-covered regions (Fig. 5), noticeable in the Southern Ocean and in the region surrounding the Labrador Sea, which has extensive seasonal sea-ice cover in CanESM5 (see below). The Tibetan Plateau, the Sahara, and the broader North Atlantic Ocean are also cooler than observed. Warm biases exist over the eastern boundary current systems (Benguela, Humboldt, and California) and over the Amazon, eastern North America, much of Siberia, and broad regions of the tropical and subtropical oceans.
Zonal-mean temperature in CanESM5
Zonal-mean zonal winds
Precipitation biases vary in sign by region (Fig. 6). The largest biases are over the tropical Pacific and Atlantic oceans, between the Equator and extending into the southern subtropics. The overall pattern of precipitation biases is very similar to that seen across the CMIP5 (Flato et al., 2013) and CMIP3 (Lin, 2007) models. The largest land biases are excessive precipitation over much of sub-Saharan Africa, southeast Asia, Canada, and Peru–Chile. In contrast, western Asia, Europe, the North Atlantic, and the subtropical to high-latitude Southern Ocean have too little simulated precipitation. The large-scale pattern of sea-level pressure is captured by CanESM5 (Fig. 7). Biases relative to ERA5 are largest over the high elevations of Antarctica (Fig. 7), possibly reflecting differences in the extrapolation of surface pressure to sea level.
Relative to ISCCP-H (Young et al., 2018) version 1.00 (Rossow et al., 2016), the total cloud fraction in CanESM5 is overestimated along the Equator, particularly in the eastern tropical Pacific and Atlantic (Fig. 8). A too-large cloud fraction is also found over Antarctica and the Arctic. Underestimations of total cloud fraction occur over most other land areas, with the largest underestimations over Asia and the Himalayas.
Zonal-mean sections of air temperature for the DJF and JJA seasonal means
are shown in Fig. 9. In both seasons, CanESM5 is biased warm relative to
ERA5 near the tropopause, across the tropics and subtropics. Warm biases
also occur in the stratosphere, notably near 60
Time-mean values of
Zonal-mean values of
Zonal-mean zonal winds are compared to ERA5 in Fig. 10 for DJF and JJA. The westerly jets in CanESM5 are biased strong, particularly aloft, and in the winter hemisphere. Surface zonal winds in CanESM5 are only slightly stronger than observed and are significantly improved over those in CanESM2 (Fig. 11), which were too strong, particularly over the Southern Hemisphere westerly jet.
Figures 12 and 13 compare the geographical distribution and zonal averages of gross primary productivity (GPP) and latent and sensible heat fluxes over land with observation-based estimates from Jung et al. (2009). The zonal averages of GPP, and latent and sensible heat fluxes compare reasonably well with observation-based estimates, although the latent heat fluxes are somewhat higher especially in the Southern Hemisphere, as discussed below (Fig. 13). Figure 12 shows the biases in the simulated geographical distribution of these quantities. In the tropics, biases in GPP, and latent and sensible heat fluxes broadly correspond to biases in simulated precipitation compared to observation-based estimates (shown in Fig. 6).
Generally over the tropics, as would be intuitively expected, the signs of GPP and latent heat flux anomalies are the same since they are both affected by precipitation in the same way. Sensible heat flux is expected to behave in the opposite way compared to GPP and latent heat flux in response to precipitation biases. For example, simulated GPP and latent heat fluxes are lower, and sensible heat fluxes higher in the northeastern Amazonian region because simulated precipitation is biased low (Fig. 6). The opposite is true for almost the entire African region south of the Sahara and most of Australia. Here, simulated precipitation that is biased high, compared to observations, results in simulated GPP and latent heat flux that are higher and sensible heat flux that is lower than observation-based estimates. At higher latitudes, where GPP and latent heat flux are limited by temperature and available energy, the biases in precipitation do not translate directly into biases in GPP and latent heat flux as they do in the tropics.
Functional response of GPP to
The biases in simulated climate imply that simulated land surface quantities
will also be biased, which makes it difficult to assess if the underlying
model behaviour is realistic. This limitation can be alleviated to some
extent by looking at the functional relationships between a quantity and its
primary climate drivers. This technique works best when a land component is
driven offline with meteorological data. In a coupled model, as is the case
here, land–atmosphere feedbacks can potentially worsen a model's performance
by exaggerating an initial bias. For example, low model precipitation can be
further reduced due to feedbacks from reduced evapotranspiration, some of
which is recycled back into precipitation. Figure 14 shows the functional
relationships between GPP and temperature, and GPP and precipitation, for
both model- and observation-based estimates. The observation-based
temperature and precipitation data used in these plots are from CRU-JRA
reanalysis data that were used to drive terrestrial ecosystem models in the
TRENDY intercomparison for the 2018 Global Carbon Budget (Le Quéré
et al., 2018). Figure 14 shows that GPP increases both with increases in
precipitation (as would be normally expected) and temperature except at mean
annual values above 25
As mentioned earlier, dynamically simulated wetland extent and wetland methane emissions in CanESM5 are purely diagnostic. Figure G1 in Appendix G compares zonal distribution of simulated annual maximum wetland extent with observation-based estimates and shows the temporal evolution of annual maximum wetland extent and wetland methane emissions over the historical period.
CanESM5 reproduces the observed large-scale features of sea-surface temperature (SST), salinity (SSS), and height (SSH) (Fig. 15). The largest SST biases are the cold anomalies southeast of Greenland and in the Labrador Sea (Fig. 15b). These negative SST biases are associated with excessive sea-ice cover, described further below, and with the surface air temperature biases mentioned above. Positive SST biases are largest in the eastern boundary current upwelling systems, as for surface air temperatures.
Sea-surface
CanESM5 zonal-mean ocean
CanESM5 residual meridional overturning circulation in the
Atlantic
Sea-surface salinity biases are largest, and positive, around the Arctic
coastline, potentially indicating insufficient runoff in this region (Fig. 15d).
Negative annual-mean SSS biases occur in the Labrador Sea and are
also found in seas of the maritime continent and eastern tropical Atlantic.
SSH is shown as an anomaly from the (arbitrary) global
mean (Fig. 15e). Significant SSH biases are associated with the positions of
western boundary currents, noticeably for the Gulf Stream and Kuroshio
Current (Fig. 15f). CanESM5 has too-low SSH around Antarctica and too-high
SSH in the southern subtropics, with an excessive SSH gradient across the
Southern Ocean. This SSH gradient is associated with the geostrophic flow of
the Antarctic Circumpolar Current (ACC). The ACC in CanESM5 is vigorous with
190 Sv of transport through Drake Passage. This is larger than observational
estimates, which range up to
The CanESM5 interior distributions of potential temperature and salinity are
well correlated with observations (Fig. 4). In the zonal mean, potential
temperature biases are largest within the thermocline, which is warmer than
observed, particularly near 50
The meridional overturning circulation in the global ocean and the
Indo-Pacific as well as Arctic–Atlantic basins is shown in Fig. 17. The
global overturning streamfunction shows the expected major features: an
upper cell with clockwise rotation, connecting North Atlantic deep water
formation to low-latitude and Southern Ocean upwelling; a vigorous Deacon
cell in the Southern Ocean (as a result of plotting in
Closely connected to the MOC is the rate of northward heat transport by the
ocean (Fig. 18). CanESM5 produces the expected latitudinal distribution of
heat transport but, consistent with a weak MOC, slightly underestimates the
transport at 24
Northward heat transport in the global ocean in CanESM5 (in petawatts), with error bars showing the inverse estimate of Ganachaud and Wunsch (2003).
The seasonal cycles of sea-ice extent and volume are shown in Fig. 19. A major change from CanESM2 is seen in the sea-ice volume (Fig. 19b, d). CanESM2 simulated very thin ice and had about 40 % less Northern Hemisphere (NH) ice volume than in the Pan-Arctic Ice Ocean Modeling and Assimilation System (PIOMAS) reanalysis (Zhang and Rothrock, 2003; Schweiger et al., 2011). By contrast, CanESM5 has a larger NH ice volume than in CanESM2 (Fig. 19b). The amplitude and phase of the annual cycle in NH sea-ice volume in CanESM5 are similar to those in PIOMAS (Fig. 19b). In the Southern Hemisphere, CanESM5 also has a larger sea-ice volume and a seasonal cycle far more consistent with the global PIOMAS (GIOMAS) reanalysis product than CanESM2 (Fig. 19d).
Seasonal cycles of sea-ice extent
While CanESM2 significantly underestimated NH sea-ice extent relative to satellite-based observations, CanESM5 generally overestimates the extent (Fig. 19a). The NH sea-ice extent biases are largest in the winter and spring. During the March maximum, excessive sea ice is present in the Labrador Sea and east of Greenland (Fig. 20a). In the summer and fall, the net NH extent bias is far smaller (Fig. 20c) and results from a cancellation between lower-than-observed concentrations over the Arctic basin and larger-than-observed concentrations around northeastern Greenland. Southern Hemisphere sea-ice extent biases are largest during the early months of the year, and in March the positive concentration biases are focused in the northeastern Weddell and Ross seas (Fig. 20b). In September, SH concentration biases between CanESM5 and the satellite observations are focused around the northern ice edge and are of varying sign (Fig. 20d).
Sea-ice concentration biases between CanESM5 and NSIDC
climatologies for the months of March
The standard configuration of CanESM5 has a significantly improved
representation of the distribution of ocean biogeochemical tracers relative
to CanESM2, despite using the same biogeochemical model (CMOC). For the
three-dimensional distributions of DIC and
Zonal-mean sections of
In CanESM5, the zonal-mean DIC concentration simulated by CMOC is generally
lower than observed, by amounts reaching up to about 5 % (Fig. 21a, b).
One exception to this is in the SH subtropical thermocline, on the northern
flank of the Southern Ocean, which shows positive DIC biases between 250 and
1000 m. This area is also one of positive nitrate biases, whose magnitude is
close to 30 % (Fig. 21d). Elsewhere, zonal-mean
The zonal-mean
The atmosphere–ocean
The El Niño–Southern Oscillation (ENSO) is a key component of climate
variability on seasonal and interannual timescales. To evaluate CanESM5's
representation of ENSO, the NINO3.4 index (average monthly SST anomaly in
the region bounded by 5
The spectral peak in the historical ensemble members (Fig. 23c) occurs at around 3–5 years in general agreement with observations. Variability on decadal timescales has a large spread between ensemble members, likely due to differences in the strength of warming trends over the historical period. Higher-frequency variability at monthly to seasonal timescales is significantly lower than observed. The lower monthly variability can also be seen by examining month-by-month interannual variability of NINO3.4 (Fig. 23d). While January remains the month of peak variability, overall, the annual cycle of NINO3.4 variability is weaker in CanESM5. In observations, ENSO variability is at its minimum between April and June but in CanESM5 the minimum variability (depending on the ensemble member) tends to be between July and September.
Characteristics of ENSO
from and the HadISST observational product. Spatial maps in panels
The Northern Annular Mode (NAM) is computed as the first EOF of extended winter
(DJFM) sea-level pressure north of 20
First empirical orthogonal functions (EOFs) of sea-level pressure
north of 20
The Southern Annular Mode (SAM) is the dominant mode of climate variability in the
Southern Hemisphere, with significant influences on atmospheric circulation,
precipitation, and the Southern Ocean. We compute the SAM pattern as the
first EOF of sea-level pressure south of 20
The global-mean screen temperature change under the idealized CMIP6 DECK
“abrupt-4xCO2” and “1pctCO2” experiments is shown in Fig. 2. From these
simulations, three major benchmarks of the model's response to
Key sensitivity metrics: transient climate response (TCR), transient climate response to cumulative emissions (TCRE), and equilibrium climate sensitivity (ECS).
The transient climate response (TCR) of the model is given by the
temperature change in the 1pctCO2 experiment, averaged over the 20 years
centred on the year of
The transient climate response to cumulative emissions (TCRE) incorporates
the transient climate sensitivity together with the carbon sensitivity of
the system (Matthews et al., 2009). It is defined as the ratio of global-mean
surface warming to cumulative carbon emissions, over the 20 years
centred on
The equilibrium climate sensitivity (ECS) is defined as the amount of global-mean
surface warming resulting from a doubling of atmospheric
A detailed explanation of the reasons behind the increased ECS in CanESM2
over CanESM5 is beyond the scope of this paper. However, the effective
radiative forcing (Forster et al., 2016) in CanESM5 due to abrupt quadrupling
of
Surface temperature trends in CanESM5
Time series of
In this section, we briefly discuss CanESM5-simulated changes in surface air temperature, sea ice, and carbon cycle fluxes over the historical period. We choose these as major emblematic variables of climate change. Here, we make use of the CanESM2 50-member large initial condition ensemble (Kirchmeier-Young et al., 2017; Swart et al., 2018). The 50 realizations in this ensemble were branched in the year 1950 from the five CanESM2 realizations submitted to CMIP5 and were forced by CMIP5 historical (1950 to 2005) and Representative Concentration Pathway (RCP) 8.5 (2006 to 2100) forcing.
Global-mean surface temperature (GMST) changes in CanESM2 and CanESM5 are generally consistent with the observations over the period from 1850 to around the end of the 20th century (Fig. 25a). However, from 2000 to 2014, the increase in GMST is larger in the models than observed. Possible reasons for the divergence are (i) forcing errors in the CMIP5 and/or CMIP6 forcing datasets, (ii) natural internal variability, (iii) incorrect partitioning of heat across components of the climate system, or (iv) a higher climate sensitivity in the model than in the real world. The 25 realizations of CanESM5 (and 50 realization of CanESM2) provide a good estimate of the contribution of internal variability in the model. The observations fall outside the range of this variability, and hence this cannot account entirely for the divergence between the model and observations (assuming the model correctly captures the scale of internal variability). Trends computed from 1981 to 2014 show that the models are warming at roughly twice the observed rate over this period (Fig. 25b). The spread across the 25 realizations from CanESM5 and 50 realizations from CanESM2 does not encompass the observations, reinforcing the point above. CanESM5 warms more rapidly than CanESM2, on average, as would be expected from its higher ECS and TCR. There is, however, significant overlap across the distribution of warming rates across the CanESM5 and CanESM2 ensembles. Interestingly, the lower tail of the trend probability distribution functions aligns for the two models, but CanESM5 has a broader distribution and a larger tail of high warming realizations.
Annual
The pattern of surface warming in CanESM5 over the historical period is
shown in Fig. 26a. The canonical features of global warming are consistent
between the model and observations: greater warming over land than ocean
and Arctic amplified warming. The zonal-mean warming trends (Fig. 26c) show
that both CanESM2 and CanESM5 warmed more than the observations over most
latitudes. Divergence between simulated and observed warming rates is
largest in the high latitudes, notably over the Southern Ocean and north of
40
CanESM5 closely reproduces the observed reduction in Arctic September sea-ice extent (Fig. 27a). The trends from both the 50 CanESM2 ensemble members and the 25 CanESM5 ensemble members show a broad spread due to internal variability (Fig. 27c). The observed trends lie close to the centre of the model distribution of trends. Given that CanESM5 warms more rapidly than observed, the sea-ice sensitivity (rate of sea-ice decline normalized by the rate of warming) is likely too low (Rosenblum and Eisenman, 2017; Winton, 2011).
In the Southern Hemisphere, observed annual-mean Antarctic sea-ice extent showed a tendency to increase before dramatic declines in the past few years (Fig. 27b). Both CanESM5 and CanESM2 show consistent declines over the historical period, with CanESM2 matching the climatological extent more closely.
The simulated global atmosphere–ocean (
In Fig. 28a, the simulated global atmosphere–ocean
Figure 28e–f show the allowable diagnosed fossil fuel emissions
and their cumulative values for the 1850–2014 period. The cumulative
diagnosed fossil fuel emissions of
CanESM5 is the latest coupled model from the Canadian Centre for Climate Modelling and Analysis. Relative to its predecessor, CanESM2, the model has new ocean, sea-ice, and coupling components, and includes updates to the atmospheric and land surface. The model produces a stable pre-industrial control climate, and notwithstanding some significant biases, CanESM5 is able to reproduce many features of the historical climate. Objective global skill metrics show that CanESM5 improves the simulation of observed large-scale climate patterns, relative to CanESM2, for most variables surveyed. A notable feature of CanESM5 is its high equilibrium climate sensitivity of 5.6 K, an emergent property of the updated physics described above. This higher climate sensitivity appears to be driven by increased cloud and sea-ice albedo feedbacks in CanESM5. The first major science application of CanESM5 is for CMIP6, with over 50 000 years of CanESM5 simulation and more than 100 PB of data submitted to the publicly available CMIP6 archive. The model source code is also openly published for the first time. Going forward, CanESM5 will continue to be used for climate science applications in Canada.
The full CanESM5 source code is publicly available at
All CanESM5 simulations conducted for CMIP6, including those described in this paper, are publicly available via the Earth System Grid Federation (ESGF). All observational data used are publicly available. Data sources and citations are provided in Appendix F.
Schematic showing the ordering of exchanges between CanCPL and
CanAM and CanNEMO. Prognostic fields (
Fields received by CanAM from CanCPL. The representative area may be the full AGCM grid cell (land, ocean, and ice), “C”; open ocean, “O”, sea ice, “I”; or the combination. Fields may be instantaneous, “inst”, or averaged over the coupling cycle, “avg”.
Fields sent from CanNEMO to CanCPL. Descriptions are as in Table A1.
Fields received by CanNEMO from CanCPL. Descriptions are as in Table 1.
Fields sent from CanAM to CanCPL. Descriptions are as in Table 1.
Code management.
Process for running CanESM.
Description of optimization improvements to CanESM5. See Fig. 1 for a graphical representation.
List of MIPs and model variants of CanESM5 planned for submission to CMIP6.
Sections 2.5 and 3.4 described the technical differences between perturbed physics members p1 and p2, submitted to the CMIP6 archive. Here, we provide a preliminary analysis of the differences between the two model variants.
Figure E1 shows surface air temperature and precipitation averaged over 200 years of the piControl experiment for p1, p2, and the difference between them. Notable in the differences are the “cold” spots in Antarctica, which arise from a misspecified land fraction in p1 and were resolved in p2. Otherwise there are no significant differences.
Climatologies of surface air temperature
Figure E2 shows the ocean surface wind stress. The blockiness of the field in p1 is evident as a result of conservative remapping from CanAM. In p2, bilinear remapping was used and the field is smooth on the NEMO grid. The non-smooth nature of wind stress in p1 resulted in, for example, banding in vertical ocean velocities at 100 m depth, as also shown in Fig. E2d. This does not occur in p2.
The response to
Climatologies of surface ocean zonal wind stress
Maps of the perturbative response, computed as the mean over the 20 years
centred on
Global averages of
Perturbation of surface air temperature
Perturbations of surface ocean zonal wind stress
In Fig. 28, the diagnosed allowable anthropogenic fossil fuel emissions are
calculated via Eq. (F1):
In these historical simulations, the concentration of atmospheric
List of figures, CanESM5 CMIP6 variables, and observations used, and the time periods of analysis in the main text. See Table F3 for a definition of variable names. “n/a” indicates “not applicable”.
List of observational products used.
List of CMIP6 variable names used and their long names.
Comparison of simulated zonally summed annual maximum wetland
area with observation-based estimates based on the Global Lakes and Wetland
(GLWD; Lehner and Döll, 2004) and a new product that is formed by
merging remote-sensing-based observations of daily surface inundation from
the Surface Water Microwave Product Series (SWAMPS; Schroeder et al., 2015)
with the static inventory of wetland area from the GLWD as explained in
Poulter et al. (2017)
Figure G1a compares the zonally summed annual maximum wetland area
with observation-based estimates based on the Global Lakes and Wetland
(GLWD; Lehner and Döll, 2004) and the new product formed by merging
remote-sensing-based observations of daily surface inundation from the
Surface Water Microwave Product Series (SWAMPS; Schroeder et al., 2015) with
the static inventory of wetland area from the GLWD from Poulter et al. (2017). Maximum wetland fraction from the model (1995–2014) and
SWAMPS and GLWD (2000–2012) product is calculated as the maximum of 12
monthly mean values for the time period noted and the multiplied by area of the
grid cells to calculate zonally summed area. The model overall captures the
broad latitudinal distribution of wetlands with higher wetland area in the
tropics and at northern high latitudes. The model yields higher wetland area
in the tropics than both observation-based estimates due to higher wetland
area simulated in the Amazonian region. The Amazonian region is densely
forested and the SWAMPS product is unable to map wetlands beneath closed
forest canopies. Biases also likely exist in the GLWD dataset since parts
of the Amazonian region are fairly remote. The global annual maximum wetland
extent of 8.65 million km
Figure G1b shows the time evolution of simulated annual maximum
wetland extent and wetland methane emissions over the 1850–2014 period from
the historical simulation. The shaded range represents the standard
deviation over the 25 ensemble members of the historical simulation. While
the simulated wetland extent does not change significantly over time, the
methane emissions increase from about 150 Tg
NCS co-led CanESM5 development, contributed to CanNEMO and CMOC development and the data request, performed simulations, led the creation of the figures, and wrote most of the manuscript; JNSC contributed to development of CanAM5 and CanCPL and tuning of CanESM5, wrote the CanAM5 section, performed simulations, and contributed to the data request; VK contributed to the development of CanAM, notably optimization, contributed to the data request, and performed production simulations; ML contributed to the development of CanAM, CanCPL, and the data request; JS co-led CanESM5 development; NG contributed to CanESM5 development and tuning; JA contributed to CanCPL development and the data request, and led publication of data on the ESGF; VA contributed to the development of CLASS and CTEM; JC developed CanOE and contributed to CMOC development; SH produced many of the figures; YJ contributed to the data request and conversion; WL contributed to CanNEMO development and ran simulations; FM contributed to the CanESM5 software infrastructure; OS led ocean physics testing and provided a specific analysis that motivated the p2 variant; ChS contributed analysis of the land component; ClS contributed to CanESM5 software infrastructure; AS created Fig. 23, contributed to CanESM5 development, and performed simulations; MS contributed to the sea-ice and atmosphere data request; LS developed CanCPL; KVS led the development and tuning of atmospheric model parameterizations; DY contributed to ocean model development, and ocean and sea-ice diagnostics for CMIP6, and performed production simulations; BW processed forcing datasets for CanAM; all authors contributed to editing the manuscript.
The authors declare that they have no conflict of interest.
CanESM has been customized to run on the ECCC high-performance computer, and a significant fraction of the software infrastructure used to run the model is specific to the individual machines and architecture. While we publicly provide the code, we cannot provide any support for migrating the model to different machines or architectures.
We acknowledge Greg Flato and William Merryfield for helpful comments on a draft of the paper. CanESM5 was the cumulative result of work by many individuals, who we thank for their contributions. CanESM5 simulations were performed on ECCC's HPC, and CanESM5 data are served via the Earth System Grid Federation.
This paper was edited by Gerd A. Folberth and reviewed by three anonymous referees.