4. Ensembles

4.1. Ensemble reduction

This tutorial will explore ensemble reduction (also known as ensemble selection) using xscen. This will use pre-computed annual mean temperatures from xclim.testing.

[2]:
import pooch
import xarray as xr
from xclim.testing.utils import nimbus

import xscen as xs

downloader = pooch.HTTPDownloader(headers={"User-Agent": f"xscen-{xs.__version__}"})

datasets = {
    "ACCESS": "EnsembleStats/BCCAQv2+ANUSPLIN300_ACCESS1-0_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc",
    "BNU-ESM": "EnsembleStats/BCCAQv2+ANUSPLIN300_BNU-ESM_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc",
    "CCSM4-r1": "EnsembleStats/BCCAQv2+ANUSPLIN300_CCSM4_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc",
    "CCSM4-r2": "EnsembleStats/BCCAQv2+ANUSPLIN300_CCSM4_historical+rcp45_r2i1p1_1950-2100_tg_mean_YS.nc",
    "CNRM-CM5": "EnsembleStats/BCCAQv2+ANUSPLIN300_CNRM-CM5_historical+rcp45_r1i1p1_1970-2050_tg_mean_YS.nc",
}

for d in datasets:
    file = nimbus().fetch(datasets[d], downloader=downloader)
    ds = xr.open_dataset(file).isel(lon=slice(0, 4), lat=slice(0, 4))
    ds = xs.climatological_op(
        ds,
        op="mean",
        window=30,
        periods=[[1981, 2010], [2021, 2050]],
        horizons_as_dim=True,
    ).drop_vars("time")
    datasets[d] = xs.compute_deltas(ds, reference_horizon="1981-2010")
Downloading file 'EnsembleStats/BCCAQv2+ANUSPLIN300_ACCESS1-0_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc' from 'https://raw.githubusercontent.com/Ouranosinc/xclim-testdata/v2025.4.29/data/EnsembleStats/BCCAQv2+ANUSPLIN300_ACCESS1-0_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc' to '/home/docs/.cache/xclim-testdata/v2025.4.29'.
Downloading file 'EnsembleStats/BCCAQv2+ANUSPLIN300_BNU-ESM_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc' from 'https://raw.githubusercontent.com/Ouranosinc/xclim-testdata/v2025.4.29/data/EnsembleStats/BCCAQv2+ANUSPLIN300_BNU-ESM_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc' to '/home/docs/.cache/xclim-testdata/v2025.4.29'.
Downloading file 'EnsembleStats/BCCAQv2+ANUSPLIN300_CCSM4_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc' from 'https://raw.githubusercontent.com/Ouranosinc/xclim-testdata/v2025.4.29/data/EnsembleStats/BCCAQv2+ANUSPLIN300_CCSM4_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc' to '/home/docs/.cache/xclim-testdata/v2025.4.29'.
Downloading file 'EnsembleStats/BCCAQv2+ANUSPLIN300_CCSM4_historical+rcp45_r2i1p1_1950-2100_tg_mean_YS.nc' from 'https://raw.githubusercontent.com/Ouranosinc/xclim-testdata/v2025.4.29/data/EnsembleStats/BCCAQv2+ANUSPLIN300_CCSM4_historical+rcp45_r2i1p1_1950-2100_tg_mean_YS.nc' to '/home/docs/.cache/xclim-testdata/v2025.4.29'.
Downloading file 'EnsembleStats/BCCAQv2+ANUSPLIN300_CNRM-CM5_historical+rcp45_r1i1p1_1970-2050_tg_mean_YS.nc' from 'https://raw.githubusercontent.com/Ouranosinc/xclim-testdata/v2025.4.29/data/EnsembleStats/BCCAQv2+ANUSPLIN300_CNRM-CM5_historical+rcp45_r1i1p1_1970-2050_tg_mean_YS.nc' to '/home/docs/.cache/xclim-testdata/v2025.4.29'.
[3]:
datasets
[3]:
{'ACCESS': <xarray.Dataset> Size: 264B
 Dimensions:                            (lon: 4, lat: 4, horizon: 2)
 Coordinates:
   * lon                                (lon) float64 32B -74.96 ... -74.71
   * lat                                (lat) float64 32B 45.04 45.12 45.21 45.29
   * horizon                            (horizon) <U9 72B '1981-2010' '2021-2050'
 Data variables:
     tg_mean_clim_mean_delta_1981_2010  (horizon, lat, lon) float32 128B 0.0 ....
 Attributes: (12/36)
     institution:                    Canadian Centre for Climate Services (CCCS)
     contact:                        Canadian Centre for Climate Services
     Conventions:                    CF-1.4
     institute_id:                   CCCS
     domain:                         Canada
     creation_date:                  2017-10-24T11:58:08PDT
     ...                             ...
     title:                          Bias Correction/Constructed Analogue Quan...
     history:
     climateindex_package_id:        https: // github.com / Ouranosinc / xclim
     downscaling_institution:        Pacific Climate Impacts Consortium (PCIC)...
     downscaling_institute_id:       PCIC
     cat:processing_level:           deltas,
 'BNU-ESM': <xarray.Dataset> Size: 264B
 Dimensions:                            (lon: 4, lat: 4, horizon: 2)
 Coordinates:
   * lon                                (lon) float64 32B -74.96 ... -74.71
   * lat                                (lat) float64 32B 45.04 45.12 45.21 45.29
   * horizon                            (horizon) <U9 72B '1981-2010' '2021-2050'
 Data variables:
     tg_mean_clim_mean_delta_1981_2010  (horizon, lat, lon) float32 128B 0.0 ....
 Attributes: (12/36)
     institution:                    Canadian Centre for Climate Services (CCCS)
     contact:                        Canadian Centre for Climate Services
     Conventions:                    CF-1.4
     institute_id:                   CCCS
     domain:                         Canada
     creation_date:                  2017-11-02T12:54:16PDT
     ...                             ...
     title:                          Bias Correction/Constructed Analogue Quan...
     history:
     climateindex_package_id:        https: // github.com / Ouranosinc / xclim
     downscaling_institution:        Pacific Climate Impacts Consortium (PCIC)...
     downscaling_institute_id:       PCIC
     cat:processing_level:           deltas,
 'CCSM4-r1': <xarray.Dataset> Size: 264B
 Dimensions:                            (lon: 4, lat: 4, horizon: 2)
 Coordinates:
   * lon                                (lon) float64 32B -74.96 ... -74.71
   * lat                                (lat) float64 32B 45.04 45.12 45.21 45.29
   * horizon                            (horizon) <U9 72B '1981-2010' '2021-2050'
 Data variables:
     tg_mean_clim_mean_delta_1981_2010  (horizon, lat, lon) float32 128B 0.0 ....
 Attributes: (12/36)
     institution:                    Canadian Centre for Climate Services (CCCS)
     contact:                        Canadian Centre for Climate Services
     Conventions:                    CF-1.4
     institute_id:                   CCCS
     domain:                         Canada
     creation_date:                  2017-08-01T19:54:05PDT
     ...                             ...
     title:                          Bias Correction/Constructed Analogue Quan...
     history:
     climateindex_package_id:        https: // github.com / Ouranosinc / xclim
     downscaling_institution:        Pacific Climate Impacts Consortium (PCIC)...
     downscaling_institute_id:       PCIC
     cat:processing_level:           deltas,
 'CCSM4-r2': <xarray.Dataset> Size: 264B
 Dimensions:                            (lon: 4, lat: 4, horizon: 2)
 Coordinates:
   * lon                                (lon) float64 32B -74.96 ... -74.71
   * lat                                (lat) float64 32B 45.04 45.12 45.21 45.29
   * horizon                            (horizon) <U9 72B '1981-2010' '2021-2050'
 Data variables:
     tg_mean_clim_mean_delta_1981_2010  (horizon, lat, lon) float32 128B 0.0 ....
 Attributes: (12/36)
     institution:                    Canadian Centre for Climate Services (CCCS)
     contact:                        Canadian Centre for Climate Services
     Conventions:                    CF-1.4
     institute_id:                   CCCS
     domain:                         Canada
     creation_date:                  2017-12-05T02:07:22PST
     ...                             ...
     title:                          Bias Correction/Constructed Analogue Quan...
     history:
     climateindex_package_id:        https: // github.com / Ouranosinc / xclim
     downscaling_institution:        Pacific Climate Impacts Consortium (PCIC)...
     downscaling_institute_id:       PCIC
     cat:processing_level:           deltas,
 'CNRM-CM5': <xarray.Dataset> Size: 264B
 Dimensions:                            (lon: 4, lat: 4, horizon: 2)
 Coordinates:
   * lon                                (lon) float64 32B -74.96 ... -74.71
   * lat                                (lat) float64 32B 45.04 45.12 45.21 45.29
   * horizon                            (horizon) <U9 72B '1981-2010' '2021-2050'
 Data variables:
     tg_mean_clim_mean_delta_1981_2010  (horizon, lat, lon) float32 128B 0.0 ....
 Attributes: (12/37)
     institution:                    Canadian Centre for Climate Services (CCCS)
     contact:                        Canadian Centre for Climate Services
     Conventions:                    CF-1.4
     institute_id:                   CCCS
     domain:                         Canada
     creation_date:                  2017-05-19T13:54:07PDT
     ...                             ...
     project_id:                     CMIP5
     NCO:                            "4.6.0"
     climateindex_package_id:        https: // github.com / Ouranosinc / xclim
     downscaling_institution:        Pacific Climate Impacts Consortium (PCIC)...
     downscaling_institute_id:       PCIC
     cat:processing_level:           deltas}

4.1.1. Preparing the data

Ensemble reduction is built upon climate indicators that are relevant to represent the ensemble’s variability for a given application. In this case, we’ll use the mean temperature delta between 2021-2050 and 1981-2010, but monthly or seasonal indicators could also be required. The horizons_as_dim argument in climatological_op can help combine indicators of multiple frequencies into a single dataset. Alternatively, xscen.utils.unstack_dates can also accomplish the same thing if the climatological operations have already been computed.

The functions implemented in xclim.ensembles._reduce require a very specific 2-D DataArray of dimensions “realization” and “criteria”. The first solution is to first create an ensemble using xclim.ensembles.create_ensemble, then pass the result to xclim.ensembles.make_criteria. Alternatively, the datasets can be passed directly to xscen.ensembles.reduce_ensemble and the necessary preliminary steps will be accomplished automatically.

In this example, the number of criteria will corresponds to: indicators x horizons x longitude x latitude, but criteria that are purely NaN across all realizations will be removed.

Note that xs.spatial_mean could have been used prior to calling that function to remove the spatial dimensions.

4.1.2. Selecting a reduced ensemble

NOTE

Ensemble reduction in xscen is built upon xclim.ensembles. For more information on basic usage and available methods, please consult their documentation.

Ensemble reduction through xscen.reduce_ensemble consists in a simple call to xclim. The arguments are:

  • data, which is the aforementioned 2D DataArray, or the list/dict of datasets required to build it.

  • method is either kkz or kmeans. See the link above for further details on each technique.

  • horizons is used to instruct on which horizon(s) to build the data from, if data needs to be constructed.

  • create_kwargs, the arguments to pass to xclim.ensembles.create_ensemble if data needs to be constructed.

  • kwargs is a dictionary of arguments to send to the clustering method chosen.

[4]:
selected, clusters, fig_data = xs.reduce_ensemble(
    data=datasets, method="kmeans", horizons=["2021-2050"], max_clusters=3
)

The method always returns 3 outputs (selected, clusters, fig_data):

  • selected is a DataArray of dimension ‘realization’ listing the selected simulations.

  • clusters (kmeans only) groups every realization in their respective clusters in a python dictionary.

  • fig_data (kmeans only) can be used to call xclim.ensembles.plot_rsqprofile(fig_data)

[5]:
selected
[5]:
<xarray.DataArray 'realization' (realization: 3)> Size: 96B
array(['ACCESS', 'BNU-ESM', 'CNRM-CM5'], dtype='<U8')
Coordinates:
  * realization  (realization) <U8 96B 'ACCESS' 'BNU-ESM' 'CNRM-CM5'
Attributes:
    axis:     E
[6]:
# To see the clusters in more details
clusters
[6]:
{np.int32(0): <xarray.DataArray 'realization' (realization: 3)> Size: 96B
 array(['ACCESS', 'CCSM4-r1', 'CCSM4-r2'], dtype='<U8')
 Coordinates:
   * realization  (realization) <U8 96B 'ACCESS' 'CCSM4-r1' 'CCSM4-r2'
 Attributes:
     axis:     E,
 np.int32(1): <xarray.DataArray 'realization' (realization: 1)> Size: 32B
 array(['CNRM-CM5'], dtype='<U8')
 Coordinates:
   * realization  (realization) <U8 32B 'CNRM-CM5'
 Attributes:
     axis:     E,
 np.int32(2): <xarray.DataArray 'realization' (realization: 1)> Size: 32B
 array(['BNU-ESM'], dtype='<U8')
 Coordinates:
   * realization  (realization) <U8 32B 'BNU-ESM'
 Attributes:
     axis:     E}
[7]:
../_images/notebooks_4_ensembles_10_0.png

4.2. Ensemble partition

This tutorial will show how to use xscen to create the input for xclim partition functions.

[8]:
# Get catalog
from pathlib import Path

import xclim as xc

output_folder = Path().absolute() / "_data"
cat = xs.DataCatalog(str(output_folder / "tutorial-catalog.json"))

# create a dictionary of datasets wanted for the partition
input_dict = cat.search(variable="tas", member="r1i1p1f1").to_dataset_dict(
    xarray_open_kwargs={"engine": "h5netcdf"}
)
datasets = {}
for k, v in input_dict.items():
    ds = xc.atmos.tg_mean(v.tas)
    ds.attrs = v.attrs
    datasets[k] = ds

--> The keys in the returned dictionary of datasets are constructed as follows:
        'id.domain.processing_level.xrfreq'
100.00% [4/4 00:00<00:00]

From a dictionary of datasets, the function creates a dataset with new dimensions in partition_dim(["source", "experiment", "bias_adjust_project"], if they exist). In this toy example, we only have different experiments.

  • By default, it translates the xscen vocabulary (eg. experiment) to the xclim partition vocabulary (eg. scenario). It is possible to pass rename_dict to rename the dimensions with other names.

  • If the inputs are not on the same grid, they can be regridded through regrid_kw or subset to a point through subset_kw. The functions assumes that if there are different bias_adjust_project, they will be on different grids (with all source on the same grid). If there is one or less bias_adjust_project, the assumption is thatsource have different grids.

  • You can also compute indicators on the data if the input is daily. This can be especially useful when the daily input data is on different calendars.

[9]:
# build a single dataset
import xclim as xc

ds = xs.ensembles.build_partition_data(
    datasets,
    subset_kw=dict(name="mtl", method="gridpoint", lat=[45.5], lon=[-73.6]),
)
ds
[9]:
<xarray.Dataset> Size: 120B
Dimensions:   (time: 2, scenario: 4, model: 1)
Coordinates:
  * time      (time) datetime64[ns] 16B 2001-01-01 2002-01-01
  * scenario  (scenario) object 32B 'ssp126' 'ssp245' 'ssp370' 'ssp585'
  * model     (model) object 8B 'NCC_NorESM2-MM_r1i1p1f1'
Data variables:
    tg_mean   (model, scenario, time) float64 64B dask.array<chunksize=(1, 1, 2), meta=np.ndarray>
Attributes: (12/18)
    cat:mip_era:           CMIP6
    cat:domain:            mtl
    cat:variable:          tas
    cat:member:            r1i1p1f1
    cat:activity:          ScenarioMIP
    cat:date_start:        2001-01-01 00:00:00
    ...                    ...
    cat:institution:       NCC
    history:               [2026-11-19 21:48:31] gridpoint spatial subsetting...
    cat:xrfreq:            D
    cat:frequency:         day
    cat:source:            NorESM2-MM
    cat:id:                CMIP6_ScenarioMIP_NCC_NorESM2-MM_r1i1p1f1_mtl

Pass the input to an xclim partition function.

[11]:
# compute uncertainty partitioning
mean, uncertainties = xc.ensembles.hawkins_sutton(ds.tg_mean)
uncertainties
[11]:
<xarray.DataArray 'getitem-15cfe0d9e29101a802513a25b521f806' (uncertainty: 4,
                                                              time: 4)> Size: 128B
dask.array<concatenate, shape=(4, 4), dtype=float64, chunksize=(1, 2), chunktype=numpy.ndarray>
Coordinates:
  * uncertainty  (uncertainty) <U11 176B 'variability' 'model' ... 'total'
  * time         (time) object 32B 2001-01-01 00:00:00 ... 2004-01-01 00:00:00
Attributes:
    model:     ['NCC_NorESM2-MM_r1i1p1f1']
    scenario:  ['ssp126' 'ssp245' 'ssp370' 'ssp585']

NOTE

Note that the figanos library provides a function fg.partition to plot the uncertainties.

5. Ensemble robustness with multiple members

Sometimes a climate model has multiple members. These can be used to more accurately calculate the robustness. For example, take an ensemble of 5 simulations with the ids specified below. (Usually, fut and ref can be created with xc.ensemble.create_ensembles).

[12]:
import xclim as xc
import numpy as np
import xarray as xr
import xscen as xs

simulation_names = [
    "CMIP6_ScenarioMIP_CCCma_CanESM5_ssp370_r1i1p1f1_NAM",
    "CMIP6_ScenarioMIP_CCCma_CanESM5_ssp370_r2i1p1f1_NAM",
    "CMIP6_ScenarioMIP_CSIRO-ARCCSS_ACCESS-CM2_ssp370_r1i1p1f1_NAM",
    "CMIP6_ScenarioMIP_CSIRO-ARCCSS_ACCESS-CM2_ssp370_r3i1p1f1_NAM",
    "CMIP6_ScenarioMIP_CAS_FGOALS-g3_ssp370_r10i1p1f1_NAM",
]


# Reference period
ref = xr.DataArray(
    20 * np.random.random_sample((5, 30, 10, 10)) + 275,
    dims=("realization", "time", "lat", "lon"),
    coords={
        "time": xr.date_range("1990", periods=30, freq="YS"),
        "lat": np.arange(40, 50),
        "lon": np.arange(-70, -60),
        "realization": simulation_names,
    },
)

# Future
fut = xr.DataArray(
    20 * np.random.random_sample((5, 30, 10, 10)) + 275,
    dims=("realization", "time", "lat", "lon"),
    coords={
        "time": xr.date_range("2070", periods=30, freq="YS"),
        "lat": np.arange(40, 50),
        "lon": np.arange(-70, -60),
        "realization": simulation_names,
    },
)

Instead of considering each simulation as independent, we see that there are 3 models each with up to 2 members. We can separate the member dimension and create a new realization defined by the rest the the id. Note that it does not matter that the members do not have the same names, we are simply accumulating all the members associated with our new definition of realization.

[13]:
ref = xs.ensembles.unstack_ensemble_member(ref)
fut = xs.ensembles.unstack_ensemble_member(fut)
display(fut)
/home/docs/checkouts/xscen/conda/latest/lib/python3.13/site-packages/xscen/ensembles.py:881: FutureWarning: the `pandas.MultiIndex` object(s) passed as 'realization' coordinate(s) or data variable(s) will no longer be implicitly promoted and wrapped into multiple indexed coordinates in the future (i.e., one coordinate for each multi-index level + one dimension coordinate). If you want to keep this behavior, you need to first wrap it explicitly using `mindex_coords = xarray.Coordinates.from_pandas_multiindex(mindex_obj, 'dim')` and pass it as coordinates, e.g., `xarray.Dataset(coords=mindex_coords)`, `dataset.assign_coords(mindex_coords)` or `dataarray.assign_coords(mindex_coords)`.
/home/docs/checkouts/xscen/conda/latest/lib/python3.13/site-packages/xscen/ensembles.py:881: FutureWarning: the `pandas.MultiIndex` object(s) passed as 'realization' coordinate(s) or data variable(s) will no longer be implicitly promoted and wrapped into multiple indexed coordinates in the future (i.e., one coordinate for each multi-index level + one dimension coordinate). If you want to keep this behavior, you need to first wrap it explicitly using `mindex_coords = xarray.Coordinates.from_pandas_multiindex(mindex_obj, 'dim')` and pass it as coordinates, e.g., `xarray.Dataset(coords=mindex_coords)`, `dataset.assign_coords(mindex_coords)` or `dataarray.assign_coords(mindex_coords)`.
<xarray.DataArray (time: 30, lat: 10, lon: 10, realization: 3, member: 2)> Size: 144kB
array([[[[[292.38965484,          nan],
          [294.11797033, 293.73337292],
          [290.91859431, 275.31083894]],

         [[284.30529063,          nan],
          [291.88477593, 288.98298799],
          [278.62601696, 284.32976114]],

         [[280.36812096,          nan],
          [282.6738747 , 292.553718  ],
          [279.71943782, 279.40831722]],

         ...,

         [[289.9251591 ,          nan],
          [289.88019272, 293.89307384],
          [281.54186993, 283.40268906]],

         [[276.0358271 ,          nan],
          [276.31375771, 275.20049446],
...
          [284.2928846 , 287.15478807],
          [278.77286117, 282.92313962]],

         [[284.25990724,          nan],
          [286.72828302, 276.52595011],
          [283.05614786, 290.14493544]],

         ...,

         [[294.72696843,          nan],
          [294.69430672, 287.7650431 ],
          [275.51723818, 292.49703335]],

         [[276.01849142,          nan],
          [289.90544726, 279.87832881],
          [275.75419853, 293.98485943]],

         [[289.47981556,          nan],
          [280.75183363, 277.28885857],
          [288.03842532, 282.13899132]]]]], shape=(30, 10, 10, 3, 2))
Coordinates:
  * time         (time) datetime64[ns] 240B 2070-01-01 2071-01-01 ... 2099-01-01
  * lat          (lat) int64 80B 40 41 42 43 44 45 46 47 48 49
  * lon          (lon) int64 80B -70 -69 -68 -67 -66 -65 -64 -63 -62 -61
  * realization  (realization) object 24B 'CMIP6_ScenarioMIP_CAS_FGOALS-g3_ss...
  * member       (member) int64 16B 0 1
    n_pool       (realization) int64 24B 1 2 2

Now, we can calculate the robustness on these new inputs. The delta for each new realization will be the average of the delta of each of its members. Different tests will be able to use the member dimension, while some will throw an error. The fractions are counted on the new realization dimension (of length 3 in this example).

[14]:
out=xs.ensembles.ensemble_stats(
    fut.to_dataset(name="tas"),
    statistics=
    {'robustness_fractions':dict(ref=ref,
                                 dim=['realization','member'],
                                 test="signal-to-noise"),
    'robustness_categories':{}}
)
out
[14]:
<xarray.Dataset> Size: 7kB
Dimensions:                    (lat: 10, lon: 10)
Coordinates:
  * lat                        (lat) int64 80B 40 41 42 43 44 45 46 47 48 49
  * lon                        (lon) int64 80B -70 -69 -68 -67 ... -63 -62 -61
Data variables:
    tas_robustness_categories  (lat, lon) int64 800B 2 2 2 2 2 2 ... 2 2 2 2 2 2
    tas_changed                (lat, lon) float64 800B 0.0 0.0 ... 0.3333 0.0
    tas_positive               (lat, lon) float64 800B 0.3333 0.6667 ... 0.0
    tas_changed_positive       (lat, lon) float64 800B 0.0 0.0 0.0 ... 0.0 0.0
    tas_negative               (lat, lon) float64 800B 0.6667 0.3333 ... 1.0
    tas_changed_negative       (lat, lon) float64 800B 0.0 0.0 ... 0.3333 0.0
    tas_valid                  (lat, lon) float64 800B 1.0 1.0 1.0 ... 1.0 1.0
    tas_agree                  (lat, lon) float64 800B 0.6667 0.6667 ... 1.0
Attributes:
    cat:processing_level:  ensemble
    ensemble_size:         1