A collection of bias correction techniques written in Python - for climate sciences.
GPL-3.0 License
Welcome to python-cmethods, a powerful Python package designed for bias correction and adjustment of climate data. Built with a focus on ease of use and efficiency, python-cmethods offers a comprehensive suite of functions tailored for applying bias correction methods to climate model simulations and observational datasets via command-line interface and API.
Please cite this project as described in https://zenodo.org/doi/10.5281/zenodo.7652755.
Bias correction in climate research involves the adjustment of systematic errors or biases present in climate model simulations or observational datasets to improve their accuracy and reliability, ensuring that the data better represents actual climate conditions. This process typically involves statistical methods or empirical relationships to correct for biases caused by factors such as instrument calibration, spatial resolution, or model deficiencies.
python-cmethods empowers scientists to effectively address those biases in climate data, ensuring greater accuracy and reliability in research and decision-making processes. By leveraging cutting-edge techniques and seamless integration with popular libraries like xarray and Dask, this package simplifies the process of bias adjustment, even when dealing with large-scale climate simulations and extensive spatial domains.
In this way, for example, modeled data, which on average represent values that are too cold, can be easily bias-corrected by applying any adjustment procedure included in this package.
For instance, modeled data can report values that are way colder than the those data reported by reanalysis time-series. To address this issue, an adjustment procedure can be employed. The figure below illustrates the observed, modeled, and adjusted values, revealing that the delta-adjusted time series ($T^{*DM}{sim,p}$) is significantly more similar to the observational data ($T{obs,p}$) than the raw model output ($T{sim,p}$).
The mathematical foundations supporting each bias correction technique implemented in python-cmethods are integral to the package, ensuring transparency and reproducibility in the correction process. Each method is accompanied by references to trusted publications, reinforcing the reliability and rigor of the corrections applied.
python-cmethods provides the following bias correction techniques:
Please refer to the official documentation for more information about these methods as well as sample scripts: https://python-cmethods.readthedocs.io/en/stable/
The training data should have the same temporal resolution.
Except for the variance scaling, all methods can be applied on stochastic and non-stochastic climate variables. Variance scaling can only be applied on non-stochastic climate variables.
Non-stochastic climate variables are those that can be predicted with relative certainty based on factors such as location, elevation, and season. Examples of non-stochastic climate variables include air temperature, air pressure, and solar radiation.
Stochastic climate variables, on the other hand, are those that exhibit a high degree of variability and unpredictability, making them difficult to forecast accurately. Precipitation is an example of a stochastic climate variable because it can vary greatly in timing, intensity, and location due to complex atmospheric and meteorological processes.
Except for the detrended quantile mapping (DQM) technique, all methods can be applied to 1- and 3-dimensional data sets. The implementation of DQM to 3-dimensional data is still in progress.
Except for DQM, all methods can be applied using cmethods.adjust
. Chunked
data for computing e.g. in a dask cluster is possible as well.
For any questions -- please open an issue at https://github.com/btschwertfeger/python-cmethods/issues
If the installation fails due to missing HDF5 headers, ensure that 'hdf5' and 'netcdf' are pre-installed, e.g. on macOS using:
brew install hdf5 netcdf
.
python3 -m pip install python-cmethods
The package is also available via conda-forge. See conda-forge/python_cmethods for more information.
The python-cmethods package provides a command-line interface for applying various bias correction methods out of the box.
Keep in mind that due to the various kinds of data and possibilities to pre-process those, the CLI only provides a basic application of the implemented techniques. For special parameters, adjustments, and data preparation, please use programming interface.
Listing the parameters and their requirements is available by passing the
--help
option:
cmethods --help
Applying the cmethods tool on the provided example data using the linear scaling approach is shown below:
cmethods \
--obs examples/input_data/observations.nc \
--simh examples/input_data/control.nc \
--simp examples/input_data/scenario.nc \
--method linear_scaling \
--kind add \
--variable tas \
--group time.month \
--output linear_scaling.nc
2024/04/08 18:11:12 INFO | Loading data sets ...
2024/04/08 18:11:12 INFO | Data sets loaded ...
2024/04/08 18:11:12 INFO | Applying linear_scaling ...
2024/04/08 18:11:15 INFO | Saving result to linear_scaling.nc ...
For applying a distribution-based bias correction technique, the following example may help:
cmethods \
--obs examples/input_data/observations.nc \
--simh examples/input_data/control.nc \
--simp examples/input_data/scenario.nc \
--method quantile_delta_mapping \
--kind add \
--variable tas \
--quantiles 1000 \
--output quantile_delta_mapping.nc
2024/04/08 18:16:34 INFO | Loading data sets ...
2024/04/08 18:16:35 INFO | Data sets loaded ...
2024/04/08 18:16:35 INFO | Applying quantile_delta_mapping ...
2024/04/08 18:16:35 INFO | Saving result to quantile_delta_mapping.nc ...
import xarray as xr
from cmethods import adjust
obsh = xr.open_dataset("input_data/observations.nc")
simh = xr.open_dataset("input_data/control.nc")
simp = xr.open_dataset("input_data/scenario.nc")
# adjust only one grid cell
ls_result = adjust(
method="linear_scaling",
obs=obsh["tas"][:, 0, 0],
simh=simh["tas"][:, 0, 0],
simp=simp["tas"][:, 0, 0],
kind="+",
group="time.month",
)
# adjust all grid cells
qdm_result = adjust(
method="quantile_delta_mapping",
obs=obsh["tas"],
simh=simh["tas"],
simp=simp["tas"],
n_quantiles=1000,
kind="+",
)
# to calculate the relative rather than the absolute change,
# '*' can be used instead of '+' (this is preferred when adjusting
# stochastic variables like precipitation)
It is also possible to adjust chunked data sets. Feel free to have a look into
tests/test_zarr_dask_compatibility.py
to get a starting point.
Notes:
max_scaling_factor
.cmethods.adjust
function./examples/examples.ipynb
/examples/input_data/*.nc
group='time.dayofyear'
. Alternatively, it is possible not to scale using… are welcome but: