Changelog

Changelog#

v8.2.0 (2026-04-16)#

Contributors to this version: Ludwig Lierhammer (@ludwiglierhammer)

Announcements#

  • This release drops support for Python 3.10 and adds support for Python 3.14 (#299)

Breaking changes#

  • obs_suite: use parquet files instead of csv files as standard input and ouptu files (#254, #268)

Internal changes#

  • implement QC tests on DataBundle objects (#253, #257)

  • update scripts to match cdm_reader_mapper == v4.2.1 (#268)

  • obs_suite: level3 no hard-coded data types needed when using parquet files instead of csv files (#254, #268)

v8.1.0 (2026-01-26)#

Contributors to this version: Ludwig Lierhammer (@ludwiglierhammer)

New features and enhancements#

  • obs_suite: adding release8.1 configuration files (#226, #216)

  • obs_suite: run level skripts in arbitrary order (#226)

  • obs_suite: additionally, write level3 result data to parquet file (#216)

Breaking changes#

  • obs_suite: level3 write all table results into one output DataFrame not only observations-slp (#216)

  • obs_suite: update level3 columns (#216)

Bug fixes#

  • obs_suite: level1e the data output format is equal to the data input format again (#189, #210)

  • obs_suite: level1d Pub47 mapping is working again with cdm_reader_mapper.map_model(…, drop_duplicates=False, drop_missing_obs=False) (#187, #215)

Internal changes#

  • pyproject.toml: pytest expects python_files to be a list (#210)

v8.0.0 (2025-10-21)#

Contributors to this version: Ludwig Lierhammer (@ludwiglierhammer) and John Kennedy (@jjk-code-otter).

Announcements#

  • This release drops support for Python 3.9 and adds support for Python 3.13 (#76).

  • qc_suite is now in glamod/marine_qc (#117).

New features and enhancements#

  • obs_suite: add new level3 for extracting pressure data and merging header and observations-slp tables into one single CDM-OBS-CORE table (#75)

  • obs_suite: add ICOADS_R3.0.0T configuration files for release_7.0 (#75)

  • obs_suite: mark QC flags of observations as 9 if they are on a blacklistd (#137)

  • add environment.yml file (#76)

  • add dummy observation configuration files for release_8.0 (#113, #102)

  • make this tool running with cdm_reader_mapper >= 2.1.0 (#113, #102)

  • new module split to split decks from single deck into new multiple deck list (#140)

  • new CLI split_suite to split decks from single deck into new multiple deck list (#140)

  • obs_suite: run jobs in parallel using nohup (#140)

  • merge_suite: not only running for level1a data (#140)

  • obs_suite: new release 8.0 config files for : ICOADS_R3.0.0T, ICOADS_R3.0.2T, C-RAID_1.1 and C-RAID_1.2 (#113, #140)

  • new decorator auto_cast to automatically convert input data type to desired data type (#140)

  • obs_suite: level1a now flags data on blacklist (6) and data with generic IDs (88) (#147, #140)

  • obs_suite: level1e now uses marine_qc to calculate quality control flags (#113, #140)

CI changes#

  • rename GitHub workflow ci to testing_suite (#76)

  • GitHub workflow for testing_suite now uses uv for environment management, replacing micromamba (#76)

  • Update GitHub workflow for testing_suite for astral-sh/uv-setup >= 6.0.0 (#124)

Internal changes#

  • obs_suite: update ICOADS_R3.0.0T release_4.0 time period list (#73)

  • obs_suite: combine loops over decks in level_slurm.py and config_array.py to one single loop (#74, #79)

  • obs_suie: name log files according to the date of the source files instead of simply numbering them consecutively (#74, #79)

  • obs_suite: standardize level scripts (#79)

  • rename ci/requirements to CI, tidy up requirements and add dependencies to pyproject.toml file (#76)

  • merge_suite: use joblib for parallelization (#140)

  • obs_suite: restructure level1b, level1c, level1d and level_slurm scripts (#113, #140)

Breaking changes#

  • obs_suite: rename level3 output from <YYYY>-<MM>-<RELEASE>-<UPDATE>-pressure_data.psv to pressure-data-<YYYY>-<MM>-<RELEASE>-<UPDATE>.psv (#79)

  • command-line interface: set default release from “release_7.0” to “release_8.0” (#113, #102)

  • extract blacklisting and some ICOADS specific functions from qc_suite to obs_suite (#131, #137)

  • refactor qc_suite and move to glamod/marine_qc (#108, #117)

  • rename module post_processing to merge and CLI post_proc to merge_suite (#140)

v7.1.0 (2024-11-25)#

Contributors to this version: Ludwig Lierhammer (@ludwiglierhammer)

Announcements#

  • make preparations for zenodo DOI assignment (#4, #42: pull:54)

  • adding readthedocs documentation (#42)

  • Now under Apache v2.0 license (#42)

  • Thanks to NOC, Cookiecutter and GNU parallel (#53, #54)

  • glamod-marine-processing has migrated its development branch name from master to main.

  • final GLAMOD marine processing data release 7.0 version (#63)

New features and enhancements#

  • obs_suite: add quality checks for both Wind speed and wind direction in level1e script (#20, #22)

  • pre_proc: add pre-processing for ICOADS data (#24)

  • post_proc: add post processing for C-RAID level1a data (#26)

  • obs_suite: optionally, set list of decks to process (#25)

  • obs_suite: optionally, set both release period init and end year (#25)

  • obs_suite: running with C-RAID data (#25)

  • obs_suite: optionally, set both source and destination level, release and dataset (#67)

  • obs_suite: optionally, set both path to NOC correction data and NOC version in level1b (#67)

  • obs_suite: optionally, set path to Pub47 data in level1d (#67)

  • obs_suite and qc_suite: optionally, run jobs in parallel with gnu_parallel (#41)´

  • post_proc: optionally, post-processing for ICOADS data (#46)

  • post_proc: optionally, merge data from additional directories (invalid, excluded) to a new deck dataset (#52)

Internal changes#

  • obs_suite: take data paths from already created configuration files (#67)

  • obs_suite: configuration files for C-RAID (#25)

  • obs_suite: adjust to cdm_reader_mapper version v0.4.0 and further versions (#21, #28)

Breaking changes#

  • delete metadata suite, config suite and not-used scripts/modules (#14, #16)

  • obs_suite: date information is NOT mandatory in filenames anymore (#25)

  • obs_suite: pass tables if no correction or quality control file are available (#25)

  • obs_suite: adjust both process deck lists and processing init/end years to release7.0 requirements (#27)

  • obs_suite: new ICOADS_R3.0.2T deck list after level 1a (#40)

  • obs_suite: starting with year 2014 (#40)

  • set BASTION do default machine (bastion01.core.ichec.ie) (#37)

  • obs_suite: use duplicate checker from cdm_reader_mapper instead of NOc correction files in level1b (#37)

  • obs_suite: create only one task for level2 (#45)

  • obs_suite: rename Pub47 data from {year}-{month}-01.csv to pub47-{year}-{month}.csv in level1d script (#48)

  • obs_suite: set release period to 2015 to 2023 (#49)

  • obs_suite: if no qc files available: set report_quality from 2 (not checked) to 0 (passed) in level1e script (#50)

  • obs_suite: if report_id is not available in any observations: remove report_id from header (and vice versa) in level1e script (#50)

  • obs_suite: update configuration file structure of previous GLAMOD data releases (#67)

  • qc_suite: no need for NOC correction files (#39)

  • qc_suite: set minimum QC end year from 2022 to 1948 (#52)

  • qc_suite: update job list for release 7.0 (#52)

Bug fixes#

  • obs_suite: fixing observation suite level1e tests (#17)

  • obs_suite: QC mapping from v7.0.0 is now running by setting values of location_quality and report_time_quality to str (#18)

  • obs_suite: use only observation reports that are also available in the header file (#44, #45)

  • qc_suite: take qc source data from level1d files instead of level1a (#47)

  • qc_suite: update deck list for quality control (#47)

  • qc_suite: ignore reports with invalid date time information (#52, #58)

  • obs_suite: allow mixed date time formats in level1c (#62)

v7.0.0 (2024-06-13)#

Contributors to this version: Ludwig Lierhammer (@ludwiglierhammer)

Announcements#

renaming release name to vX.Y.Z

release_7.0.0 (2024-06-13)#

Contributors to this version: Ludwig Lierhammer (@ludwiglierhammer)

Breaking changes#

  • delete empty and not used files, functions and folders (#3)

  • create requirements for each suite (#3)

  • rebuild to a installable python package (#3)

  • install package and requirements via a pyproject.toml file (:pul::3)

  • change QC mapping in obs_suite level1e (#7, #8):

    • if location_quality is equal 2 set both report_quality and quality_flag to 1

    • if report_time_quality is equal 4 or 5 set both report_quality and quality_flag to 1

New features and enhancements#

  • add some information files: AUTHORS.rst, CHANGES.rst, CONTRIBUTING.rst and LICENSE (#3)

  • make us of pre-commit (#3)

  • make use of an command-line interface to create suite PYTHON and SLURM scripts (#3, #5)

  • add new release 7.0 configuration files (#3)

  • set some default directories and SLURM settings for both HPC systems KAY and MeluXina (#3)

Internal changes#

  • reduce complexity of some functions (#3)

  • adding observational testing suite (#5, #5)

  • load data from cdm-testdata (#11)