RCE experimental dataset — processing code

Code that builds the RCE experimental dataset (Radiative Collector and Emitter, University of Lleida, 2019–2025) from the original laboratory files. The dataset itself is on Zenodo: https://doi.org/10.5281/zenodo.23386246 (all versions). This code produced version 0.2.4 (https://doi.org/10.5281/zenodo.23529834).

Layout

Path	Role
data/etl/index_raw.py	Catalogue of the original files (path, size, SHA-256) → data/catalog/raw_index.csv
data/etl/standardize.py	Source layer: reads the 2019 data-logger CSVs, the monthly workbooks Data_YYYY_MM.xlsx and the WinCC SCADA exports; one wide table per day with the files behind it
dataset_release/build_release.py	Builds the package: 5-minute harmonised tables, cell-level quality flags, per-day labels, quality tables, selection of original files, documentation, metadata and validation
dataset_release/release/	vmap (variable map), harmonize, flags, labels, quality, rawpack, docs_gen, validate; variable_map.yaml maps source columns to the canonical variables
dataset_release/docs_src/	Documentation copied into the package
dataset_release/tests/	Tests on synthetic data

Use

pip install -r requirements.txt
cd dataset_release && python -m pytest tests -q          # tests on synthetic data

The full build needs the laboratory archive in its original folder layout (raw_data/, placed next to this repository folder). The published raw/ folder of the dataset and its raw/index.csv identify every input file by its original name and SHA-256.

python data/etl/index_raw.py
python data/etl/standardize.py --out data/11_standardized_v2
python dataset_release/build_release.py --version 0.2.4

Licence

Code: MIT (see LICENSE). Dataset: CC BY 4.0 (see the Zenodo record).

Citation

If you use the data, cite the dataset (see CITATION.cff in the Zenodo record).