Skip to content

Split cloud cover support out of the Varda-single 1.0 inference configs - #283

Open
dnerini wants to merge 1 commit into
mainfrom
fix/varda-single-oper-without-clouds
Open

dnerini wants to merge 1 commit into
mainfrom
fix/varda-single-oper-without-clouds

Conversation

@dnerini

@dnerini dnerini commented Oct 30, 2026

Copy link
Copy Markdown
Member

#270 added cloud cover (tcc, lcc, mcc, hcc) and skt to the shared -oper forecaster and downscaler configs, including a cloud cover rescale on the ICON-CH1 cutout. Varda-single 1.0 does not use these variables, and the rescale fails (AssertionError: tcc) with the 2024-2026 v2.0 dataset, which names them CLCT, CLCL, CLCM, CLCH and SKT.

#270 added cloud cover (tcc, lcc, mcc, hcc) and skt to the shared -oper
forecaster and downscaler configs, including a cloud cover rescale on the
ICON-CH1 cutout. Varda-single 1.0 does not use these variables, and the
rescale fails (AssertionError: tcc) with the 2024-2026 v2.0 dataset, which
names them CLCT, CLCL, CLCM, CLCH and SKT.

- Move the cloud cover setup of #270 unchanged to new -clouds configs and
  metadata patches (still on the v1.0 dataset).
- Restore the -oper configs and patches to their pre-#270 state, keeping
  the SKT fixes (downscaler input namer, IFS GRIB encoding of skt).
- Point the -oper patches at mch-ich1-1km-2024-2026-1h-pl13-ifsnames-v2.0
  restricted to 2024-05-01..2026-05-01, so 2026 reference times can be
  evaluated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dnerini
dnerini requested a review from Louis-Frey October 30, 2026 13:44
@Louis-Frey

Copy link
Copy Markdown
Contributor

Thanks for spotting this Daniele, I'll check it.

@cosunae

cosunae commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

I will insist on my comment here:
#282 (comment)

to avoid these kind of issues, inconsistencies of configs with checkpoints and a high maintenance in evalml, we could try to make the effort to move them to the workspaces.

@Louis-Frey

Copy link
Copy Markdown
Contributor

Thanks Daniele, and sorry for introducing this in #270. I tested the PR and it looks good.

Test: one config with two init times, 2025-06-01 (in both KENDA-CH1 v1.0 and v2.0) and 2026-02-01 (v2.0 only), short lead times. I ran it on this PR and, as a reference, on my branch (which reads v2.0 and renames CLCT etc. back to tcc etc.), then compared the GRIB outputs byte by byte:

Run Configs 2025-06-01 2026-02-01
Varda-single 1.0 (forecaster + downscaler) -oper runs, identical to reference runs, identical to reference
9f4626a7 (forecasters-ich1-oper-fixed.yaml) -oper runs, identical to reference runs, identical to reference
RC (forecaster 07f64210 + downscaler d642c4ba) -clouds runs, identical to reference fails: date not in v1.0

So the split doesn't change any output, and Varda-single 1.0 now works on v2.0 including 2026 dates.

One follow-up (doesn't block this PR): since the -clouds patches still read v1.0, the RC can't be evaluated for 2026 dates. Pointing them at v2.0 with a rename would fix that, as on my branch:

- dataset: /store_new/mch/msopr/ml/datasets/mch-ich1-1km-2024-2026-1h-pl13-ifsnames-v2.0.zarr
  rename: {CLCT: tcc, CLCL: lcc, CLCM: mcc, CLCH: hcc, SKT: skt, SSO_THETA: anor, SSO_GAMMA: isor}
  rescale: {tcc: [0.01, 0.0], lcc: [0.01, 0.0], mcc: [0.01, 0.0], hcc: [0.01, 0.0]}

Maybe worth doing at some point.

@cosunae I agree that moving the inference configs into the workspaces would avoid this kind of issue. I don't have time for it right now, but I'm happy to help with it later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants