Skip to content

migrate Zarr backend to ZarrCore and add ZarrZip extension support - #73

Open
lazarusA wants to merge 6 commits into
masterfrom
la/zarrcore
Open

lazarusA wants to merge 6 commits into
masterfrom
la/zarrcore

Conversation

@lazarusA

@lazarusA lazarusA commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

as hinted in #72

Plus is needed to properly implement the multi scales work on PyramidSchemes.jl

@codecov

codecov Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.23810% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.11%. Comparing base (49d7155) to head (9040952).

Files with missing lines Patch % Lines
ext/ZarrCoreExt.jl 94.82% 3 Missing ⚠️
ext/ZarrZipExt.jl 93.75% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #73      +/-   ##
==========================================
+ Coverage   52.04%   60.11%   +8.07%     
==========================================
  Files          12       13       +1     
  Lines         490      514      +24     
==========================================
+ Hits          255      309      +54     
+ Misses        235      205      -30     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ressed

create_dataset now forwards `compressor` (Zarr) and `compress` (NetCDF)
to the coordinate arrays, not only to the data variables. Other keyword
arguments such as filters stay data-variable only, since they can be
type specific.

NetCDF iscompressed always returned false because NetCDF.jl does not read
the deflate level back when opening a variable. Query it with
nc_inq_var_deflate instead.
Zarr v3 allows omitting dimension_names, which zarr-python does for
0-dimensional arrays. get_var_dims threw for such arrays because neither
dimension_names nor _ARRAY_DIMENSIONS was present. Return an empty list
for arrays without dimensions instead.
Comment thread ext/ZarrCoreExt.jl Outdated
@felixcremer

Copy link
Copy Markdown
Member

I changed to using the zarr_format function from ZarrCore and renamed the corresponding keyword to format. Why is this breaking?

Comment thread ext/ZarrCoreExt.jl

function __init__()
@debug "new driver key :zarr, updating backendlist."
YAB.backendlist[:zarr] = ZarrDataset

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as for the breaking thing, I'm not sure about this backend list, since now the dependency is in ZarrCore and not Zarr, do we still use the key :zarr here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still use the key zarr here, because this is about the format and I would say the existence of ZarrCore is an implementation detail.

@lazarusA

lazarusA commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator Author

by loading Zarr.jl should not affect end users, but in isolation is breaking, then in order to access other features, like Zip, you need to be explicit about what to load.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants