[dependabot]

Updates the requirements on datasets to permit the latest version.

Release notes

Sourced from datasets's releases.

5.1.0

New Formats

• Harbor ⚓: RL environments:
  • Add harbor benchmarks by @​lhoestq in huggingface/datasets#8729
• Vortex 🌀: High performance columnar data
  • Add support for datasets backed by vortex file format by @​robert3005 in huggingface/datasets#8502
• Bio formats 🧬: store biological data like DNA sequences, protein chains, and 3D molecular structures
  • feat(fasta): add lightweight FASTA file format support by @​behroozazarkhalili in huggingface/datasets#7923
  • Decode BioSequence and BioStructure records with universal newlines by @​behroozazarkhalili in huggingface/datasets#8600
  • Add lightweight FASTQ file format support by @​behroozazarkhalili in huggingface/datasets#7924
  • feat: Add GenBank file format support for biological sequence data by @​behroozazarkhalili in huggingface/datasets#7951
  • Add lightweight PDB (Protein Data Bank) file support by @​behroozazarkhalili in huggingface/datasets#7926
  • feat: Add mmCIF file support for macromolecular structures by @​behroozazarkhalili in huggingface/datasets#7925
  • feat(features): add BioSequence and BioStructure decodable features by @​behroozazarkhalili in huggingface/datasets#8542

Dataset Features

• Add a strategy argument to split_dataset_by_node for iterable datasets by @​behroozazarkhalili in huggingface/datasets#8565
• Allow IterableDataset.push_to_hub num_shards above dataset.num_shards by @​behroozazarkhalili in huggingface/datasets#8716

Other improvements and bug fixes

• Fix fixed size binary by @​Alex-PLACET in huggingface/datasets#8281
• Fix offline simulation with huggingface_hub 1.27 by @​hanouticelina in huggingface/datasets#8414
• Fix info from data_dir/data_files or malformed yaml by @​lhoestq in huggingface/datasets#8462
• Forward iter_arrow, is_typed and features from RepeatExamplesIterable by @​ebarkhordar in huggingface/datasets#8395
• Fix archive path check allowing sibling-directory escapes by @​icn5381 in huggingface/datasets#8490
• disallow hdf5 external links by @​lhoestq in huggingface/datasets#8523
• Fix deepcopy bottleneck in state_dict() by @​yuxin00j in huggingface/datasets#8387
• Optimize PyArrow block shuffling with dynamic rebatched chunk interleaving by @​yuxin00j in huggingface/datasets#8407
• Fix Pdf.decode_example raising TypeError on remote non-Hub paths by @​behroozazarkhalili in huggingface/datasets#8566
• Fix dead PDF processing link in README by @​aarushkandukoori in huggingface/datasets#8512
• Forward max_buffer_input_shards from IterableDatasetDict.shuffle by @​vineethsaivs in huggingface/datasets#8594
• Preserve storage options across URL hops by @​tandede in huggingface/datasets#8496
• Better error when a column written as null gets values in a later batch by @​ebarkhordar in huggingface/datasets#8591
• Advance the user-supplied generator in Dataset.shuffle even on cache hits by @​BenYang12 in huggingface/datasets#8589
• Bump fsspec upper bound to 2026.7.0 by @​roshanjrajan in huggingface/datasets#8561
• Shard the sources when skipping a distributed IterableDataset by @​ebarkhordar in huggingface/datasets#8569
• Fix infinite repetition of empty iterable datasets by @​Jokasa7 in huggingface/datasets#8577
• Reject a non-callable generator in GeneratorConfig by @​LeSingh1 in huggingface/datasets#8585
• Fix async map checkpoint accounting across repeated resumes by @​SBrouck in huggingface/datasets#8575
• [httpx migration] Import httpx from huggingface_hub, require huggingface_hub>=1.31.0 by @​Wauplin in huggingface/datasets#8598
• chore: enable Dependabot weekly GitHub Actions bumps by @​hf-dependantbot-rollout[bot] in huggingface/datasets#8221
• [docs] Fix legacy hf CLI references (datasets) by @​Wauplin in huggingface/datasets#8657
• Scope GITHUB_TOKEN permissions per job by @​hf-security-analysis[bot] in huggingface/datasets#8672
• Pin GitHub Actions to commit SHAs by @​hf-security-analysis[bot] in huggingface/datasets#8673
• Fix push_to_hub for IterableDataset with unknown features by @​davanstrien in huggingface/datasets#8658
• Vectorize interleave_datasets index generation (probabilities + first/all_exhausted) by @​saforem2 in huggingface/datasets#8318
• Fix torchcodec docs links by @​pratikgx in huggingface/datasets#8700
• [docs] Fix HF_HOME example in cache guide by @​ZainnQureshii in huggingface/datasets#8694
• Fix broken internal anchor link in share.mdx by @​aarushkandukoori in huggingface/datasets#8666
• Keep Json fields inside a struct from being double-encoded when some struct rows are null by @​breken-ai in huggingface/datasets#8712

... (truncated)

Commits

• efe6ea4 release 5.1.0 (#8745)
• 7f58239 harbor: fix paths on windows (#8739)
• c657c2d fix default iterable dataset dict and polars exports (#8632)
• 278d485 Fix repeated Arrow rebatch checkpoint restoration (#8727)
• b83f8b5 Speed up JSON export of indexed datasets (#8662)
• 2989571 Preserve source shards when shuffling iterable datasets (#8670)
• dcf79fd Bump huggingface_hub upper bound to <3.0 (datasets) (#8678)
• f63c726 Fix IterableDataset.filter with arrow, pandas and polars formats (#8725)
• 0358671 Allow IterableDataset.push_to_hub num_shards above dataset.num_shards (#8716)
• a826eb2 Fix epoch iteration after skipping or taking a streaming subset (#8707)
• Additional commits viewable in compare view

[dependabot]

OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting @dependabot ignore this major version or @dependabot ignore this minor version. You can also ignore all major, minor, or patch releases for a dependency by adding an ignore condition with the desired update_types to your config file.

If you change your mind, just re-open this PR and I'll resolve any conflicts on it.