Skip to content

MCR-3858 Add archive (zip/tar) download endpoints to REST API v2 - #3162

Merged
sebhofmann merged 11 commits into
mainfrom
issues/MCR-3858-Add_archive_endpoints_to_REST_API_v2
Oct 22, 2026
Merged

sebhofmann merged 11 commits into
mainfrom
issues/MCR-3858-Add_archive_endpoints_to_REST_API_v2

Conversation

@sebhofmann

@sebhofmann sebhofmann commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Link to jira

Adds archive (zip/tar) download to the REST API v2, so that links to MCRZipServlet and MCRTarServlet in export formats can later be replaced by REST API links.

There is no separate archive endpoint. The existing contents resource of a derivate delivers a directory as archive:

  • GET /api/v2/objects/{mcrid}/derivates/{derid}/contents (or any sub directory) with Accept: application/zip or Accept: application/x-tar streams all files of the directory as archive. Without such an Accept header the directory is listed as before.
  • GET /api/v2/objects/{mcrid}/derivates/{derid}/contents.zip and contents.tar are shortcuts for the whole derivate. They are mapped to the Accept headers by MCRContentNegotiationViaExtensionFilter.

There is no object level archive.

Design decisions

  • The format specific entry code of MCRZipServlet and MCRTarServlet is moved to the servlet independent classes MCRArchiver, MCRZipArchiver and MCRTarArchiver in the new package org.mycore.datamodel.niofs.archive and reused by MCRRestDerivateContents. The servlets delegate to them. Their protected API is unchanged, because overlays extend them (e.g. reposis_ditav, reposis_soviet_survivors, dbt), so MCRCompressServlet keeps its own file visitor.
  • Mime type and file extension of each format are defined once as constants in the archiver classes (MIME_TYPE, FILE_EXTENSION). @Produces, the content negotiation variants and the extension mapping of MCRContentNegotiationViaExtensionFilter are derived from them.
  • Content-Disposition contains an ASCII fallback in filename and the exact name in filename* (RFC 6266/8187), see MCRRestUtils.getContentDisposition().
  • Streaming: the entity stream is wrapped in a CloseShieldOutputStream, so closing the archive does not end the MCRSession too early. If an error occurs while the archive is written, the archive is not finished, so the client receives a broken download instead of a valid but incomplete archive (the status 200 is already sent at that point).
  • Access: read permission on the derivate is checked by MCRRestAuthorizationFilter, as for every other contents request.
  • The extension shortcuts only apply to contents.zip and contents.tar of the derivate root. The regex MCR.RestAPI.V2.ContentNegotiationViaExtensionFilter.RegEx still ignores everything behind /contents/, so a file called archive.zip in a derivate is not touched. An application that sets its own regex has to add contents.zip and contents.tar to use the shortcuts. The Accept header works without it.
  • Caching: archives have no Last-Modified header and conditional requests are not supported, because that would need a second walk over the whole directory tree. Cache-Control is the one of the contents resource (one day), as for single files. The archive comment contains the creation time.
  • Locking while an archive is created is out of scope, see the comment in the ticket.
  • Files are not affected: a file is delivered as before, whatever Accept header is sent.
  • Error responses: there is no special handling for archives. If a client accepts nothing but an archive type (this includes contents.zip), an error response cannot be written in that type and Jersey answers with a server error. Files behave the same way when a client sends a single specific Accept type. Fixing this belongs to the general error handling of the REST API (MCRErrorResponse is written with the negotiated type, even if that type cannot hold an error) and is out of scope of this PR.

How to test

  • curl -OJ -H "Accept: application/zip" <base>/api/v2/objects/{mcrid}/derivates/{derid}/contents and with application/x-tar
  • curl -OJ <base>/api/v2/objects/{mcrid}/derivates/{derid}/contents.zip and contents.tar
  • the same for a sub directory (Accept header only)
  • curl <base>/api/v2/objects/{mcrid}/derivates/{derid}/contents without Accept header: expect the directory listing
  • unknown derivate without archive Accept header: expect 404 as JSON
  • a sub directory with special characters in its name: Content-Disposition contains filename*=UTF-8''...

The entries of the archives are identical to the ones of MCRZipServlet/{derid} and MCRTarServlet/{derid}.

Status of testing: Before the review, tested locally against a deployed MIR with the MyCoRe branches of this PR and MCR-3860, as guest and as logged in user:
contents.zip, contents.tar, Accept: application/zip and application/x-tar for the derivate and for a sub directory (entries identical to MCRZipServlet/MCRTarServlet), directory listing without Accept header, files (also a file called archive.zip, with and without an archive Accept header), 304 for If-Modified-Since, 404 (unknown derivate, derivate of another object, unknown directory) and 401 (derivate not readable for guests) as JSON, contents.xml and contents.json as before. At that time the build of mycore-restapi and mycore-base was run without tests (-DskipTests), only checkstyle and PMD.

The changes from the review (removed Accept header fallback, encoded Content-Disposition, shared mime types, streaming without Last-Modified, new package of the archivers) are only verified by the build with all tests of mycore-base and mycore-restapi (including the new MCRRestUtilsTest), not yet against a deployed MIR. In mycore-base the only failing test locally is MCRCheckPermissionChainURIResolverTest.resolveReadObjectAllowed, which fails the same way without this PR.

Pull Request Checklist (Author)

Ticket & Documentation

  • The issue in the ticket is clearly described and the solution is documented.
  • Design decisions (if any) are explained.
  • The ticket references the correct source and target branches.
  • The fixed-version is correctly set in the ticket and matches the PR's target branch (main).

Feature & Improvement Specific Checks

  • Instructions on how to test or use the feature are included or linked (e.g. to documentation).
  • For UI changes: before & after screenshots are attached. (no UI changes)
  • New features or migrations are documented.
  • Does this change affect existing applications, data, or configurations? (the default of the regex property is extended)
    • Yes: Is a migration required? If yes, describe it. (only for applications with their own regex, see above)
    • Breaking change is marked in the commit message.

Testing

  • I have tested the changes locally.
  • The feature behaves as described in the ticket.
  • Were existing tests modified? (no)

MCR Conventions & Metadata

  • MCR naming conventions are followed
  • If the public API has changed: (new classes added, the API of the servlets is unchanged)
  • Java license headers are added where necessary.
  • Javadoc is written for non-self-explanatory classes/methods (Clean Code).
  • All configuration options are documented in Javadoc and mycore.properties.
  • No default properties are hardcoded — all set via mycore.properties.

Multi-Repo Considerations

…act archiver classes from servlets

Move the format specific archive entry code of MCRZipServlet and MCRTarServlet
into the servlet independent classes MCRArchiver, MCRZipArchiver and
MCRTarArchiver, so it can be reused outside of servlets. The servlets delegate
to them, their protected API is unchanged.
…derivate archive resource

Add GET /objects/{mcrid}/derivates/{derid}/archive/zip and /archive/tar.
The archives contain all files of the derivate and are streamed.
Last-Modified is the latest modification date of all files and directories
of the derivate and conditional requests are supported.
…e archives via contents

Remove the separate archive endpoints. The contents resource of a derivate
delivers a directory as archive if the Accept header is application/zip or
application/x-tar. contents.zip and contents.tar are mapped to these media
types by the content negotiation via extension filter, files in a derivate
are not affected.
…ess review

- remove Accept header manipulation for archive requests from
  MCRContentNegotiationViaExtensionFilter
- encode file name in Content-Disposition (RFC 6266/8187) via
  MCRRestUtils.getContentDisposition()
- derive archive media types, variants and file extensions from the
  archivers (MIME_TYPE, FILE_EXTENSION constants)
- do not finish an archive if streaming fails, so clients get a broken
  download instead of a valid but incomplete archive
- shield the entity stream from closing, so the MCRSession is not ended
  too early
- walk the directory tree only once (no Last-Modified for archives)
- use the creation time in the archive comment
- move MCRArchiver, MCRZipArchiver and MCRTarArchiver to
  org.mycore.datamodel.niofs.archive
- MCRCompressServlet uses MCRArchiver.sendTree instead of its own visitor

BREAKING CHANGE: MCRCompressServlet now requires T extends
ArchiveOutputStream<?> and a getArchiver() implementation; the abstract
methods sendCompressedDirectory, sendCompressedFile,
sendMetadataCompressed, getMimeType, getFileExtension, createContainer
and disposeContainer were removed.
… protected API of compress servlets

Restore MCRCompressServlet, MCRZipServlet and MCRTarServlet with their
previous protected API, because overlays extend them (reposis_ditav,
reposis_soviet_survivors, dbt). Only the imports of the moved archiver
classes change.

This reverts the BREAKING CHANGE announced in the previous commit: there
is no breaking change in this pull request anymore.
…ess second review

Send Last-Modified for archives and answer conditional requests, deliver errors as JSON if only archives are accepted, close the license header of package-info.java and deprecate the unused getFilename of MCRCompressServlet.
… getFilename of MCRCompressServlet for removal
@sebhofmann
sebhofmann merged commit 1e48ccf into main Oct 22, 2026
3 checks passed
@sebhofmann
sebhofmann deleted the issues/MCR-3858-Add_archive_endpoints_to_REST_API_v2 branch October 22, 2026 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants