Repository navigation
Bring the v3.0.0 work into main (#20 to #23) - #25
Merged
Merged
Conversation
- Dataclass models with ISO dates and integer tonnages for vessels, inspections, history, search results, and fleets - Exception hierarchy that separates authentication failures, missing records, expired sessions, network problems, and page layout changes - Parsers locate panels by title and table columns by header text instead of position, and ignore mobile duplicate rows - Inspections reported to more than one PSC regime are merged into a single inspection instead of producing records with shifted columns - Parse safety management certificates, P&I insurance, class surveys, classification history, human element deficiencies, and fleet roles - Map the XCM flag image code to the ISO code for Comoros - Anonymized fixtures recorded from equasis.org, a scrubbing tool, and golden tests for every page type
Client and transport - Retry only transient failures (connection errors, 429, 5xx) with exponential backoff and Retry-After support; client errors fail fast - Pace requests with a configurable minimum interval (--delay) - Verify logins, detect expired sessions, and log in again once - Distinguish missing vessels and companies from errors and layout changes; secondary pages that fail to parse become warnings - Follow pagination for search results and fleets - Look up ships by exact IMO number, MMSI, or call sign Output - JSON documents with a schema version and retrieval timestamp - JSON Lines output that streams batch results as they complete - CSV written with the csv module, with headers and proper quoting - Output format inferred from the --output-file extension; files are written atomically as UTF-8 Command line - Output options work before or after the command name - Results go to stdout; progress, warnings, and errors go to stderr - Documented exit codes for scripting (not found, authentication, partial batch failure, layout change) - Batch input supports inline comments, stdin, de-duplication, and IMO check-digit warnings; batches continue on error unless --fail-fast - Fleet lookups resolve company names, prompt when a name is ambiguous on a terminal, and support --company-id and --first-match - --debug logs HTTP activity and --save-html captures pages for bug reports - configure gains --test and verifies credentials during --setup Shared service layer used by both the command line and the interactive shell, so batch handling and company resolution behave the same way. Credentials are written atomically with owner-only permissions, and a warning is shown when the credentials file is readable by others.
The canary runs the live test suite against equasis.org when the EQUASIS_USERNAME and EQUASIS_PASSWORD repository secrets are set, so website changes are detected before users report them.
The bs4 package does not re-export NavigableString in its type hints, which failed strict type checking in CI.
Core reliability: parsers, client, output, and CLI rebuilt and tested
tempfile.mkstemp creates owner-only files, so saved results were not readable by other users or tools that expect default permissions. Write to a uniquely named sibling file instead, then replace the target.
The October 2025 shell rewrite dropped the fleet and batch handlers and the output-format detection for saved files. The shell now runs every command through the same service layer as the command line. - batch processes vessels (/imos, /file) and companies (/companies, /company-file) with per-item progress and a combined report (#16) - /output FILE writes JSON, JSON Lines, or CSV according to the file extension instead of table text (#17) - fleet resolves company names, lists the matches when a name is ambiguous, and accepts /id or /first (#18) - search supports /imo, /mmsi, and /callsign exact lookups, /type, and /pages; help COMMAND shows parameters for each command - Commands are parsed with shell quoting, typographic quotes, and hyphenated parameters; unknown parameters are reported - Output from worker threads is posted to the event loop instead of redirecting sys.stdout, and log messages appear once - Menus follow the typed text, Tab completes, ? only toggles help on an empty line, j/k scroll in scroll mode, and output is capped at 5,000 lines - Command parsing and execution live in tui/commands.py with no terminal code, so they are fully covered by tests Fixes #16 Fixes #17 Fixes #18
Shell-style parsing treated backslashes as escape characters, so Windows paths such as C:\Users\me\imos.txt lost their separators and saved files ended up in the wrong place. '#' is also no longer treated as the start of a comment inside values.
Interactive shell: batch, fleet, and file output work again
Indicators summarise signals researchers look for, each taken directly from Equasis data and reported without an arbitrary composite score: - name, flag, and owner or manager changes in the last 12 and 36 months - inspections, detentions, and deficiencies in the last 36 months - flag on a Paris or Tokyo MoU grey or black list, USCG targeting - IACS classification and recent class withdrawals or suspensions - P&I insurance on record - whether the MMSI's Maritime Identification Digits match the flag state Indicators appear as a plain-language section in table output and as an indicators object in JSON and JSON Lines; CSV gains summary columns. Country and MID reference data ship with the package; the MID table is attributed in NOTICE.
Repeated lookups no longer send the same requests to Equasis: regular content pages are stored on disk for 24 hours and reused, which keeps load on Equasis low, makes repeat lookups near-instant, and lets an interrupted batch be re-run without fetching completed items again. - Login, error, and not-found pages are never cached - The cache directory is private to the user because pages contain the account name; its location follows XDG_CACHE_HOME or LOCALAPPDATA and can be set with EQUASIS_CACHE_DIR - --no-cache and --refresh control caching on the command line, and /refresh does the same in the interactive shell - equasis cache [info|clear] and the shell's cache command manage it - Results record when their pages were retrieved; table output shows it and JSON uses it for retrieved_at - Later fleet pages are only served after page 1 in the same Equasis session, so page 1 is reloaded when it came from the cache but the following pages did not Fixes #2
File systems can record a modification time a few microseconds after the in-memory timestamp, which made the comparison fail on Windows.
Vessel indicators and a page cache
Timestamps inside JSON results now use the same UTC 'Z' form as the document envelope, and the interactive usage lines list /refresh.
- README: concise overview, installation with pipx, getting started, usage for each command and the interactive shell, output formats and exit codes, the Python API, responsible use, and troubleshooting - docs/usage.md: reference for every command, option, shell command, keyboard shortcut, and environment variable - docs/output-schema.md: JSON, JSON Lines, and CSV layouts, indicator definitions, and the stability policy for machine-readable output - docs/development.md: setup, checks, live tests, architecture, and how to record anonymized fixtures - CHANGELOG.md following Keep a Changelog, with upgrade notes for 3.0.0
- CONTRIBUTING.md covers reporting problems without exposing credentials, development setup, pull request expectations, and the contributor license agreement that keeps dual licensing possible - SECURITY.md describes private vulnerability reporting - Contributor Covenant 2.1 code of conduct - Issue forms for bugs, Equasis page changes, and feature requests, and a pull request checklist - pre-commit no longer rewrites recorded test fixtures
The zsh completion covered only the v1 commands and offered options in positions the parser rejected. Both scripts now cover every command and option, and a test fails when the parser and the scripts drift apart.
The release workflow builds once, checks that the tag matches the package version, and publishes with PyPI trusted publishing: manual runs go to TestPyPI and published GitHub releases go to PyPI after approval. RELEASING.md walks through the one-time account setup and each release.
Documentation, community files, and the 3.0.0 release workflow
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#20, #21, #22, and #23 were merged as a stack, so they landed on
chore/foundation(the branch of #19) instead ofmain. This pull request brings that branch intomain. It contains no changes beyond those four reviewed pull requests, and the resulting tree is identical to the tip that passed CI in #23.Fixes #16
Fixes #17
Fixes #18
Fixes #2