Skip to content

feat(cli): gh-style api fields, --paginate/--slurp, --jq and --template - #21

Merged
purpshell merged 5 commits into
devfrom
feat/api-gh-style
Sep 20, 2026
Merged

purpshell merged 5 commits into
devfrom
feat/api-gh-style

Conversation

@purpshell

@purpshell purpshell commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Summary

Brings polymorfa api closer to gh api:

  • Fields: -f/--raw-field (always string) and typed -F/--field (true/false/null/integers become JSON; @file and @- read file/stdin). Both support name[key]=v nesting and name[]=v appends. GET/HEAD query parameters accept scalars and name[] arrays; nested objects are rejected. Short aliases -X, -H, -i.
  • --paginate (GET only) follows page.nextCursor (or top-level nextCursor) as the cursor query parameter, stops on hasMore: false/null cursor, rejects a repeated cursor, and caps at --max-pages (default 100). --json prints one document {data: [pageBodies], metadata: {..., pages}}; human output streams each page.
  • --slurp combines page bodies into one array before filters.
  • -q/--jq uses the embedded WebAssembly jq 1.8 (jq-wasm@3.0.0-jq-1.8.2, MIT, no dependencies, no native binary). Strings print raw like gh. jq's $ENV is the WASM sandbox env, so host credentials are not exposed (verified).
  • -t/--template: Go-template subset (paths, range/if/else/end, $, json, len, upper, lower, join, string literals, comments, {{-/-}} trimming).

Behavior change: --field was string-only; it is now typed like gh api -F. -f/--raw-field keeps the old string behavior. README documents this.

No overlap with open PRs #10–#16 (none touch src/commands/api.ts or src/api/).

Five-part assessment

  • API — unaffected. Client-side only; pagination follows the documented cursor query parameter and page.nextCursor/hasMore envelope (CursorPageMetadata in the management spec).
  • SDK — unaffected. Uses RawClient.request through the existing executeRawRequest.
  • CLI — updated. New flags, help text and examples, README "Fields" and "Pagination and output filters" sections, one new production dependency (jq-wasm, pinned exactly; npm audit --omit=dev clean).
  • Docs — unaffected. No public CLI page exists in the Mintlify site; README is the CLI reference.
  • Feature releases — unaffected. No flags or server behavior.

Verification

  • npm run verify passes.
  • test/api/fields.test.ts, test/api/template.test.ts (unit), test/commands/api-gh.test.ts (typed body, GET query, multi-page cursor following, per-page jq, slurp jq, template, jq error exit code 2, --slurp/--paginate misuse, --max-pages stop). Existing test/commands/api.test.ts unchanged and passing.
  • Checked the built dist/api/jq.js loads the WASM module outside vitest.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.

@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

  • Ask an admin to enable usage-based reviews

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 89 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

The included review limit has been reached and this organization has disabled usage-based review continuation. Wait for reviews to reset or ask a billing admin to change After included review limits.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 73286ca1-2b1e-4a08-bb3d-1962a0c3dc9a

📥 Commits

Reviewing files that changed from the base of the PR and between cc29c68 and f35f690.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (11)
  • README.md
  • package.json
  • src/api/fields.ts
  • src/api/jq.ts
  • src/api/paginate.ts
  • src/api/request.ts
  • src/api/template.ts
  • src/commands/api.ts
  • test/api/fields.test.ts
  • test/api/template.test.ts
  • test/commands/api-gh.test.ts

Comment @coderabbitai help to get the list of available commands.

@purpshell

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@purpshell

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@purpshell

Copy link
Copy Markdown
Member Author

Reviewed locally because CodeRabbit is rate-limited (it answered @coderabbitai review with "Review rate limited" on the current head). Ran the CodeRabbit CLI against dev plus a separate local code review.

Fixed:

  • --jq failed on a JSON string body. jq-wasm's raw() parses a string argument as JSON text, not as a JSON string value, so a body of "plain text" produced jq: parse error: Invalid numeric literal, surfaced as Invalid --jq filter: … and blamed on the user. Confirmed empirically against the installed jq-wasm. Now serialized with JSON.stringify first.
  • --paginate stopped after one page for bodies with both a page envelope and a top-level nextCursor. nextCursor() picked record.page as the container and never fell back, contradicting its own doc comment. Now checks the envelope, then the top level; an array-valued page no longer counts as an envelope.
  • Template function lookup resolved inherited properties. FUNCTIONS[head] on an object literal meant {{toString}} and {{constructor …}} resolved and were invoked. Restricted to own properties, so they take the unknown function path.
  • {{else if .cond}} silently dropped the condition and rendered the else branch unconditionally, because dispatch used only the first word. Now rejected with a message pointing at nested {{if}}…{{end}}; else if was never documented.

Tests added for all four.

Left unfixed: the action lexer finds }} with a raw indexOf, so a string literal containing }} (e.g. {{join "}}" .tags}}) is truncated and errors. Real divergence from Go's lexer, but a narrow edge case that fails loudly rather than producing wrong output; making the lexer quote-aware is a larger change than this PR warrants.

@coderabbitai

coderabbitai Bot commented Sep 20, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@purpshell Thanks for the detailed local review and the added test coverage.

The four fixes address incorrect behavior and error reporting in the new CLI features. The remaining }} lexer limitation fails loudly. Deferring quote-aware action scanning is reasonable for this PR scope.


✏️ Learnings added
Learnt from: purpshell
URL: https://github.com/polymorfa/cli/pull/21

Timestamp: 2026-09-20T07:05:36.137Z
Learning: In the CLI template parser, function names must be resolved only from own properties of the `FUNCTIONS` object. Inherited names such as `toString` and `constructor` must produce the normal `unknown function` error.

Learnt from: purpshell
URL: https://github.com/polymorfa/cli/pull/21

Timestamp: 2026-09-20T07:05:36.137Z
Learning: In the CLI template parser, `{{else if ...}}` is unsupported. Reject it with guidance to use nested `{{if}}...{{end}}` blocks instead of treating it as an unconditional `else`.

Learnt from: purpshell
URL: https://github.com/polymorfa/cli/pull/21

Timestamp: 2026-09-20T07:05:36.137Z
Learning: In the CLI template lexer, action termination currently uses a raw search for `}}`. A string literal that contains `}}`, such as `{{join "}}" .tags}}`, is not supported and must fail rather than produce an incorrect result.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

-f/--raw-field and typed -F/--field with nesting, arrays and @file,
--paginate with --slurp and --max-pages, embedded jq via jq-wasm,
and a Go-style --template.
Reject prototype keys and object overwrites in fields, stop on any repeated
cursor, and drop a dead stdin guard.
…okups

- jq-wasm parses a string argument as JSON text, so a JSON string body
  failed with a parse error blamed on the user's filter. Serialize first.
- nextCursor never fell back to the top level once a page envelope was
  present, silently stopping after one page.
- Template function lookup resolved inherited names such as toString and
  constructor; restrict it to own properties.
- {{else if .cond}} dropped the condition and rendered unconditionally;
  reject it with a clear message instead.
@purpshell

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

…ile reads

--quiet disabled per-page streaming, so a paginated --jq filter ran once
over an array of page bodies instead of each body, changing the exit code
for filters like .data[].id. Only --json aggregates now. Also stat field
files before reading, so the 10 MiB limit rejects rather than buffers.
@purpshell

Copy link
Copy Markdown
Member Author

Rebased onto dev after #18 and #19 merged, then re-reviewed. CodeRabbit is still rate-limited ("Your included review limit is currently reached under our Fair Usage Limits Policy"), so this was reviewed locally again with the CodeRabbit CLI plus a local code review.

Two further fixes:

  • --quiet changed --jq semantics, not just output. Streaming was gated on mode === 'human', so --paginate --jq .data[].id --quiet skipped per-page processing and ran the filter once over an array of page bodies — which that filter cannot match, so the command exited non-zero while the identical run without --quiet succeeded. Streaming is now disabled only for --json, where aggregating into one document is the point. Test added.
  • readFieldFile buffered the whole file before enforcing its 10 MiB limit, so the limit did not actually bound memory. It now stats first and rejects, keeping the post-read check for the race.

Left unfixed: nothing outstanding from this round. From the earlier round, the action lexer still finds }} with a raw indexOf, so a string literal containing }} (e.g. {{join "}}" .tags}}) is truncated and errors — a narrow edge case that fails loudly rather than producing wrong output.

--paginate only rejected a cursor passed via query fields, so a cursor
already in the endpoint path produced a duplicate parameter. A response
claiming more pages with no cursor also ended pagination successfully,
hiding an incomplete result; it now fails with a clear message. Name the
field flags accurately in the conflicting-body error.
@purpshell

Copy link
Copy Markdown
Member Author

A second local pass turned up three more issues, now fixed:

  • --paginate only guarded a cursor passed via -F cursor=…, not one already in the endpoint path. api "/platform/customers?cursor=abc" --paginate slipped through and sent ?cursor=abc&cursor=n1, which either refetches page 1 or silently drops the caller's starting cursor depending on the server. Now rejected the same way.
  • A response claiming more pages with no cursor ended pagination successfully. {page: {hasMore: true, nextCursor: null}} returned exit 0 and metadata.pages: 1, silently truncating. It now fails with an explicit message rather than under-reporting.
  • The conflicting-body error said "Use either --input or --field" while the check also covers -f/--raw-field, so it named a flag the user had not passed.

Tests added for the first two.

Deliberately left unfixed:

  • --field semantics changed from string-only to typed values with @file expansion, with no alias for the old behaviour: --field caption=@everyone now tries to read a file, and --field projectId=12345 now sends a number. This is the PR's intended gh parity and the README documents it, so reverting it would defeat the change — but it is a breaking change for existing scripts and deserves a changelog entry calling it out explicitly.
  • --paginate --slurp --include accepts --include but drops metadata, so the output matches omitting it.
  • jq-wasm trims captured stdout, so --jq loses leading/trailing whitespace in raw string output.
  • The template lexer closes an action at the first }} regardless of context, so }} inside a string literal or a {{/* … }} … */}} comment errors misleadingly.
  • --max-pages discards already-fetched pages under --json/--slurp but prints them when streaming.

None of these produce silently wrong data; they are documented-behaviour or loud-failure cases better handled as follow-ups than folded into this PR.

@purpshell
purpshell merged commit 6fec968 into dev Sep 20, 2026
3 checks passed
@purpshell
purpshell deleted the feat/api-gh-style branch September 20, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant