Skip to content

feat(voice): audio library and provider credentials (Voice Automation beta) - #279

Merged
purpshell merged 15 commits into
devfrom
t3code/voice-audio-library
Sep 23, 2026
Merged

purpshell merged 15 commits into
devfrom
t3code/voice-audio-library

Conversation

@purpshell

@purpshell purpshell commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Voice audio library

Adds the TypeScript Client.voice.audio and Client.voice.providerCredentials resources for the Voice Automation beta. The SDK can create and complete audio uploads, synthesize speech, list and manage assets, get preview URLs, wait for processing, and manage write-only provider credentials. voice.asset_ready and voice.asset_failed webhooks are typed. The upload helper sends bytes only to the returned URL, strips Polymorfa credentials, refuses redirects and non-loopback HTTP, and keeps URL and provider-key material out of errors, including network-error causes.

The SDK contract is pinned to merged API dev commit 087d0e34b53eec82ebc5d04c5b4c75eaaa556b4f (API #227). The Messaging and Platform OpenAPI snapshots match that commit byte for byte. Strict coverage reconciles 485 operations: 361 covered, 124 explicitly excluded, zero missing or changed. The branch includes the SDK dev baseline through eaa7f67a6dd24e67fea4a1a22fc5a85fad39deb9, so Voice coexists with Calls analytics, usage, and gates.

constructWebhookEvent also accepts project-scoped deliveries with an empty session. Those envelopes are emitted for customer.* events and the new voice.* events.

Verification

Node 22 local format, lint (zero errors; two existing unused-disable warnings), typecheck, all workspace builds, 1,205 tests in 110 files, strict coverage, Functions and retired-name checks, workspace pack dry run, and production dependency audit (zero findings) pass. Voice tests cover method routes and bodies, project confinement, upload credential isolation and error paths, provider-key redaction, unknown enums, polling states, and project-scoped webhook parsing. The exact API snapshots have matching SHA-256 hashes. CI and current-head review are pending. A live API, provider upload, and real call were not exercised.

Contract and release assessment

Part Status
API Merged predecessor. API #227 owns the Voice handlers, scopes, authoritative specs, feature gate, and runtime. This SDK PR adds no API handler or wire change.
SDKs Updated. TypeScript Voice resources, types, webhook/error parsing, examples, and contract ledger are included. Other language SDKs are outside this repository's supported language set.
CLI Blocked on exact SDK publication. Existing CLI #25 must pin the published dev package from this PR, validate its commands, and become review-ready.
Docs Updated. SDK README, package README, examples and changelog describe the beta contract. API #227 carries the public API references and Calls guide.
Feature releases Availability unchanged. calls.voice-automation remains a gated beta requiring enrollment and server authority. Merging this PR only publishes a development SDK package; it does not enable an audience or establish deployed availability. No rollout or production flag changes are made.
Admin Unaffected. This package contains no Admin UI, capability, or worker change.

Calls remains unreleased. Cloud/VoIP immutable project ownership is a separate Calls release-acceptance gap; no production release or deployment is part of this PR.

@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

  • Ask an admin to enable usage-based reviews

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Next included review available in 49 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 72 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 25ae5b64-031a-4015-8ca0-43358e4f507e

📥 Commits

Reviewing files that changed from the base of the PR and between eaa7f67 and 6010302.

📒 Files selected for processing (24)
  • CHANGELOG.md
  • README.md
  • contracts/README.md
  • contracts/coverage.json
  • contracts/openapi.messaging.json
  • contracts/openapi.platform.json
  • contracts/source.json
  • examples/voice-audio/README.md
  • examples/voice-audio/index.ts
  • examples/voice-audio/package.json
  • packages/typescript/README.md
  • packages/typescript/src/client.ts
  • packages/typescript/src/errors.ts
  • packages/typescript/src/index.ts
  • packages/typescript/src/platform/voice.ts
  • packages/typescript/src/transport/http.ts
  • packages/typescript/src/webhooks/events.ts
  • packages/typescript/src/webhooks/index.ts
  • packages/typescript/src/webhooks/verify.ts
  • packages/typescript/test/coverage-reconciliation.test.ts
  • packages/typescript/test/coverage.test.ts
  • packages/typescript/test/voice-contract.test.ts
  • packages/typescript/test/voice.test.ts
  • packages/typescript/test/webhook-contract.test.ts

Comment @coderabbitai help to get the list of available commands.

Format the voice resources, upload transport and tests, and drop lint
directives the open enum types do not need. Tests cover every method, the
upload helper sending no Authorization header, the new error codes, the
write-only provider key and unknown enum values.
Add the Voice audio library section to the SDK README, list Client.voice in
the root README, add the unreleased changelog entry, and record in the
contracts README that voice-audio-v1 is typed ahead of the pinned snapshots.
Upload a recording, store an ElevenLabs key, render text-to-speech, preview
and list the library of one project.
@purpshell

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-23T14:04:40.518230Z 6010302 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f0f3c0719c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

...(wait.signal === undefined ? {} : { signal: wait.signal }),
};
while (true) {
const response = await this.retrieve(assetId, read);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enforce the polling deadline during retrieval

When a status request takes longer than wait.timeoutMs, this await remains governed by the independent request timeout (30 seconds by default), and a terminal response is returned before the elapsed deadline is checked. Consequently, waitUntilReady(..., { timeoutMs: 1000 }) can block far beyond one second and even succeed after the promised timeout; cap each retrieval by the remaining time and check the deadline after it completes.

Useful? React with 👍 / 👎.

value.byteOffset,
value.byteLength,
).slice();
return { body: bytes.buffer, sizeBytes: input.sizeBytes ?? bytes.length };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Derive byte-array upload sizes from the body

For a Buffer or other typed-array body, the actual byte length is already known, but this line prefers an optional caller-supplied sizeBytes. If that value is stale—for example, it is the size of a parent buffer while body is a slice—the asset is created with one length and different bytes are uploaded, which can make storage reject the upload or leave an unusable pending asset. Ignore sizeBytes for known-size bodies or reject values that differ from bytes.length.

Useful? React with 👍 / 👎.

… sizes from byte bodies

waitUntilReady now bounds every status read, including transport retries,
by the time left before timeoutMs, combined with the caller's signals, and
checks the deadline after each read, so a result arriving after the deadline
raises PolymorfaTimeoutError (request_timeout) instead of resolving late.

upload takes the size of Buffer, typed-array, ArrayBuffer and Blob bodies
from their byte length and refuses a mismatched sizeBytes with a
PolymorfaValidationError before any request, so no asset is created.
sizeBytes stays required for streams.
…e changelog

Calls and voice are unreleased; the entry is kept as a draft in the
monorepo release record instead.
@purpshell

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: dec8d54b02

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Record all 13 existing voice methods and Console exclusions against exact pending API b361dab snapshots. Replace fake pending error/event additions with real schema parity, including required and nullable voice fields.

Narrow request enums to accepted API values while preserving documented response handling. Reject unsupported Blob MIME types before creating assets and type preview audio as Ogg. Preserve the bounded polling and exact upload-size fixes.

Node 20 and 22 typechecks and builds pass. Final full suites each pass 1112 tests with only the two known SDK #282 contract failures; final source repin after API #227 merge remains required before publication.
…rary

# Conflicts:
#	CHANGELOG.md
#	README.md
#	contracts/README.md
#	contracts/coverage.json
#	contracts/openapi.platform.json
#	contracts/source.json
#	packages/typescript/test/coverage-reconciliation.test.ts
#	packages/typescript/test/coverage.test.ts
…rary

# Conflicts:
#	CHANGELOG.md
#	README.md
#	contracts/README.md
#	contracts/coverage.json
#	contracts/openapi.platform.json
#	contracts/source.json
#	packages/typescript/README.md
#	packages/typescript/src/client.ts
#	packages/typescript/test/coverage-reconciliation.test.ts
#	packages/typescript/test/coverage.test.ts
#	packages/typescript/test/webhook-contract.test.ts
…rary

# Conflicts:
#	CHANGELOG.md
#	README.md
#	contracts/README.md
#	contracts/coverage.json
#	contracts/openapi.messaging.json
#	contracts/openapi.platform.json
#	contracts/source.json
#	packages/typescript/README.md
#	packages/typescript/src/client.ts
#	packages/typescript/src/transport/http.ts
#	packages/typescript/src/webhooks/events.ts
#	packages/typescript/test/coverage-reconciliation.test.ts
#	packages/typescript/test/coverage.test.ts
@purpshell
purpshell marked this pull request as ready for review September 23, 2026 13:53
@purpshell

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6010302048

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

if (left <= 0) throw timedOut();
const pause = linkSignals(callerSignals);
try {
await defaultSleep(Math.min(intervalMs, left), pause.signal);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize aborts during polling delays

When either caller signal is aborted after a transcoding response while this interval sleep is active, defaultSleep rejects directly with signal.reason, so waitUntilReady exposes a raw AbortError or caller-supplied value rather than the PolymorfaCancelledError with code: "request_cancelled" produced when the same abort occurs during retrieval. Catch and normalize cancellation around the sleep as well so cancellation behavior does not depend on which phase of polling receives the signal.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6010302048

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

if (left <= 0) throw timedOut();
const pause = linkSignals(callerSignals);
try {
await defaultSleep(Math.min(intervalMs, left), pause.signal);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize aborts between polling reads

When an asset remains pending and either caller signal aborts during this interval, defaultSleep rejects directly with the signal's reason (normally a raw DOMException). An abort during the preceding retrieve is instead translated by HttpTransport into PolymorfaCancelledError with code: "request_cancelled", so callers receive different error types depending only on whether cancellation lands during a request or the pause. Catch the sleep rejection and normalize caller aborts to the SDK cancellation error.

Useful? React with 👍 / 👎.

@purpshell

Copy link
Copy Markdown
Member Author

Merge gate for exact head 60103020482a1d9922ac3881964896c4437eff99 (base dev): current-head Node 20 and Node 22 CI passed, and the current-head GitHub Codex review completed clean. Local Node 22 verification passed 1,205 tests in 110 files, workspace builds, strict 485-operation contract coverage with zero missing/changed operations, package dry run, and production dependency audit. A local Codex finding about a signed upload URL retained in error causes was fixed and covered by a regression before this head. CodeRabbit reported “Review rate limited”; its green status is not a clean review. The owner explicitly approved merging SDK #279 with CI + Codex and recording the missing CodeRabbit review. This exception applies to this PR/head only. Calls remains unreleased; no audience, deployment, or production flag is enabled by this merge.

@purpshell
purpshell merged commit 50e9109 into dev Sep 23, 2026
4 checks passed
@purpshell
purpshell deleted the t3code/voice-audio-library branch October 26, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant