Skip to content

fix(openai): preserve managed request metadata - #306

Open
usnavy13 wants to merge 1 commit into
danny-avila:mainfrom
usnavy13:fix/gpt56-managed-request-followup
Open

fix(openai): preserve managed request metadata#306
usnavy13 wants to merge 1 commit into
danny-avila:mainfrom
usnavy13:fix/gpt56-managed-request-followup

Conversation

@usnavy13

@usnavy13 usnavy13 commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Preserve the managed OpenAI request metadata that LibreChat needs for reliable explicit caching and tier-aware accounting.

  • Forward the server-generated prompt cache key through OpenAI and Azure Chat Completions and Responses requests.
  • Keep the instruction breakpoint and the two latest prior-history breakpoints, within OpenAI's marker limit.
  • Preserve provider-reported service_tier on non-streaming Chat Completions messages.
  • Keep the current user turn unmarked.
  • Cover Chat Completions and Responses rollover behavior.

Why

Live sequential-request testing showed that replacing the prior history marker on every turn could produce another cache write instead of a cache read. LibreChat's generated prompt cache key also needed to be represented in the shared managed-request options so it reaches all four OpenAI/Azure transport paths. Non-streaming Chat Completions dropped the response-level service tier during LangChain conversion, forcing downstream accounting to infer the requested tier instead of using the provider-reported tier.

Change Type

  • Bug fix (non-breaking change which fixes an issue)

Testing

  • npx jest src/llm/openai/managedRequests.test.ts src/llm/openai/llm.spec.ts --runInBand — 71 tests passed.
  • npx tsc --noEmit — passed.
  • npm run build — passed.
  • ESLint on all changed files — passed.
  • git diff --check — passed.

Test Configuration

  • Rebased onto current Agents main at v3.7.1.
  • Node.js 24.x.

Checklist

  • My code adheres to this project's style guidelines
  • I have performed a self-review of my own code
  • My changes do not introduce new warnings
  • I have written tests demonstrating that my changes are effective
  • Local unit tests and build pass with my changes

@usnavy13

usnavy13 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

Marked ready for review as the small follow-up to danny-avila/agents#303.

It fixes provider-tier metadata and cache-breakpoint retention needed by LibreChat model support, independently of native PTC. Because the Agents CI is scoped to PRs targeting main or dev, this stacked PR will receive its full CI after #303 lands and this branch is rebased onto main. It will then be included in the first published Agents version.

@danny-avila
danny-avila changed the base branch from feat/gpt56-responses-state-slim to main July 13, 2026 12:27
@danny-avila
danny-avila force-pushed the fix/gpt56-managed-request-followup branch from e1d4ea6 to 4826739 Compare July 13, 2026 12:28
@danny-avila

Copy link
Copy Markdown
Owner

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4826739647

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/llm/openai/index.ts Outdated
@usnavy13
usnavy13 force-pushed the fix/gpt56-managed-request-followup branch from 4826739 to 05f80e3 Compare July 15, 2026 21:36
@usnavy13

Copy link
Copy Markdown
Contributor Author

Follow-up update: this branch is now rebased directly onto current main (v3.2.65). The valid cache-breakpoint rollover review finding is fixed, with an exact regression test, and I replied on the inline thread.

Local validation on the rebased single-commit head: 59 focused tests, TypeScript, ESLint for the changed files, and the full Agents build all pass.

@danny-avila

Copy link
Copy Markdown
Owner

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 05f80e379f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/llm/openai/index.ts
@usnavy13
usnavy13 force-pushed the fix/gpt56-managed-request-followup branch from 05f80e3 to b399e88 Compare July 21, 2026 21:28
@usnavy13

Copy link
Copy Markdown
Contributor Author

Addressed the latest review in b399e88 after rebasing onto main v3.2.67.

Cache-breakpoint selection now exits as soon as it has the two newest assistant/tool completion candidates. Added a long-history regression confirming the selected breakpoints.

Validation: focused tests, TypeScript, targeted ESLint, and the full production build all pass.

@usnavy13
usnavy13 force-pushed the fix/gpt56-managed-request-followup branch from b399e88 to e117c9b Compare July 24, 2026 17:16
@usnavy13
usnavy13 force-pushed the fix/gpt56-managed-request-followup branch from e117c9b to 30aa46b Compare August 22, 2026 03:09
@usnavy13 usnavy13 changed the title fix(openai): preserve GPT-5.6 managed request metadata fix(openai): preserve managed request metadata Aug 22, 2026
@usnavy13
usnavy13 force-pushed the fix/gpt56-managed-request-followup branch from 30aa46b to bb47b86 Compare August 22, 2026 03:31
@usnavy13
usnavy13 force-pushed the fix/gpt56-managed-request-followup branch from bb47b86 to b5007fe Compare August 24, 2026 20:15
@usnavy13

Copy link
Copy Markdown
Contributor Author

Rebased this focused follow-up onto current Agents main (v3.7.1). The patch is unchanged in substance and remains cleanly mergeable. Revalidated: 71 focused tests, TypeScript, production build, changed-file ESLint, and diff whitespace checks all pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants