[ktz03]

Problem

Assistant chat streams silently discard the first event when a text/event-stream response starts with a UTF-8 byte order mark. The BOM remains attached to the initial data: field, so the parser skips it. This affects both chat and chat_completions, with synchronous and asynchronous clients. A leading BOM on [DONE] also causes the terminator to be skipped.

Solution

Decode the four streaming responses with utf-8-sig before iterating lines. This removes the single initial BOM, including when its bytes arrive in separate chunks, and preserves U+FEFF inside response text. It follows the SSE UTF-8 decoding contract.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update
• Infrastructure change (CI configs, etc)
• Non-code change (docs, etc)
• None of the above: (explain here)

Test Plan

• uv run --frozen pytest tests/unit/test_assistant_stream_encoding.py -q: 32 passed. These cover both client modes and endpoints, BOM split across bytes, Unicode and embedded U+FEFF, comments, empty streams and [DONE]. The same tests on unchanged main produce 12 failures and 20 passing controls.
• A separate real loopback HTTP server exercised the public clients with the same 32 scenarios: all passed with the fix, compared with 12 failures and 20 passing controls on unchanged main. The split-BOM response used HTTP chunked transfer encoding.
• uv run --frozen mypy --strict pinecone/: passed for 202 source files. uv run --frozen ruff check . and uv run --frozen ruff format --check .: passed (717 files formatted).
• uv run --frozen pytest tests/unit --cov --cov-report=term-missing: 9,793 passed, 13 skipped, 8 failed. Unchanged main with the same environment and test scope produced 9,763 passed, 13 skipped, 6 failed. Six failures repeat in the complete main run: two Windows temporary-file sharing cases, three real-sleep timing assertions and the quota-starvation assertion. The other two async-sleep assertions also reproduce in an independent focused run on unchanged main, at approximately 47 ms against a 50 ms threshold. These paths are unchanged. The full suite is not green.

Validation used Python 3.12 on Windows with the frozen dev dependencies and the compiled Rust extension. The full suite used process-local UTF-8 settings after the initial Windows GBK documentation-collection errors. No live Pinecone service, model execution or other platform was tested.

---

Note

Low Risk
Narrow change to SSE line decoding in assistant chat streams; behavior improves for BOM-prefixed responses and should be unchanged for streams without a leading BOM.

Overview
Fixes assistant SSE streaming when text/event-stream responses begin with a UTF-8 byte order mark (BOM). Without BOM-aware decoding, the first data: line (and sometimes [DONE]) failed prefix checks and was dropped, so callers lost the opening chunk.

Sync and async clients now set response.encoding = "utf-8-sig" before line iteration on all four stream helpers (chat and chat_completions in pinecone/client/assistants.py and pinecone/async_client/assistants.py). That strips only the leading BOM (including when split across chunks) while keeping U+FEFF inside message text.

Adds tests/unit/test_assistant_stream_encoding.py with parametrized coverage for both APIs and clients (plain UTF-8, BOM, split BOM, comments, empty streams, and [DONE] variants).

^{Reviewed by Cursor Bugbot for commit 4c20eb3. Bugbot is set up for automated code reviews on this repo. Configure here.}