Skip to content

feat(go): response body buffer allocates fresh on every poll #3452

Open

Description

@matanper

Description

The Go client's TCP read path allocates a new []byte for the response body on every RPC. For consumers polling at high rates, this is the largest remaining contributor to allocation churn on the read side.

A previous issue (#3441) addressed the request-path allocations and the 8-byte response status header. This issue covers what remains: the variable-sized response body, which the parsed PolledMessage references via slices in Message.Payload and Message.UserHeaders. Because of that aliasing, the buffer must outlive deserialization, so the transport layer can't recycle it on its own.

Affected area / component

Go SDK

Proposed solution

Add a new method PollMessagesInto that reads the response body into a caller-provided buffer, growing it as needed, and returns the parsed PolledMessage plus the (possibly reallocated) backing slice for reuse on the next call.

func (c *IggyTcpClient) PollMessagesInto(
    ctx context.Context,
    streamId iggcon.Identifier,
    topicId iggcon.Identifier,
    consumer iggcon.Consumer,
    strategy iggcon.PollingStrategy,
    count uint32,
    autoCommit bool,
    partitionId *uint32,
    buf []byte,
) (*iggcon.PolledMessage, []byte, error)

Usage

var buf []byte
for {
    var polled *iggcon.PolledMessage
    polled, buf, err = client.PollMessagesInto(ctx, ..., buf)
    if err != nil { ... }
    // process polled.Messages — Payload/UserHeaders alias buf
}
// buf gets reused next iteration; zero allocations once cap is sufficient.

Alternatives considered

No response

Contribution

  • I'm willing to submit a pull request to implement this feature

Good first issue

  • I think this could be a good first issue for a new contributor

Activity

  1. justinmclean commented on Sep 30, 2026

    @justinmclean
    Member

    @matanper checking in. #3499 closed without merging. Still planning to carry this on? If not I'll unassign it so someone else can pick it up.

  2. matanper commented on Oct 1, 2026

    @matanper
    ContributorAuthor

    @justinmclean you can unassign me.
    There wasn't an agreement on how or whether to implement this feature though, so it will need to be addressed before implementing

  3. p4rtridge commented on Oct 17, 2026

    @p4rtridge

    Hello @justinmclean Could I be assigned this issue? Thanks

  4. justinmclean commented on Oct 25, 2026

    @justinmclean
    Member

    @p4rtridge this one is on hold until the Go maintainers agree on an approach (see the discussion on #3499). You have #4640 open for #2883, so finish that first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    goPull requests that update go code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions