Repository navigation
Do not lose streamed log entries - #164
Merged
Merged
Conversation
Streaming subscribers could miss entries in two ways. Publishing used a fixed 100-entry channel and silently dropped data when it was full, so a large log chunk uploaded by a device (hundreds of lines published in a tight loop) lost its tail. In addition, the stream handler flushed the response headers before subscribing, so entries published between the flush and the subscription were lost even though the client already considered the stream open. Give each subscriber an unbounded queue drained by its own goroutine, and subscribe before the headers are written. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
4 of 7 tasks
milan-zededa
added a commit
to lf-edge/eve
that referenced
this pull request
Nov 12, 2026
Brings the latest Adam patch (lf-edge/adam#164), which stops the log stream from silently losing entries. Previously Adam dropped published entries once a subscriber's 100-entry channel was full, so a large log chunk uploaded by the device (e.g. the hundreds of lines of a kernel panic stack trace) could lose its tail even though it was stored. Adam also flushed the stream response before subscribing, so entries published in between could be missed. Affected tests in the telemetry suite: TestKernelPanicReport and TestDeviceLogs. Both failed intermittently because log lines present in Adam's store never reached the live log watch. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
eriknordmark
pushed a commit
to lf-edge/eve
that referenced
this pull request
Nov 14, 2026
Brings the latest Adam patch (lf-edge/adam#164), which stops the log stream from silently losing entries. Previously Adam dropped published entries once a subscriber's 100-entry channel was full, so a large log chunk uploaded by the device (e.g. the hundreds of lines of a kernel panic stack trace) could lose its tail even though it was stored. Adam also flushed the stream response before subscribing, so entries published in between could be missed. Affected tests in the telemetry suite: TestKernelPanicReport and TestDeviceLogs. Both failed intermittently because log lines present in Adam's store never reached the live log watch. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com> (cherry picked from commit fc84110)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Streaming subscribers of
/admin/device/<uuid>/logs(and the other streamed endpoints) could silently miss entries:stream.publishdelivered into a fixed 100-entry channel and dropped data when it was full. A device uploading a large log chunk (hundreds of lines, e.g. a kernel panic stack trace) makesnewLogsProcesspublish all lines in a tight loop while the HTTP handler flushes one entry at a time, so the tail of the chunk was dropped even though it was written to the store.subscribe. A client that treats the response as "stream is open" could miss entries published between the flush and the subscription.Changes:
publishnever blocks and never drops for live subscribers.Content-typebeforeWriteHeader, otherwise it was a no-op).Found via recurring EVE evetest
TestTelemetrySuitefailures where log lines present in Adam's store were never delivered to a live log watch. Verified with a locally built Adam image.Note: the stream still only carries entries published after subscription; it does not replay what is already stored.