Skip to content

[eas-cli] Ingest realtime incrementally instead of rebuilding full logs on every message - #4293

Open
AHGIJMKLKKZNPJKQR wants to merge 1 commit into
realtime-logsfrom
AHGIJMKLKKZNPJKQR/incremental-realtime-logs
Open

[eas-cli] Ingest realtime incrementally instead of rebuilding full logs on every message#4293
AHGIJMKLKKZNPJKQR wants to merge 1 commit into
realtime-logsfrom
AHGIJMKLKKZNPJKQR/incremental-realtime-logs

Conversation

@AHGIJMKLKKZNPJKQR

@AHGIJMKLKKZNPJKQR AHGIJMKLKKZNPJKQR commented Aug 27, 2026

Copy link
Copy Markdown
Member

Why

Logs displayed by workflow:run --wait rebuild the entire stored logs on every realtime logs message, which hurts the performance when displaying large workflow runs with many MB of logs.

How

  • Updated WorkflowJobLogsState to maintain the current parsed logs and update them in place on realtime logs publications instead of rebuilding them from scratch on every publication.
  • Updated groupLogLinesIntoSteps to take an accumulator argument for the result instead of building it from scratch
  • Updated stepLogTail to slice then map, to prevent iterating over log lines

Separated this out of #4228 because I didn't want to complicate that PR too much. We should merge this into #4228 before merging to main.

Test Plan

CI passes with new tests, workflow:run --wait tested manually


Stack created with GitHub Stacks CLIGive Feedback 💬

@github-actions

Copy link
Copy Markdown

Subscribed to pull request

File Patterns Mentions
packages/eas-cli/** @douglowder

Generated by CodeMention

Warning: The preamble and epilogue options in commentConfiguration are deprecated. Use template instead.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves the performance of workflow:run --wait log rendering by switching workflow job log state management from “rebuild everything on every realtime message” to incremental in-place updates, reducing repeated parsing/grouping work for large log streams.

Changes:

  • Refactored WorkflowJobLogsState to maintain grouped logs incrementally and reveal buffered realtime logs at the appropriate time (catch-up or completion).
  • Updated log parsing/grouping to support an accumulator (groupLogLinesIntoSteps) and tracked step result separately to avoid recomputation.
  • Adjusted rendering/helpers and expanded Jest coverage for the new incremental behavior.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/eas-cli/src/commandUtils/workflow/utils.ts Uses precomputed step result, optimizes tail rendering, and marks completed jobs to reveal buffered realtime logs without rebuilding.
packages/eas-cli/src/commandUtils/workflow/types.ts Introduces StepLogs and extends per-step log state with optional result.
packages/eas-cli/src/commandUtils/workflow/logs/watcher.ts Implements incremental ingestion/deduping of realtime log lines and incremental grouping into a persistent WorkflowLogs map.
packages/eas-cli/src/commandUtils/workflow/logs/parseLogs.ts Adds accumulator support to groupLogLinesIntoSteps and captures first end-marker result per step.
packages/eas-cli/src/commandUtils/workflow/logs/tests/watcher-test.ts Updates/extends tests to validate incremental ingestion, completion reveal, and deduplication.
packages/eas-cli/src/commandUtils/workflow/logs/tests/parseLogs-test.ts Adds tests for step result capture semantics (first end marker wins, undefined before end).
packages/eas-cli/src/commandUtils/workflow/tests/utils-test.ts Expands formatting tests (default tail size, current step selection, skipped handling, non-in-progress behavior, failed-step output).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@AHGIJMKLKKZNPJKQR
AHGIJMKLKKZNPJKQR force-pushed the AHGIJMKLKKZNPJKQR/incremental-realtime-logs branch from 4fec1e7 to d3f0579 Compare August 28, 2026 11:33
@AHGIJMKLKKZNPJKQR
AHGIJMKLKKZNPJKQR force-pushed the AHGIJMKLKKZNPJKQR/incremental-realtime-logs branch 2 times, most recently from 6bac8a1 to 310d0f5 Compare August 28, 2026 12:39
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.72727% with 15 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (realtime-logs@67dfdbb). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...ackages/eas-cli/src/commandUtils/workflow/utils.ts 28.58% 15 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff                @@
##             realtime-logs    #4293   +/-   ##
================================================
  Coverage                 ?   64.74%           
================================================
  Files                    ?     1052           
  Lines                    ?    48694           
  Branches                 ?    10257           
================================================
  Hits                     ?    31523           
  Misses                   ?    17055           
  Partials                 ?      116           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@AHGIJMKLKKZNPJKQR
AHGIJMKLKKZNPJKQR force-pushed the AHGIJMKLKKZNPJKQR/incremental-realtime-logs branch from 310d0f5 to 7318bc9 Compare August 28, 2026 15:47
@github-actions

Copy link
Copy Markdown

❌ It looks like a changelog entry is missing for this PR. Add it manually to CHANGELOG.md.
⏩ If this PR doesn't require a changelog entry, such as if it's an internal change that doesn't affect the user experience, you can add the "no changelog" label to the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants