Skip to content

fix(task_step): sync_env_clock raises StepContextError for a server that can't take the clock the task asks for - #632

Open
andrewpark-scaleai wants to merge 1 commit into
mainfrom
andrew.park/sync-clock-timezone-not-retried
Open

andrewpark-scaleai wants to merge 1 commit into
mainfrom
andrew.park/sync-clock-timezone-not-retried

Conversation

@andrewpark-scaleai

@andrewpark-scaleai andrewpark-scaleai commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Why

From 01:00 UTC on 11-12, every run of three new task families failed at sync-clock, 216 step failures in about 2.5 hours, and each run spent all 3 attempts. Every failure was the step's own check in _sync_one:

RuntimeError: airtable advertises urn:agentenv:clock/v1 but its sync_time takes no timezone; rebuild its image or drop timezone='America/Chicago' from the task

(168 for airtable, with America/Chicago or America/Detroit; 48 for amazon-fresh, all in env openclaw-grace_liu_20261020-multi.)

The tasks pin a timezone, and those servers' images predate sync_time taking one. That has to be fixed in the task or the image. But the check depends only on the server's card, so a retry asks the same card and fails the same way. As a plain RuntimeError, the worker retried it twice per run.

What

Two capability checks in _sync_one now raise StepContextError, which the agent-env worker fails without retrying (scaleapi/scaleapi#174619):

  • a server whose sync_time takes no timezone when the task sets one
  • a server that doesn't advertise clock/v1 when the step doesn't tolerate that

Both are a RuntimeError as before, so callers that catch RuntimeError behave the same. "Has no env card" stays a plain RuntimeError, since its message covers a deploy that couldn't read the card, which can be transient.

Validation

sync_env_clock_test.py now expects StepContextError for both checks. On main its 4 updated cases fail, and with this change they pass. sync_env_clock_test.py and test_step_context_error.py: 49 passed.

RetriggerView in GreptileConfidence Score: 5/5

This PR appears safe to merge; no actionable issues were found.

Summary

Changes two clock-capability failures to StepContextError so the worker can recognize failures that another attempt cannot fix.

  • Clock capability mismatches now stop task retries.

Reviews (1) · Last reviewed commit: "fix(task_step): sync_env_clock raises St..." · Reviewed by Greptile

@andrewpark-scaleai
andrewpark-scaleai requested a review from a team as a code owner November 12, 2026 03:42
@andrewpark-scaleai
andrewpark-scaleai enabled auto-merge (squash) November 12, 2026 03:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant