Repository navigation
fix(task_step): sync_env_clock raises StepContextError for a server that can't take the clock the task asks for - #632
Open
andrewpark-scaleai wants to merge 1 commit into
Conversation
…hat can't take the clock the task asks for
andrewpark-scaleai
enabled auto-merge (squash)
November 12, 2026 03:42
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
From 01:00 UTC on 11-12, every run of three new task families failed at
sync-clock, 216 step failures in about 2.5 hours, and each run spent all 3 attempts. Every failure was the step's own check in_sync_one:(168 for
airtable, withAmerica/ChicagoorAmerica/Detroit; 48 foramazon-fresh, all in envopenclaw-grace_liu_20261020-multi.)The tasks pin a timezone, and those servers' images predate
sync_timetaking one. That has to be fixed in the task or the image. But the check depends only on the server's card, so a retry asks the same card and fails the same way. As a plainRuntimeError, the worker retried it twice per run.What
Two capability checks in
_sync_onenow raiseStepContextError, which the agent-env worker fails without retrying (scaleapi/scaleapi#174619):sync_timetakes notimezonewhen the task sets oneclock/v1when the step doesn't tolerate thatBoth are a
RuntimeErroras before, so callers that catchRuntimeErrorbehave the same. "Has no env card" stays a plainRuntimeError, since its message covers a deploy that couldn't read the card, which can be transient.Validation
sync_env_clock_test.pynow expectsStepContextErrorfor both checks. On main its 4 updated cases fail, and with this change they pass.sync_env_clock_test.pyandtest_step_context_error.py: 49 passed.

Confidence Score: 5/5
This PR appears safe to merge; no actionable issues were found.Summary
Changes two clock-capability failures to
StepContextErrorso the worker can recognize failures that another attempt cannot fix.Reviews (1) · Last reviewed commit: "fix(task_step): sync_env_clock raises St..." · Reviewed by Greptile