Skip to content

Refactor/structured concurrency v1 - #1524

Open
burnpanck wants to merge 33 commits into
PyLabRobot:mainfrom
burnpanck:refactor/structured-concurrency-v1
Open

burnpanck wants to merge 33 commits into
PyLabRobot:mainfrom
burnpanck:refactor/structured-concurrency-v1

Conversation

@burnpanck

Copy link
Copy Markdown
Contributor

Pull Request: Structured concurrency and scoped lifespans (HasLifespan)

This PR sharpens PyLabRobot's asynchronous execution semantics across device lifespans, background task ownership, cancellation and timeout boundaries, I/O framing, and temporary physical states (such as shaking or freedrive) by migrating the codebase to AnyIO structured concurrency (anyio>=4.7). Previously, imperative setup()/stop() calls and detached asyncio tasks/threads left cleanup, partial-initialization rollback, and mid-command cancellation under-specified—allowing failed startups to leak handles, timeouts to abandon in-flight hardware writes, and background readers to outlive their transports. Binding every resource and worker to a lexical scope (async with) makes hardware teardown, error recovery, and cancellation deterministic across both asyncio and trio.

TL;DR — Key Semantic Changes

  1. Lexical device & transport lifespans (async with): Every resource-owning driver, legacy Machine, Visualizer, and pylabrobot.io transport inherits from HasLifespan. Entering async with device: guarantees the device is initialized on entry and inert after exit; failed partial entry or body exceptions unwind all acquired resources in reverse order.
  2. Shielded safety teardown & exception chaining: Cleanup actions registered via AsyncExitStackWithShielding are all attempted in reverse order even if an earlier step fails, and async safety actions (e.g., stopping heaters/shakers/pumps before closing the transport) run under cancellation shielding. Cleanup failures propagate to the caller with any body/cancellation error preserved on __context__.
  3. Interactive setup() / stop() compatibility bridge: await device.setup() and await device.stop() remain available as a deprecated, asyncio-only bridge for cross-cell notebook sessions (backed by an application-global manager task), but are forbidden inside library/driver code.
  4. Level-triggered cancellation & commit boundaries: Deadlines use with anyio.fail_after(timeout):. Cancellation before a command is dispatched aborts cleanly; once a hardware write or stateful SDK call is dispatched, the driver/transport reconciles the finite reply or abort path before propagating cancellation so hardware is never left mid-transaction.
  5. Scoped temporary operations (shaking, freedrive, objective_maintenance, acquisition): Long-lived physical states expose nested async with context managers inside an active device lifespan that reject conflicting imperative calls while active and guarantee shielded stop on scope exit.
  6. Single ownership vs. borrowing (_enter_lifespan): Composite drivers implement _enter_lifespan(self, stack, **kwargs) to enter owned transports and task groups (await stack.enter_task_group()) while leaving borrowed shared interfaces (InhecoTECControlBox, VSpin, injected HTTP) untouched on teardown. Overriding setup, stop, __aenter__, or __aexit__ is rejected at class creation.
  7. Sharpened I/O stream, boundary & connection contracts (pylabrobot.io): Transports distinguish per-call lifespans (IOBase), uninterrupted handles (ContinuousConnectionIO: Serial, HID), and explicit mid-lifespan reconnection (ReconnectableConnectionIO: USB, FTDI, Socket). read(size) has true byte-stream semantics (EndOfStream on EOF, unreturned bytes retained on cancellation), while read_to_boundary() / discard_to_boundary() handle packet- and report-framed protocols.
  8. Dual-backend test harness (AnyioTestBase & MockIO): Async unit tests inherit from AnyioTestBase (running each test in a fresh lifespan under both asyncio and trio) and use loop-agnostic MockIO doubles instead of unittest.IsolatedAsyncioTestCase and AsyncMock.

Detailed User & Developer Reference

User-Facing Details

  • Using devices in a scope:
    async with lh, plate_reader:
      await lh.move_plate(plate, plate_reader)
      async with plate_reader.shaking(speed=500):
        await anyio.sleep(10)
      data = await plate_reader.read_luminescence(focal_height=0)
  • Non-reentrant lifespans: Entering an already-active HasLifespan instance concurrently or recursively raises RuntimeError.
  • Scoped operation inventory:
    • shaking(...): Legacy Shaker / HeaterShaker, Incubator (Cytomat, Liconic, InhecoIncubatorShakerStack), BigBearOrbitalShaker, BioShake, InhecoThermoShake, BioTek plate readers (Cytation1, Cytation5, SynergyH1 + legacy backends), and SpectraMax plate readers (v1 + legacy).
    • freedrive(...): PreciseFlex, XArm6, and legacy ExperimentalSCARA.
    • objective_maintenance(...): ImageXpress Pico (v1 and legacy backend).
    • acquisition(): AravisCamera / CytationMicroscope and legacy CytationBackend / Cytation5Backend.
  • EventBus integration: HasLifespan emits correlated .setup.started / .completed / .failed and .stop.started / .completed / .failed events via _lifespan_event_info(). Failed setup (including rollback of partial entry) emits only setup.failed; stop.* events fire only after setup has succeeded.

Developer-Facing Details

  • Writing a driver lifespan (pylabrobot/concurrency.py):
    class MyDriver(HasLifespan):
      async def _enter_lifespan(self, stack: AsyncExitStackWithShielding) -> None:
        await super()._enter_lifespan(stack)
        await stack.enter_async_context(self._io.lifespan())
        tg = await stack.enter_task_group()
        stack.callback(tg.cancel_scope.cancel)
        tg.start_soon(self._reader_loop)
        stack.push_shielded_async_callback(self._safe_park, shield_timeout=5.0)
        await self._initialize()
  • Additional concurrency primitives:
    • non_cancelling_task_group(): runs finite concurrent child tasks without cancelling siblings on ordinary child errors, raising SiblingTaskErrors(ExceptionGroup) after all children finish.
    • attempt_in_order(*actions): runs a sequence of cleanup callables, attempting all of them even if earlier ones fail and chaining exceptions via __context__.
  • Mid-lifespan reconnection: Drivers that replace an owned transport while remaining active (BenchCel, HighResSampleStorage, ExperimentalTecanInfinite200ProBackend) keep a reusable nested AsyncExitStackWithShielding in _enter_lifespan and close/re-enter that nested stack on reconnect.
  • Blocking SDK & transport calls: Blocking calls run in non-abandoning anyio.to_thread.run_sync workers (or a dedicated single-thread executor in HID to preserve HIDAPI thread affinity) so cancellation waits for in-flight native calls to complete before closing handles.
  • Contributor guides & migration skill: See docs/contributor_guide/structured-concurrency.md, docs/contributor_guide/structured-concurrency-gotchas.md, and .agents/skills/convert-backend-structured-concurrency/SKILL.md.

…t infrastructure

Introduce HasLifespan, AsyncExitStackWithShielding, NonCancellingTaskGroup,
attempt_in_order, and the deprecated asyncio GlobalManager compatibility bridge
in pylabrobot.concurrency, along with LifespanEventInfo and lifespan_events in
pylabrobot.events.

Change-Id: Ic8a5b6df9dc3806231dd6ffce332c9dc98bc2d33
… skill, and README updates

Add structured-concurrency.md, structured-concurrency-gotchas.md, and the
convert-backend-structured-concurrency agent skill; update contributing.md,
device-driver-guide.md, event-bus.md, event-schemas.md, and README.md.

Change-Id: Ic708c6ddc364fe03ebbfb4ba977d235866061923
…ncurrency

Add loop-agnostic AnyioTestBase (running tests under both asyncio and Trio with
per-test class lifespans and @lifespan_kwargs) and MockIO asynchronous read/write
test doubles.

Change-Id: I0a5c46fd6a6adf4099f296c90de037c0e500294b
…ed lifespans

Migrate IOBase, ContinuousConnectionIO, ReconnectableConnectionIO, boundary
framing, Serial, FTDI, USB, HID, Socket, HTTP, CommandLineTransport, and SiLA
discovery to AnyIO structured lifespans and non-abandoning workers.

Change-Id: I32de5fb096a7bba05acce860336348ce58f85ada
Scope the Visualizer websocket server, HTTP file server, resource callbacks,
and bounded outbox worker within a single asyncio-guarded HasLifespan.

Change-Id: I313c3c44339ad6518a136d802edc93cafedc4a3c
…, and Prep driver

Migrate Hamilton USB and TCP transports, text protocol ReplyRouter/USBReplyReader,
and the Hamilton Prep driver and simulator to structured lifespans.

Change-Id: I9d9f2c1bdc873d84be14535a37319504e9c4e33a
Migrate the Hamilton STAR driver, module firmware exclusion, and device/simulator
tests to structured lifespans.

Change-Id: I4c931c81ba328a53b9830d3152685cf24b9a9a1a
…tured lifespans

Migrate OpentronsRun, OT-2, and Flex drivers and their lifecycle tests to
structured lifespans.

Change-Id: Iba8fda4d14986aa108101c747b84094550bdd2a3
…tation microscope

Migrate BioTekPlateReaderDriver, SynergyH1, EL406, Cytation1, Cytation5,
CytationMicroscope, and AravisCamera (including shaking() and acquisition()
operation scopes) to structured lifespans.

Change-Id: I7d99d53c9fd9bca1c0f8b8d25a037944409e9ccc
…vers

Migrate Agilent BenchCel, PlateLoc, VSpin, and Access2 drivers to structured
lifespans.

Change-Id: I54181adc0aa5fd0dbdd5e22d67279fe8d5b7db81
Migrate Revvity Celigo driver, LumeneraCamera, and lifecycle/protocol tests to
structured lifespans.

Change-Id: Iec345763caa508465ccacffbcbe2a843b2a22e51
…nd NanoDrop drivers

Migrate Byonoy Absorbance96/Luminescence96, Molecular Devices SpectraMax and
ImageXpress Pico (including shaking() and objective_maintenance() scopes),
LI-COR Odyssey, and Thermo Fisher NanoDrop 1000 drivers to structured lifespans.

Change-Id: Iec610363a377a050ea9e9a681a8b09b924d58599
…SCILA drivers

Migrate InhecoTECControlBox, InhecoCPAC, InhecoThermoShake (including shaking()
scope), SCILA, and the Inheco SiLA transport to structured lifespans.

Change-Id: Ia690cfffb547350e14a2c13fc64558b67a1a80ff
Migrate Thermo Scientific ALPS 300/3000/5000 plate sealers and BTX Gemini X2
electroporator drivers to structured lifespans.

Change-Id: I46e517c213348f647ae4462120af34aa8d0fbc6a
…o structured lifespans

Migrate Brooks PreciseFlex and UFactory XArm6 (including freedrive() scopes)
along with HighRes LidValet and SampleStorage drivers to structured lifespans.

Change-Id: Ib0a5dee267fc538d5fe51e883044e4faf92e2a99
…Bioscience drivers

Migrate Azenta A4S, XPeel, and FluidX IntelliXcap96, KBiosystems Ultraseal
EPRO/PRO/XTPro, and KBioscience KUBE drivers to structured lifespans.

Change-Id: I6559957fa9d2d40fee732e40cfa7c035dc25cad0
…er, Agrow, and Curiox drivers

Migrate BigBearOrbitalShaker and QInstruments BioShake (including shaking()
scopes), Cole-Parmer GenoGrinder and Masterflex, AgrowPumpArray, and Curiox
HT2000 drivers to structured lifespans.

Change-Id: I457072f18eaece693e6961a4ee3cfb1766cfb8ab
…, Micronic, and generic scanners

Migrate Mettler Toledo and Sartorius Entris2 scales along with generic serial,
Keyence, and Micronic code reader barcode scanners to structured lifespans.

Change-Id: Icde5e284b333aff7c3b2c3752cb32fb176ad994e
…asses and tip tracker

Migrate legacy Machine and MachineBackend to HasLifespan, add the pytest
collection audit for concrete MachineBackend subclasses, and restore mounted
tip volume and callback state in TipTracker.load_state.

Change-Id: Ic4e4566daad5941f78048bfc98eebcef09d3af04
…generic backends

Migrate legacy LiquidHandler, LiquidHandlerBackend, LiquidHandlerChatterboxBackend,
and SerializingBackend to structured lifespans.

Change-Id: I7aa2563d2f4258a9930928e5df6587d4386a13e8
…ckends

Migrate HamiltonLiquidHandler, STARBackend, STARChatterboxBackend, VantageBackend,
NimbusBackend, and HamiltonTCPBackend to structured lifespans.

Change-Id: I430a546e4e1ab831de7355335b1e64efa8c8da77
…EVO backends

Migrate legacy Opentrons OT-2 HTTP/chatterbox/simulator backends and Tecan EVO
backends to structured lifespans.

Change-Id: I0c07bcc025d653104f604ebac6d428c1f3e88d5b
…Tek reader and washer backends

Migrate legacy PlateReader and Imager frontends, Agilent BioTek Cytation and
SynergyH1 backends (including shaking() and acquisition() scopes), and BioTek
EL406 plate washing backends to structured lifespans.

Change-Id: I059b86c5cca14b414398b97b93a22d59c2b7d7f9
…es, BMG, and Byonoy backends

Migrate legacy Tecan Infinite/Spark20M, Molecular Devices SpectraMax and Pico
microscope, BMG CLARIOstar, and Byonoy plate reader backends to structured
lifespans.

Change-Id: If25c432c0181ff0cca6b601e79c72900fe1e9a10
…at, Inheco, and Liconic backends

Migrate legacy Incubator, IncubatorBackend, Heraeus Cytomat, Inheco incubator
shaker stack and SCILA, and Liconic backends (including shaking() scopes) to
structured lifespans.

Change-Id: I9187e0a2b860544e09468edaf8f9bd47034f210d
…e-controlling subsystems

Migrate legacy Thermocycler and TemperatureController frontends and Inheco,
Opentrons, and Thermo Fisher ATC/ProFlex backends to structured lifespans.

Change-Id: Ibdbeee1231ade87c5545fa0c30ccb78b03657c75
…Fan, and Tilter subsystems

Migrate legacy Shaker, HeaterShaker (including shaking() scopes), Fan, and
Tilter frontends and Hamilton, BioShake, and Inheco backends to structured
lifespans.

Change-Id: I56dfac3f1e5ec19780819c81b5e9cba75721bca8
…ers, sealers, and peelers

Migrate legacy ExperimentalSCARA (including freedrive() scope) and PreciseFlex
arm backends, VSpin/Access2/MicroSpin centrifuge backends, Keyence barcode
scanner, A4S sealer, and XPeel peeler backends to structured lifespans.

Change-Id: I3ded3f1fc4f688662e5984096b2bc180f2294b11
…rs, and scales

Migrate legacy Pump and PumpArray frontends, Masterflex and Agrow pump backends,
PowderDispenser and Chemspeed backends, and Mettler Toledo scale backends to
structured lifespans.

Change-Id: Icf074b66ded8d5d42a6232e94cee3e533ec51b9b
…red concurrency

Migrate user guide, cookbook, and resource notebooks to async with lifespans
(or explicitly labelled cross-cell compatibility sessions) and add the
docs/notebook_lifecycle_tests.py AST/JSON validation suite.

Change-Id: I0127989d4be486768654d5108b9f5b68a1ecc4e6
Bring upstream's additions onto the structured-concurrency lifecycle:

- rpc and Celigo gRPC: DeviceServer and DeviceClient are HasLifespan,
  limited to asyncio (grpc.aio) and rejecting other backends before
  acquisition; Job.wait uses anyio.fail_after. CeligoSession owns the
  hardware lock, job store, accepted commands, job executions, and driver
  lifespan in one task group; CeligoServer enters the session before
  accepting RPCs, the CLI uses async with and anyio.run, and scan timing
  uses anyio.current_time(). structured-concurrency.md lists the
  asyncio-only service boundaries.
- BMG and Viewer3D: QueuedReads is a lifespan whose libusb event loop runs
  in a non-abandoning worker owned by its task group. Viewer3D serves
  static files on connections accepted on the event loop and answered by
  workers owned by the lifespan's task group.
- PreciseFlex: connect()/disconnect() entry and exit paths are removed.
  PreciseFlexDevice is HasLifespan and delegates lifespan entry (including
  skip_home) to its driver. PreciseFlexSimulationDriver substitutes an
  in-memory lifespan without opening a socket, registers shielded teardown
  callbacks, opens jaws on simulated homing, and supports scoped
  freedrive() and simulated GraspPlate/ReleasePlate guards with AnyIO
  checkpoints. Jaw-clearance guards, force-limited GraspPlate/ReleasePlate
  and MoveOneAxis jaw moves, and post-move joint-state reads compose with
  motion admission, freedrive scope checks, evented operation signatures,
  shielded motion-profile restoration, and post-commit halt-and-resync on
  non-blocking motion cancellation.
- Hamilton Prep and STAR: Prep exposes _enter_link() / _link() for the
  firmware type recorder. STAR pipettes shields post-failure tip sensing,
  channel Z recovery, state recording, batch safe-Z retraction, and
  temporary drive-profile restoration inside CancelScope(shield=True)
  without swallowing BaseException.
- Legacy backends and tests: HeaterShakerChatterboxBackend, Incubator,
  Shaker, and EVOBackend lifespan and timeout signatures align with
  structured concurrency; PreciseFlex, STAR, SCILA, ODTC, Keyence, and
  PlateLoc suites run under AnyioTestBase across asyncio and Trio.
@burnpanck
burnpanck force-pushed the refactor/structured-concurrency-v1 branch from 2eb2f6a to 4169e87 Compare October 31, 2026 14:12
…es, and collect PreciseFlex tests

- Keep TCPSession's background reader alive inside a shielded CancelScope until LIFO teardown callbacks registered after the session have completed (matching ReplyRouter.lifespan), so Prep safety teardown commands receive replies when the owning scope is cancelled.
- Shield the finally blocks that restore volatile drive speed/acceleration registers in STARBackend (head96_move_y, head96_move_stop_disk_z, slow_iswap) and Head (move_to_y_position, move_stop_disc_to_z_position).
- Rename PreciseFlexBackendTests(AnyioTestBase) to TestPreciseFlexBackend so pytest collects its 170 test methods.
- Check for '\r' in the accumulated buffer in legacy KeyenceBarcodeScannerBackend.send_command (matching KeyenceBarcodeScanner).

Change-Id: I4419ed5f62e4d5281b0cf005755a442839775c8a
…factor VANTAstar, KingFisher Presto, PreciseFlex, and HID/USB to AnyIO structured concurrency

Change-Id: Ib1315316607b843161e8980dd5305a84b2ea378f

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant