feat: patrol leg metrics, groupers and name resolution (new EarthRanger patrol schema) by Yun-Wu · Pull Request #859 · wildlife-dynamics/ecoscope

Yun-Wu · 2026-10-21T21:34:11Z

Summary

Makes the new EarthRanger patrol leg fields (team, members, assets, details, pauses) available to patrol metrics, without changing any existing metric at a site that lacks the patrol_schemas flag.

Ingest: get_patrol_observations carries patrol_segment_id and, when present, patrol_team, patrol_members, patrol_assets, patrol_segment_details, patrol_type_details (details as JSON strings); pause legs are skipped. include_pauses is never sent.
Leg frame: new IO tasks get_patrol_segments_from_patrols_df (+ combined-params variant) build one row per non-pause leg with clipped leg_active_seconds, patrol_elapsed_seconds (first leg per patrol) and leg_member_days.
Presets: Active Duration, Elapsed Duration, Leg Count, Member-days added to the patrol summary union; select_track_metrics / select_leg_metrics route the user's list to the trajectory and leg frames and join_summary_tables joins the two summaries, so existing presets and defaults are untouched.
Names: add_patrol_team_names (via the patrol configuration route), add_patrol_roster_names (subjects lookup, include_inactive), add_patrol_type_display_names; the name columns always exist, filled "None" when a site lacks the flag.
Groupers: explode_for_groupers explodes list-valued grouper columns at run time with full credit per member (member-days split per member); apportion_legs_by_spatial_index splits leg metrics across spatial features by share of track length.
Test data: scripts/seed_patrol_legs_mep_dev.py seeds mep-dev (idempotent, non-destructive) and a live test reads it back; example-return parquets for the five new IO tasks.
Docs: docstrings and built-in-tasks.md.

Design doc, including the workflow wiring contract the follow-up workflow PRs implement: https://claude.ai/artifact/CaEvhS5ZPwJbHApTCXde5F

Review fixes

patrol_segments_from_patrols_df returns a typed empty frame, ends open legs at the window end even without truncation, and counts member-days in the time range's timezone
apportion_legs_by_spatial_index passes empty frames through and spreads the patrol-level elapsed seconds by the whole patrol's track share
add_patrol_type_display_from_legs reads the slug from patrol_type__value (the track frame holds the type UUID in patrol_type)
explode_for_groupers splits member-days for patrol_members too; get_patrol_teams tolerates a config without teams; get_patrol_observations includes inactive types and survives an unknown type
StatSummaryParam.blank_if_all_nan replaces the leg_member_days special case

Test plan

pytest tests/platform -m "not io and not smart_io": 1081 passed (statsmodels-dependent trend tests excluded locally).
mypy ecoscope: clean.
Live: pytest tests/platform/tasks/test_io_earthranger.py -k live_patrol_legs -m io against mep-dev passes (3 legs, 9 h active, 8 h / 2 h elapsed, resolved names).

Closes #858

🤖 Generated with Claude Code

codecov-commenter · 2026-10-21T21:35:42Z

Codecov Report

❌ Patch coverage is 99.33110% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.95%. Comparing base (a8f701d) to head (4f715e1).

Files with missing lines	Patch %	Lines
ecoscope/io/earthranger_utils.py	97.95%	1 Missing ⚠️
...oscope/platform/tasks/transformation/_exploding.py	94.44%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #859      +/-   ##
==========================================
+ Coverage   92.74%   92.95%   +0.21%     
==========================================
  Files         126      128       +2     
  Lines        8640     8934     +294     
==========================================
+ Hits         8013     8305     +292     
- Misses        627      629       +2

Flag	Coverage Δ
core	`26.33% <23.41%> (+0.26%)`	⬆️
io	`15.09% <6.02%> (-0.33%)`	⬇️
io-platform	`46.90% <54.51%> (+0.26%)`	⬆️
platform	`75.44% <90.96%> (+0.54%)`	⬆️
wizard	`100.00% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…urations Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

… optional columns Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…e count) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…ith track/leg routing Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…nputs Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…roupers Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…y track share Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…ields Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…ber-days, merge reset summaries Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…es, CI-visible leg tests - get_patrol_observations / patrol_segments_from_patrols_df store the two details objects as JSON strings so the frames write to parquet - leg frame patrol_serial_number is a string, matching the track frame group key - new add_patrol_type_display_names task (+ example-return parquet); leg fixtures carry string serial numbers - pause/missing-leader skip runs before the patrol type lookup - mock-only leg tests move out of io-marked modules so CI runs them Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…name task; seed side effects Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The design doc and plan live in Claude Docs; ignore the local copies. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…t; apportion idempotency guard - add_leg_names_to_tracks(df, legs): copies patrol_team_name / patrol_member_names / patrol_asset_names from the leg frame onto a track frame by patrol_segment_id, "None" where the column or the leg is missing. Names are resolved once (on the legs) because mock-io keys fixtures per task function, so the same io task cannot serve both frames in a mock run. - join_summary_tables(tables: list[AnyDataFrame]): reduce left to right so a groupbykey pair with one side missing still renders. - apportion_legs_by_spatial_index skips a spatial grouper whose level is already on the leg frame (second pass with an overlapping grouper set). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…patrol_segment_id Observations recorded before the leg schema carry no leg id, so no track share can be computed; every leg then lands whole in the "None" spatial group instead of raising KeyError (found by the patrol-effort-table before/after regression on the packaged 2015 fixtures). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…litter, patrol-type display copy by slug Three contracts for the patrol-effort-plot chart workflow: - EncounterRateMetric gains ActiveDurationMetric, ElapsedDurationMetric, LegCountMetric and MemberDaysMetric (title order kept), so the chart config, figures and metrics tasks accept the four leg presets. Consumers' positional uiSchema oneOf lists grow 13 -> 17. - get_chart_mode_category_groupers(config) -> UserDefinedGroupers: the Category breakdown as a one-element ValueGrouper list, so explode_for_groupers can explode a member/asset category before the chart splits series. - add_patrol_type_display_from_legs(df, legs): copies the display names resolved on the leg frame onto tracks/events by slug, keeping the slug in patrol_type__value; unknown slugs and empty legs keep the slug. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Off by default (the table keeps its "None" row). On, legs whose track matches no spatial feature — every leg when the fixes carry no leg id, i.e. no patrol_schemas flag — are dropped instead of forming a "None" group. The chart workflow binds it so a flag-off site's spatial breakdown does not gain a zero-valued "None" series. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

A summary side with no selected metrics (select_leg_metrics over a track-only metric list) carries only the group keys; in the outer join a key present on that side alone became an all-blank row (a leg whose start bucket has no track row, or the "None" spatial view at a flag-off site). Such a table has nothing to contribute, so it is skipped; when every table is key-only the first is kept so an all-key run still renders. Found by patrol-encounter-rate-stats' before/after comparison. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

atmorling

A couple of questions to sanity check (the patrol type slug assumption + the io tags) but otherwise LGTM

atmorling · 2026-10-27T19:16:45Z

    return cast(EventGDF, patrol_events)


+@register(tags=["io"])


This doesn't need an io tag does it? As in this task itself doesn't interact with the ER API

The io tag here also enables a mock data parquet with several patrol legs. We could also replace the existing parquet but the existing tests will be affected. What would you recommend?

I'd like to avoid io tagging tasks that don't need it if possible. Updating the existing mocks seems reasonable - we'll have to redo snapshots on some workflows but that seems worth doing if it means our mock data is changing to reflect the evolution of patrols in the ER API

☝️ Happy for this to be follow up tech-debt

atmorling · 2026-10-27T19:17:15Z

+    return cast(AnyDataFrame, legs)
+
+
+@register(tags=["io"])


Same Q as above

atmorling · 2026-10-27T20:22:08Z

+def add_patrol_type_display_from_legs(
+    df: Annotated[
+        AnyDataFrame,
+        Field(description="Track or event frame whose `patrol_type` holds the type's value (slug).", exclude=True),
+    ],
+    legs: Annotated[
+        AnyDataFrame,
+        Field(description="Leg frame with `patrol_type` as the display name and the slug in `patrol_type__value`."),
+    ],
+) -> Annotated[
+    AnyDataFrame,
+    Field(description="The frame with `patrol_type` as the display name and the slug in `patrol_type__value`."),
+]:


Just a sanity check on the assumption that patrol_type is a slug and not the UUID? IIUC the client returns patrol_type as a UUID

ecoscope/ecoscope/io/earthranger.py

Lines 1199 to 1203 in a8f701d

patrol_type = (

df_pt[df_pt["value"] == patrol_segment.get("patrol_type")].reset_index()["id"][0]

if patrol_segment.get("patrol_type")

else None

)

Good point. fixed inline

atmorling · 2026-10-27T20:24:29Z

+    if isinstance(groupers, AllGrouper):
+        return legs
+
+    out = legs


Do we definitely want to rename here?

atmorling · 2026-10-27T20:26:59Z

+        merged = out.assign(_pos=range(len(out))).merge(shares, on="patrol_segment_id", how="left")
+        merged.index = out.index[merged.pop("_pos")]
+        if drop_unplaced:
+            merged = merged[merged["share"].notna()]
+        out = merged
+        out["share"] = out["share"].fillna(1.0)
+        out[level] = out[level].astype(object).where(out[level].notna(), "None")


Is this safe against leg/out being an empty df?

…-typed inputs - patrol_segments_from_patrols_df returns a typed empty frame (LEG_COLUMNS), takes open_leg_end so in-progress legs are counted when not truncating, and counts member-days in the time range's timezone - apportion_legs_by_spatial_index passes an empty frame through and spreads the patrol-level elapsed seconds over the whole patrol's track share - add_patrol_type_display_from_legs reads the slug from patrol_type__value, which the track frame already carries next to the type UUID in patrol_type - explode_for_groupers splits member-days for patrol_members as well as the names - get_patrol_teams tolerates a config without a teams key - get_patrol_observations includes inactive patrol types and no longer fails on a leg whose type is unknown - StatSummaryParam.blank_if_all_nan replaces the leg_member_days special case Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…t-in-tasks Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…tead of an out alias Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

atmorling

Thanks @Yun-Wu - can we make a tech-debt issue to follow up on the io tags? We probably want to land wildlife-dynamics/wt#217 which will help keep the mock data used by workflow x bleeding into workflow y

Yun-Wu and others added 29 commits October 27, 2026 09:44

docs: design and implementation plan for patrol leg metrics

33120d7

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs(spec): add workflow wiring contract for patrol leg metrics

5a3810b

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(io): carry patrol leg fields onto observations and skip pause legs

6760655

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(io): add patrol leg frame task with clipped active and elapsed d…

aab2252

…urations Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(io): leg member-days at midnight, rename combined_params arg, pin…

2d836ef

… optional columns Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs(plan): record Task 2 rulings (combined_params name, midnight dat…

03cf7d5

…e count) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(analysis): add leg duration, leg count and member-days presets w…

4cfb7e6

…ith track/leg routing Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(io): resolve patrol team and roster display names

53ff9bf

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(io): harden patrol name resolution for empty, unnamed and array i…

eb02f98

…nputs Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(transformation): explode list columns for selected categorical g…

0086765

…roupers Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(transformation): explode ndarray and tuple cells for groupers

86000b9

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(transformation): apportion leg metrics across spatial features b…

aa10483

…y track share Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

test: pin list and dict extra columns through trajectory conversion

a206b0e

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

test: use distinct per-fix values to prove start-fix carry-through

bd236d4

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs: describe patrol leg metrics, groupers and name resolution

cbf0667

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(io): take patrol team names from display/value fields

5d1fe31

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

test: seed mep-dev with patrol legs carrying team, roster and pause f…

0f82940

…ields Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

test: make mep-dev seed repair half-built patrols and guard the server

89c42b8

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

chore: add example-return fixtures for patrol leg IO tasks

83b6ca4

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(analysis): split member-days on member explode, blank all-NaN mem…

3c4e9d2

…ber-days, merge reset summaries Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs: wiring contract for group-key parity, JSON details and display-…

cda383b

…name task; seed side effects Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(typing): satisfy mypy for patrol leg tasks

bcfc43d

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

chore: keep superpowers planning docs out of the repo

ea2326b

The design doc and plan live in Claude Docs; ignore the local copies. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Yun-Wu force-pushed the worktree-patrol-leg-metrics-plan branch from 9d50ee5 to d3d5647 Compare October 27, 2026 16:44

Yun-Wu and others added 2 commits October 27, 2026 09:58

fix: import EarthRangerClient after rebase; drop mep-dev seed script

9225f1a

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

test: pass the API client to the name tasks in the live patrol legs test

d758c9f

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Yun-Wu requested a review from atmorling October 27, 2026 18:45

atmorling reviewed Oct 27, 2026

View reviewed changes

Yun-Wu and others added 4 commits October 27, 2026 17:02

docs: open legs, member-day timezone and elapsed apportioning in buil…

73e6ff6

…t-in-tasks Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

ci: retrigger checks

d1513f8

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

refactor: rebind legs directly in apportion_legs_by_spatial_index ins…

4f715e1

…tead of an out alias Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Yun-Wu requested a review from atmorling October 28, 2026 00:42

atmorling approved these changes Oct 28, 2026

View reviewed changes

Navigation Menu

Repository navigation

feat: patrol leg metrics, groupers and name resolution (new EarthRanger patrol schema) - #859

feat: patrol leg metrics, groupers and name resolution (new EarthRanger patrol schema)#859
Yun-Wu wants to merge 35 commits into
masterfrom
worktree-patrol-leg-metrics-plan

Yun-Wu commented Oct 21, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Oct 21, 2026 •

edited

Loading

Uh oh!

atmorling left a comment

Uh oh!

atmorling Oct 27, 2026

Uh oh!

Yun-Wu Oct 28, 2026

Uh oh!

atmorling Oct 28, 2026

Uh oh!

atmorling Oct 28, 2026

Uh oh!

atmorling Oct 27, 2026

Uh oh!

atmorling Oct 27, 2026

Uh oh!

Yun-Wu Oct 28, 2026

Uh oh!

atmorling Oct 27, 2026

Uh oh!

atmorling Oct 27, 2026

Uh oh!

atmorling left a comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	patrol_type = (
	df_pt[df_pt["value"] == patrol_segment.get("patrol_type")].reset_index()["id"][0]
	if patrol_segment.get("patrol_type")
	else None
	)

Conversation

Yun-Wu commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Review fixes

Test plan

Uh oh!

codecov-commenter commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

atmorling left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

atmorling left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Yun-Wu commented Oct 21, 2026 •

edited

Loading

codecov-commenter commented Oct 21, 2026 •

edited

Loading