Derive item state from the ledger, and fix what the drivers found

instance/skills/dex-run/SKILL.md

@@ -108,7 +108,9 @@ Then do a run now, whatever the host.
 
 8. **Backstop.** `bin/dex enrich status` — any item listed under
    "enrichment newer than digest" is an interrupted previous session:
-   complete its digest → place → wiki steps now.
+   complete its digest → place → wiki steps now. Every item listed is
+   digestible; one still owing a unit is `raw` and never appears there,
+   however long it stays parked.
 
 9. **Health check.** If `wiki/log.md` shows no health check (a `| lint`
    entry) in the past 7 days, run the dex-lint skill

instance/skills/dex-run/references/ingest-item.md

@@ -36,8 +36,10 @@ page or video title, once known). The body is the owner's note **verbatim
 and stays that way** — your interpretive context (what a linked video is,
 how a thread relates) belongs in the digest, never the item body; thread
 context lives in the enrichment via walk-up. After creation, exactly two
-frontmatter fields ever change (`status`, `enrichment:`), both derived
-from disk, both written by the engine.
+frontmatter fields ever change (`status`, `enrichment:`), both derived by
+the engine: the listing from the enrichment directory, the status from the
+ledger — `enriched` only once every unit the item owns has landed (or is
+confirmed gone or deliberately skipped), `raw` while any is still owed.
 
 Then delete the capture file — the capture is preserved in git history and
 its content lives on in the corpus.
@@ -120,7 +122,10 @@ otherwise 2–5 kebab-case candidates. Then
 
 A parked item (waiting/blocked/manual) still exists — provenance and note
 were captured at ingest — but gets no digest or wiki work until its
-sources land. Don't force it; the report tracks it.
+sources land. Don't force it; the run report's **incomplete** section names
+each such item and what it is still owed ("3 of 4 units landed — 1 waiting
+on transcription"), and the item stays `status: raw` until the last unit
+lands.
 
 ## 7. Place (judgment)
 

instance/skills/dex-run/references/schema.md

@@ -45,8 +45,10 @@ media:                                      # media captures: repo-relative path
   files the phone shortcut does), exporter name (e.g. `discord`) for
   backfills. Every source goes through the same pipeline and carries the
   same weight; nothing about a source implies age or importance.
-- `status` / `enrichment` — engine-owned, derived from disk: the pipeline
-  flips `status` to `enriched` and lists the files under `enrichment/<id>/`.
+- `status` / `enrichment` — engine-owned and derived, never hand-edited: the
+  listing from the files under `enrichment/<id>/`, the status from the
+  ledger — `enriched` only once every unit the item owns has landed (or is
+  confirmed gone or deliberately skipped), `raw` while any is still owed.
   After creation these are the ONLY two frontmatter fields that ever change.
 - `urls` is immutable capture provenance. Harvest-promoted URLs are ledger
   entries (`via: harvest`), never frontmatter edits.

src/dex_engine/drivers/audio.py

@@ -0,0 +1,86 @@
+"""The drivers' audio-on-a-page seam: what audio a page carries as its own.
+
+Two drivers read this one signal from opposite ends, and neither owns it:
+the web driver asks it of every page it fetches, to decide whether the page
+in hand is an episode page at all, and the podcast driver asks it again of
+a page already routed, to resolve the enclosure the transcribe drain will
+reach for.
+
+It sits beside :mod:`dex_engine.drivers.transport` and
+:mod:`dex_engine.drivers.gh`, for those modules' reason: a seam several
+drivers share is not itself a driver, and a driver must never import
+another driver. Like them it reaches only for the stdlib — nothing here
+knows about work units or results.
+"""
+
+import html as html_lib
+import re
+from dataclasses import dataclass
+from urllib.parse import urljoin
+
+__all__ = ["AudioEnclosure", "audio_enclosure"]
+
+# The two ways a page carries audio: the og:audio pointer and an <audio>
+# element (its own src, or a nested <source>).
+_OG_AUDIO_RES = (
+    re.compile(
+        r"<meta[^>]+(?:property|name)=[\"']og:audio[\"'][^>]+content=[\"']([^\"'\r\n]+)",
+        re.IGNORECASE,
+    ),
+    re.compile(
+        r"<meta[^>]+content=[\"']([^\"'\r\n]+)[\"'][^>]+(?:property|name)=[\"']og:audio[\"']",
+        re.IGNORECASE,
+    ),
+)
+_AUDIO_ELEMENT_RE = re.compile(
+    r"<audio\b[^>]*>.*?</audio\s*>|<audio\b[^>]*/?>", re.IGNORECASE | re.DOTALL
+)
+_SRC_RE = re.compile(r"\bsrc=[\"']([^\"'\r\n]+)[\"']", re.IGNORECASE)
+
+
+@dataclass(frozen=True, slots=True, kw_only=True)
+class AudioEnclosure:
+    """Audio a page carries, and how strongly the page claims it.
+
+    ``declared`` separates the two markups, because they say different
+    things. ``og:audio`` is the publisher naming this audio as the page's
+    own object — the machine-readable claim "this page IS the audio".  An
+    ``<audio>`` element is only a player: an episode page uses one, and so
+    does an article's read-aloud widget or an encyclopedia's media sample.
+    """
+
+    url: str
+    declared: bool
+
+
+def audio_enclosure(page: str, base_url: str) -> AudioEnclosure | None:
+    """The audio this page carries as its own, or None.
+
+    A bare link to an audio file is deliberately not audio the page
+    carries: a post linking one mp3 is still a post.
+
+    Args:
+        page: The fetched HTML.
+        base_url: The page's URL — relative sources resolve against it.
+
+    Returns:
+        The enclosure (absolute http(s) URL, and whether ``og:audio``
+        declared it), or None.
+    """
+    for pattern in _OG_AUDIO_RES:
+        match = pattern.search(page)
+        if match:
+            resolved = _absolute(match.group(1), base_url)
+            if resolved is not None:
+                return AudioEnclosure(url=resolved, declared=True)
+    for element in _AUDIO_ELEMENT_RE.finditer(page):
+        for source in _SRC_RE.finditer(element.group(0)):
+            resolved = _absolute(source.group(1), base_url)
+            if resolved is not None:
+                return AudioEnclosure(url=resolved, declared=False)
+    return None
+
+
+def _absolute(value: str, base_url: str) -> str | None:
+    candidate = urljoin(base_url, html_lib.unescape(value).strip())
+    return candidate if candidate.startswith(("http://", "https://")) else None

src/dex_engine/drivers/file.py

@@ -1,11 +1,16 @@
 """The file driver: local repo files and URL-served binaries, routed by Format.
 
-Two work shapes, one driver: ``file:<repo-path>`` keys (materialized media
-captures) read from the instance tree; http(s) URLs (a PDF served from an
-arbitrary address, rerouted here by detection's HEAD sniff) fetch through
-the transport with classified failures. Bytes are then byte-signature
-sniffed — authoritative over whatever a server claimed — and handed to
-the first available mechanical extractor for the format.
+Three work shapes, one driver: ``file:<repo-path>`` keys (materialized
+media captures) read from the instance tree; http(s) URLs (a PDF served
+from an arbitrary address, rerouted here by detection's HEAD sniff) fetch
+through the transport with classified failures; and github blob URLs, whose
+bytes come from the shared authenticated seam in
+:mod:`dex_engine.drivers.gh` because the URL itself serves an HTML viewer
+page (and, on a private repo, 404s to an unauthenticated fetch) — the same
+seam the github driver reads them through, so a blob's ref resolves the one
+way for both. Bytes are then byte-signature sniffed —
+authoritative over whatever a server claimed — and handed to the first
+available mechanical extractor for the format.
 
 No provider for the format → ``waiting`` + ``needs: extract`` with the
 registry's stated reason — and so does a provider that reported available
@@ -27,6 +32,7 @@
 
 from dex_engine.capabilities import Capabilities
 from dex_engine.pipeline.classify import (
+    Classification,
     ProviderUnavailableError,
     ScannedDocumentError,
     classify_connection,
@@ -44,6 +50,7 @@
 )
 from dex_engine.pipeline.urls import base_canonical, resolve_repo_path
 
+from .gh import BlobRef, Gh, blob_ref, fetch_blob, run_gh
 from .transport import Transport, urllib_transport
 
 __all__ = ["FileDriver"]
@@ -63,19 +70,23 @@ def __init__(
         capabilities: Capabilities,
         root: Path | None = None,
         transport: Transport = urllib_transport,
+        gh: Gh = run_gh,
     ) -> None:
-        """Wire the extract registry, the instance root, and the HTTP seam.
+        """Wire the extract registry, the instance root, and the two fetch seams.
 
         Args:
             capabilities: The resolved capability registries.
             root: The instance root for ``file:`` work; ``None`` is legal
                 only for registries that never fetch local files (pattern
                 matching, normalize).
             transport: The HTTP seam for URL-served binaries.
+            gh: The gh-CLI seam — a github blob URL's bytes come from the
+                authenticated contents API, never from the viewer page.
         """
         self._capabilities = capabilities
         self._root = root
         self._transport = transport
+        self._gh = gh
 
     def matches(self, url: str) -> bool:
         """True for local-file work keys; URLs reach this driver by sniff."""
@@ -107,11 +118,7 @@ def fetch(self, unit: WorkUnit) -> Result:
         # header over signature-less bytes (a real CSV) must proceed to
         # extraction under its claimed format. Only for http(s) work: a
         # local captured file is not a page to fetch.
-        if (
-            not unit.url.startswith("file:")
-            and sniff_format(data) is None
-            and looks_like_html(data)
-        ):
+        if _rerouteable(unit.url) and sniff_format(data) is None and looks_like_html(data):
             return Result(status=Status.QUEUED, meta={}, redetect=Redetection(kind=Kind.WEB))
         fmt = sniff_format(data, name=name) or unit.format
         if fmt is None:
@@ -128,8 +135,23 @@ def fetch(self, unit: WorkUnit) -> Result:
     def _load(self, unit: WorkUnit) -> tuple[bytes, str | None] | Result:
         if unit.url.startswith("file:"):
             return self._read_local(unit.url.removeprefix("file:"))
+        ref = blob_ref(unit.url)
+        if ref is not None:
+            return self._read_blob(ref)
         return self._download(unit.url)
 
+    def _read_blob(self, ref: BlobRef) -> tuple[bytes, str | None] | Result:
+        """A repo-committed document's bytes, through the gh seam.
+
+        The plain transport cannot serve these: a blob URL answers with the
+        HTML viewer page, and on a private repo the unauthenticated fetch
+        404s — which classified live content ``dead``.
+        """
+        blob = fetch_blob(self._gh, ref)
+        if isinstance(blob, Classification):
+            return Result(status=blob.status, meta={}, reason=blob.reason)
+        return blob.data, blob.path.rsplit("/", 1)[-1] or None
+
     def _read_local(self, repo_path: str) -> tuple[bytes, str | None] | Result:
         if self._root is None:
             # An engine wiring bug, not a content problem: the run layer's
@@ -195,3 +217,15 @@ def _extract(self, data: bytes, fmt: Format, name: str | None) -> Result:
         return Result(
             status=Status.DONE, meta=meta, body=extraction.markdown, assets=extraction.assets
         )
+
+
+def _rerouteable(url: str) -> bool:
+    """Whether HTML bytes at ``url`` mean "this was never file work".
+
+    Only for plain http(s) fetches. A captured local file is not a page to
+    fetch, and a github blob's bytes came from the contents API — HTML
+    there is an HTML file someone committed, not a viewer page, and
+    re-routing it to ``web`` would fetch that viewer and bounce straight
+    back (a loop park).
+    """
+    return not url.startswith("file:") and blob_ref(url) is None