fix: session lifecycle, harbor cache, console verdicts and paging, no generation caps or trimmed output

deepgym/CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## Unreleased
+
+- **Unsandboxed runs leave a trace:** every local `RunResult` records how its solution was isolated (`isolation`: `macos`, `bwrap`, or `none` under `DEEPGYM_ALLOW_UNSANDBOXED`), and the process logs a warning once when local solutions run unsandboxed. A local session's results record `none`, since its agent actions run on the host, and creating one warns once.
+- **Closed sessions stay closed:** once `close()` runs (even if the delete fails), `start`, `step`, `run_cli`, `finalize_cli` and `score` raise the new `SessionClosedError` before touching the sandbox; `score()` still returns a result a failed step already fixed. The API answers a request that raced a DELETE with 409, so a host-deleted sandbox is never scored as the agent's 0. A session whose setup fails is closed through the same pipeline as a DELETE, so a failed delete keeps its slot, is retried by the sweep and is reported in `leaked_sandboxes` if it never succeeds.
+- **Harbor:** remote datasets are cached in a folder keyed by a digest of the full URL and reused only while git confirms the folder is a clone of that URL; two URLs that read alike no longer share tasks. Test output is kept whole in results and reward errors.
+- **Console:** episodes with no verdict show as unscored (`has_verdict` and `error_type` are stored, and existing rows are migrated from their saved results) instead of as a failed 0. Episodes and runs are paged on the server with a total, with no 200-row default and no upper bound, and a page that a refresh empties moves back to the last page.
+- **No generation caps:** the Z.ai client sends no `max_tokens` unless asked, has no read timeout by default and returns a `Completion` with its `finish_reason`; `Completion.whole` is True only for a natural stop, False when a limit or filter cut the reply, None when unknown, and the QA difficulty probe, the CyberGym scripts and the seed generator score only whole answers, so a probe is inconclusive, never `too_hard`, when a sample isn't one. Breaking: `make_lm_eval_task` takes a required `max_gen_toks`, the model's maximum output (lm-eval needs one and would otherwise default to 256). Claude calls in the adversarial tester and RL exploit discovery stream at the model's maximum output, read a smaller model's maximum from its error, and refuse a reply cut off at that limit or a stream that ended before `message_stop`, so a partial attack list can't certify a verifier as robust. The CyberGym eval scripts cache each reply with its finish reason (`cached_completion`, in a record beside any legacy text cache, which stays opaque text of unknown completion), drop their default response deadlines, average only judged runs of whole answers (cut-off, unknown-completion, fallback and no-verdict runs are counted apart), and no longer build a fallback answer from the expected answer.
+- **No trimmed output:** errors, adversarial and exploit reports, and QA and `deepgym check` details carry command output, code and details whole. Audit ids default to the environment's identity (`environment_id`: its name, else a digest of its whole task) instead of the task's first 80 characters, which two tasks could share, and the stored version hash (`grading_fingerprint`) covers the whole environment definition, Harbor's test scripts included, so any changed grading input invalidates the old result.
+
 ## 0.6.1 (2026-10-23)
 
 - **Local sandbox reads only what a run needs:** under `sandbox-exec` and `bwrap`, a solution can read the operating system, the software trees a Python links against, the Python installation and its own run's files, and nothing else. Before, only the home directory, this process's runs and daemon sockets were hidden, so a solution could read files under `/tmp` and `$TMPDIR`, including another DeepGym process's verifier copies. Every deepgym package on the import path stays unreadable, not just the one running. Runs roots that a killed process left behind are removed by the next process.

deepgym/README.md

@@ -228,7 +228,11 @@ import lm_eval
 from deepgym import load_environment
 from deepgym.integrations.lm_eval import make_lm_eval_task
 
-tasks = [make_lm_eval_task(load_environment(n), n) for n in ['coin_change', 'two_sum']]
+# max_gen_toks: your model's maximum output (Qwen2.5 generates up to 8K tokens).
+tasks = [
+    make_lm_eval_task(load_environment(n), n, max_gen_toks=8192)
+    for n in ['coin_change', 'two_sum']
+]
 results = lm_eval.simple_evaluate(
     model='hf', model_args='pretrained=Qwen/Qwen2.5-0.5B-Instruct', tasks=tasks
 )

deepgym/console-ui/components/episodes/episode-replay.tsx

@@ -10,7 +10,7 @@ import { Kbd } from '@/components/base/kbd/kbd';
 import { CodeBlock } from '@/components/primitives/code-block';
 import { ThinkingTrace } from '@/components/primitives/thinking-trace';
 import { Card } from '@/components/ui/card';
-import { PassBadge } from '@/components/ui/status-badge';
+import { VerdictBadge } from '@/components/ui/status-badge';
 import { formatScore } from '@/lib/format';
 import type { EpisodeDetail } from '@/lib/types';
 import { useHotkey } from '@/lib/use-hotkey';
@@ -140,9 +140,9 @@ export function EpisodeReplay({ episode }: { episode: EpisodeDetail }) {
             <Card title="Final score">
               <div className="flex items-center gap-3">
                 <span className="font-mono text-title-2-semibold text-text-primary">
-                  {formatScore(episode.result.score)}
+                  {formatScore(episode.score)}
                 </span>
-                <PassBadge passed={episode.result.passed} />
+                <VerdictBadge verdict={episode} />
               </div>
               {episode.result.output && (
                 <p className="mt-1.5 text-body-2-regular text-text-secondary">{episode.result.output}</p>

deepgym/console-ui/components/episodes/episodes-page.test.tsx

@@ -0,0 +1,139 @@
+import { afterEach, beforeEach, expect, jest, test } from 'bun:test';
+import { act } from 'react';
+import type { EpisodeDetail, EpisodePage, EpisodeSummary } from '@/lib/types';
+import { cleanup, fakeFetch, render, settle } from '@/test/dom';
+import { navigation } from '@/test/navigation';
+
+// After the navigation mock, so the page reads ?id= from it.
+const { EpisodesPage } = await import('./episodes-page');
+
+afterEach(cleanup);
+beforeEach(() => navigation.reset('/episodes/'));
+
+const episode = (id: string, extra: Partial<EpisodeSummary> = {}): EpisodeSummary => ({
+  id,
+  environment: `env-${id}`,
+  steps: 2,
+  total_reward: 0,
+  score: 1,
+  passed: true,
+  has_verdict: true,
+  error_type: null,
+  created_at: '2026-10-23T10:00:00Z',
+  ...extra,
+});
+
+/** The provider failed before the verifier judged it: no score, no pass. */
+const crashed = episode('crashed', {
+  score: null,
+  passed: null,
+  has_verdict: false,
+  error_type: 'provider_error',
+});
+/** Judged and failed: a real 0. */
+const zero = episode('zero', { score: 0, passed: false });
+
+const page = (episodes: EpisodeSummary[], total: number) =>
+  Response.json({ episodes, total } satisfies EpisodePage);
+
+const rowText = (text: string) =>
+  [...document.querySelectorAll('[role="row"]')].find((row) => row.textContent?.includes(text))?.textContent;
+
+test('an episode without a verdict is listed as no verdict with its error, not as a failed 0', async () => {
+  fakeFetch(() => page([crashed, zero], 2));
+  await render(<EpisodesPage />);
+  await settle();
+
+  expect(rowText('env-crashed')).toContain('no verdictprovider_error');
+  expect(rowText('env-crashed')).not.toContain('fail');
+  expect(rowText('env-zero')).toContain('0.00fail');
+});
+
+test('replaying an episode without a verdict shows its error and result output, not a failed 0', async () => {
+  navigation.reset('/episodes/', 'id=crashed');
+  const detail: EpisodeDetail = {
+    ...crashed,
+    frames: [
+      {
+        index: 0,
+        content: '',
+        done: true,
+        metadata: null,
+        action: null,
+        reward: null,
+        has_screenshot: false,
+      },
+    ],
+    result: {
+      score: 0,
+      passed: false,
+      output: 'provider unreachable',
+      stderr: '',
+      exit_code: 1,
+      execution_time_ms: 0,
+      sandbox_id: 'error',
+      reward_components: null,
+      error_type: 'provider_error',
+      cases: null,
+    },
+  };
+  fakeFetch((url) => (url === '/api/episodes/crashed' ? Response.json(detail) : page([crashed], 1)));
+  await render(<EpisodesPage />);
+  await settle();
+
+  const card = [...document.querySelectorAll('section')].find((section) =>
+    section.textContent?.startsWith('Final score'),
+  );
+  expect(card?.textContent).toBe('Final score-no verdictprovider_errorprovider unreachable');
+});
+
+test('every episode is reachable: the pager fetches the next page from the server', async () => {
+  const newest = Array.from({ length: 25 }, (_, index) => episode(`new${index}`));
+  const requested: string[] = [];
+  fakeFetch((url) => {
+    requested.push(url);
+    return url.includes('offset=25') ? page([episode('oldest')], 26) : page(newest, 26);
+  });
+  await render(<EpisodesPage />);
+  await settle();
+  expect(rowText('env-oldest')).toBeUndefined();
+
+  const next = document.querySelector<HTMLButtonElement>('button[aria-label="Go to page 2"]');
+  if (!next) throw new Error('no pager');
+  await act(async () => next.click());
+  await settle();
+
+  expect(requested).toEqual(['/api/episodes?offset=0&limit=25', '/api/episodes?offset=25&limit=25']);
+  expect(rowText('env-oldest')).toContain('env-oldest');
+  expect(rowText('env-new0')).toBeUndefined();
+});
+
+test('a poll that leaves the page shown past the last one moves back to the last page', async () => {
+  jest.useFakeTimers();
+  const recorded = Array.from({ length: 26 }, (_, index) => episode(`e${index}`));
+  const requested: string[] = [];
+  fakeFetch((url) => {
+    requested.push(url);
+    const offset = Number(new URL(url, 'http://localhost').searchParams.get('offset'));
+    return page(recorded.slice(offset, offset + 25), recorded.length);
+  });
+  await render(<EpisodesPage />);
+  await settle();
+  const next = document.querySelector<HTMLButtonElement>('button[aria-label="Go to page 2"]');
+  if (!next) throw new Error('no pager');
+  await act(async () => next.click());
+  await settle();
+  expect(rowText('env-e25')).toContain('env-e25');
+
+  // One episode is deleted, so the next poll of page 2 finds it empty, with one page left.
+  recorded.pop();
+  await act(async () => jest.advanceTimersByTime(10_000));
+  await settle();
+
+  expect(requested.slice(-2)).toEqual([
+    '/api/episodes?offset=25&limit=25',
+    '/api/episodes?offset=0&limit=25',
+  ]);
+  expect(document.querySelectorAll('[role="row"]')).toHaveLength(26);
+  expect(rowText('env-e0')).toContain('env-e0');
+});

deepgym/console-ui/components/episodes/episodes-page.tsx

@@ -7,10 +7,11 @@ import { Card } from '@/components/ui/card';
 import { DataTable } from '@/components/ui/data-table';
 import { EmptyState } from '@/components/ui/empty-state';
 import { PageHeader } from '@/components/ui/page-header';
-import { PassBadge } from '@/components/ui/status-badge';
+import { VerdictBadge } from '@/components/ui/status-badge';
 import { useApi } from '@/lib/api';
 import { formatScore, timeAgo } from '@/lib/format';
-import type { EpisodeDetail, EpisodeSummary } from '@/lib/types';
+import type { EpisodeDetail, EpisodePage, EpisodeSummary } from '@/lib/types';
+import { usePagedApi } from '@/lib/use-paged-api';
 import { useQueryParam } from '@/lib/use-query-param';
 import { EpisodeReplay } from './episode-replay';
 
@@ -38,16 +39,15 @@ const columns: ColumnDef<EpisodeSummary>[] = [
   { id: 'steps', accessorKey: 'steps', header: 'Steps', meta: { align: 'end', className: 'font-mono' } },
   {
     id: 'score',
-    accessorFn: (episode) => episode.score ?? -1,
+    accessorKey: 'score',
     header: 'Score',
     cell: ({ row }) => formatScore(row.original.score),
     meta: { align: 'end', className: 'font-mono' },
   },
   {
-    id: 'passed',
+    id: 'verdict',
     header: '',
-    enableSorting: false,
-    cell: ({ row }) => <PassBadge passed={row.original.passed} />,
+    cell: ({ row }) => <VerdictBadge verdict={row.original} />,
   },
 ];
 
@@ -63,19 +63,18 @@ function Replay({ id }: { id: string }) {
   return <EpisodeReplay key={id} episode={data} />;
 }
 
-/** Recorded episodes on the left; the selected one (?id=) replays on the right. */
+/** Recorded episodes on the left, a server page at a time; the selected one (?id=) replays on the right. */
 export function EpisodesPage() {
-  const { data, error } = useApi<EpisodeSummary[]>('/episodes', 10_000);
+  const { data, error, paging } = usePagedApi<EpisodePage>('/episodes', 10_000);
   const [selected, setSelected] = useQueryParam('id');
-  const episodes = data ?? [];
 
   return (
     <>
       <PageHeader
         title="Episodes"
         description="Recorded multi-turn and computer-use trajectories, replayed step by step."
       />
-      {data && episodes.length === 0 ? (
+      {data?.total === 0 ? (
         <Card>
           <EmptyState icon={RiFilmLine} title="No episodes recorded yet">
             Store a finished episode from your agent loop and it shows up here:
@@ -87,10 +86,10 @@ export function EpisodesPage() {
           <Card title="Recorded" flush className="self-start">
             <DataTable
               label="Episodes"
-              data={episodes}
+              data={data?.episodes ?? []}
               columns={columns}
               onRowAction={(episode) => setSelected(episode.id)}
-              pageSize={25}
+              serverPaging={paging}
               empty={
                 <EmptyState icon={RiFilmLine} title={error ? 'Could not load episodes' : 'Loading episodes'}>
                   {error}

deepgym/console-ui/components/overview/overview-page.tsx

@@ -13,7 +13,7 @@ import { RangeControl, useRange } from '@/components/ui/range-control';
 import { useApi } from '@/lib/api';
 import { timeAgo } from '@/lib/format';
 import { RANGES } from '@/lib/stats';
-import type { QAReport, RunStats, RunSummary } from '@/lib/types';
+import type { QAReport, RunPage, RunStats } from '@/lib/types';
 import { recentActivity } from '@/lib/verdict';
 import { EnvHealth } from './env-health';
 
@@ -22,9 +22,9 @@ export function OverviewPage() {
   const router = useRouter();
   const [range, setRange] = useRange();
   const { data: stats, error } = useApi<RunStats>(`/runs/stats?hours=${RANGES[range]}`, 5000);
-  const { data: runs } = useApi<RunSummary[]>('/runs?limit=10', 5000);
+  const { data: latest } = useApi<RunPage>('/runs?limit=10', 5000);
   const { data: reports } = useApi<QAReport[]>('/qa', 15_000);
-  const activity = recentActivity(runs ?? [], reports ?? [], 10);
+  const activity = recentActivity(latest?.runs ?? [], reports ?? [], 10);
 
   return (
     <>
@@ -58,8 +58,8 @@ export function OverviewPage() {
           {activity.length ? (
             <TaskRows rows={activity.map((item) => ({ ...item, meta: timeAgo(item.at) }))} />
           ) : (
-            <EmptyState icon={RiPulseLine} title={runs && reports ? 'No activity yet' : 'Loading activity'}>
-              {runs && reports ? 'Runs and QA verdicts show up here as they happen.' : null}
+            <EmptyState icon={RiPulseLine} title={latest && reports ? 'No activity yet' : 'Loading activity'}>
+              {latest && reports ? 'Runs and QA verdicts show up here as they happen.' : null}
             </EmptyState>
           )}
         </Card>

deepgym/console-ui/components/runs/run-detail-page.tsx

@@ -13,7 +13,7 @@ import { PageHeader } from '@/components/ui/page-header';
 import { VerdictBanner } from '@/components/ui/verdict-banner';
 import { api, errorText, useApi } from '@/lib/api';
 import { formatScore, timeAgo } from '@/lib/format';
-import type { RunDetail, RunSummary } from '@/lib/types';
+import type { RunDetail, RunPage, RunSummary } from '@/lib/types';
 import { useQueryParam } from '@/lib/use-query-param';
 import { failuresFirst, isActive, runTaskStatus, verdictLine } from '@/lib/verdict';
 import { RunProgress } from './run-progress';
@@ -22,9 +22,10 @@ import { RunSummary as RunSummaryStrip } from './run-summary';
 import { ScoreBreakdown } from './score-breakdown';
 
 function GroupRuns({ run }: { run: RunDetail }) {
-  const { data } = useApi<RunSummary[]>(`/runs?group_id=${run.group_id}`, 2500, (runs) =>
+  const { data: page } = useApi<RunPage>(`/runs?group_id=${run.group_id}`, 2500, ({ runs }) =>
     runs.some(isActive),
   );
+  const data = page?.runs;
   if (!data || data.length < 2) return null;
   const running = data.filter(isActive).length;
   const failed = data.filter((item) => !isActive(item) && !item.passed).length;

deepgym/console-ui/components/runs/run-progress.tsx

@@ -4,17 +4,17 @@ import { Card } from '@/components/ui/card';
 import { useApi } from '@/lib/api';
 import { formatDuration } from '@/lib/format';
 import { etaNote, medianDuration } from '@/lib/stats';
-import type { RunDetail, RunSummary } from '@/lib/types';
+import type { RunDetail, RunPage } from '@/lib/types';
 import { cx } from '@/utils/cx';
 
 export const KEEPS_GOING = 'You can leave this page, the run keeps going.';
 
 /** Median duration of the latest finished runs of `environment` (null while loading or with none). */
 export function useTypicalDuration(environment: string | null): number | null {
-  const { data } = useApi<RunSummary[]>(
+  const { data } = useApi<RunPage>(
     environment ? `/runs?environment=${encodeURIComponent(environment)}&status=completed&limit=20` : null,
   );
-  return data ? medianDuration(data) : null;
+  return data ? medianDuration(data.runs) : null;
 }
 
 /**