perf: unblock startup and Chat loading

docs/startup-chat-project-performance/intent.md

@@ -0,0 +1,36 @@
+# Intent: Reduce startup, Chat, and Project loading latency
+Author: SpireCode maintainer. Status: approved.
+
+## Problem
+
+SpireCode feels blocked during cold startup, when opening a Chat conversation, and when adding a Project. Read-only diagnosis found that expensive watcher, Git, Pi resource, extension, session snapshot, and durable persistence work is awaited on user-visible critical paths.
+
+On the current development machine, startup restores 17 Projects and 23 Worktrees. The startup path performs one serial Git lookup and watcher initialization per Worktree before loading the Renderer. Opening a cold Chat session initializes Pi resources and extensions before returning its snapshot. Adding a Project waits for watcher initialization before returning to the Renderer.
+
+## Proposed outcome
+
+1. Show the application without waiting for all persisted Worktree watchers.
+2. Return newly added Projects after durable catalog registration while initializing their watchers in the background.
+3. Avoid redundant Chat session scans and repeated identical resource discovery where lifecycle-safe.
+4. Bound Chat snapshots in Electron Main before IPC transfer.
+5. Add privacy-safe phase timing logs so startup, Project open, and Chat attach improvements are measurable.
+
+## Affected users and systems
+
+- All users starting SpireCode with persisted Projects and Worktrees.
+- Users adding local Git repositories.
+- Users opening restored or historical Chat sessions.
+- Electron Main watcher, Git, Chat, settings/resource, diagnostics, and IPC domains.
+
+## Constraints
+
+- Preserve canonical root validation and watcher coverage.
+- Project catalog durability must complete before Project open reports success.
+- Background watcher failures must not become unhandled rejections or remove Projects.
+- Do not share session-bound Pi state across conversations unless the SDK explicitly permits it.
+- Keep Renderer sandbox and narrow IPC contracts unchanged unless a bounded DTO field is required.
+- Performance logs must not include paths, prompts, credentials, message content, or extension configuration.
+
+## Open questions
+
+None.

docs/startup-chat-project-performance/plan.md

@@ -0,0 +1,41 @@
+# Plan: Reduce startup, Chat, and Project loading latency (from docs/startup-chat-project-performance/spec.md 2026-09-23)
+
+## Files that change
+
+Expected files, refined only if tests reveal a narrower boundary:
+
+- `electron/appState.ts` and tests — background, bounded watcher initialization and startup timing.
+- `electron/domains/chat/chatService.ts` and tests — exact session open, shared in-flight lifecycle, bounded snapshots, attach timing.
+- `electron/domains/chat/piAdapter.ts` and tests — expose exact-ID open support and cache only lifecycle-safe discovery work.
+- `electron/domains/diagnostics/service.ts` and tests — privacy-safe performance timing helper if the existing API is insufficient.
+- `docs/startup-chat-project-performance/{intent,spec,plan}.md` — approved requirement, design, and proof chain.
+
+## Order of work
+
+1. Add failing tests proving `AppState.create()` and Project open do not await injected slow watcher initialization.
+2. Implement active-first, bounded background watcher scheduling with safe failure handling and shutdown behavior.
+3. Add failing tests proving cold Chat attach can resolve an exact session without adapter list and duplicate cold opens share one in-flight lifecycle operation.
+4. Implement exact-ID session opening through the Pi adapter without weakening cwd/session-root checks.
+5. Add failing tests proving Main returns at most the newest 2,000 projected Chat timeline items.
+6. Move snapshot bounding before IPC return.
+7. Add privacy-safe phase duration logging and tests that reject sensitive metadata.
+8. Run focused Electron domain tests, then `pnpm check`.
+9. Measure the local 17-Project/23-Worktree startup path to confirm watcher Git processes are no longer in the pre-Renderer critical path.
+10. Review the diff, commit, push, open a PR, and reinstall the application for interactive validation.
+
+## Risks
+
+- The most dangerous change is watcher backgrounding because shutdown or rapid Project close can race initialization. Use idempotent registry operations, disposal guards, and settled promises.
+- Sharing Pi session-bound services is rejected; only exact path resolution and safe discovery/in-flight work may be cached.
+- Returning Project success before watcher readiness creates a brief eventual-consistency window. Initial explicit Files/Git reads cover current state; diagnostics capture watcher failures.
+- Main-side snapshot bounding can hide old UI history by design but must never alter persisted agent context.
+- Unbounded parallel watcher startup is rejected because it would trade startup blocking for a process/I/O spike.
+
+## Proof
+
+- Focused watcher/AppState tests demonstrate delayed watcher promises do not delay startup or Project response.
+- Focused Chat tests demonstrate no redundant list scan, in-flight deduplication, and newest-2,000 snapshot bounds.
+- Diagnostics tests demonstrate timing metadata allowlisting and absence of sensitive values.
+- `pnpm check` exits 0.
+- `pnpm bundle` and macOS App/DMG smoke pass before local reinstall.
+- `git diff --check` exits 0.

docs/startup-chat-project-performance/spec.md

@@ -0,0 +1,68 @@
+# Spec: Reduce startup, Chat, and Project loading latency
+
+## Requirements
+
+### Startup and watchers
+
+1. `BrowserWindow.loadFile()` / `loadURL()` must not wait for persisted Worktree watcher initialization.
+2. Persisted Worktree watchers must initialize in the background with bounded concurrency and active Worktree priority.
+3. Background failures must be logged safely and must not abort startup.
+4. Shutdown must remain safe while background initialization is pending.
+
+### Add Project
+
+5. Opening a Project must still validate its Git root and durably persist the catalog before returning.
+6. Watcher initialization must not delay the successful Project response.
+7. Watcher initialization failures must be observable through diagnostics without invalidating the Project.
+
+### Chat
+
+8. Attaching a cold historical session must avoid a second full session-list scan when an exact session identifier/path can be resolved safely.
+9. Identical in-flight adapter/session initialization must be shared rather than duplicated.
+10. Settings and package/resource discovery may be cached only behind inputs that make invalidation explicit; session-bound resource loaders and extension runtimes must not be shared unsafely.
+11. Main must limit timeline snapshot items before structured-clone IPC transfer. The Renderer limit remains a defensive backstop.
+12. The bounded snapshot must preserve chronological order and retain the newest 2,000 timeline items.
+
+### Measurement
+
+13. Emit structured, privacy-safe duration records for startup AppState, background watcher batches, Project open phases, and Chat attach phases.
+14. Timing records contain phase, duration, success, and safe counts/booleans only; no filesystem paths, project names, prompts, model credentials, or message content.
+
+## Design
+
+### Startup watcher scheduling
+
+`AppState.create()` loads durable services and returns immediately after constructing `AppState`. It schedules persisted Worktree watcher initialization through an internal bounded worker queue. The active Worktree, if any, is first. Remaining watchers run with a small concurrency limit so startup does not launch one Git process per Worktree simultaneously.
+
+`dispose()` marks the state disposed and awaits/cancels only as needed to prevent late unhandled work. Watcher registry operations remain idempotent.
+
+### Project watcher scheduling
+
+`openProject()` awaits `ProjectService.openPath()` and then schedules each Worktree watcher on the same background scheduler. The returned Project is not rolled back if watcher setup fails because watcher setup is runtime infrastructure, not catalog validity.
+
+### Chat attach
+
+Use Pi SDK exact-ID lookup when available rather than `SessionManager.list(cwd)` followed by `.find()`. The resolved path remains constrained to the SDK-owned session directory and is reopened with the expected cwd.
+
+Keep the global `ModelRuntime` and adapter singleton. Cache only pure discovery inputs/results with explicit settings/source fingerprints or short-lived in-flight promises. Continue creating a separate resource loader, extension runtime, and AgentSession per Chat because they own session lifecycle and subscriptions.
+
+At snapshot time, normalize the active branch then retain only the newest 2,000 timeline items before returning from Electron Main.
+
+### Timing diagnostics
+
+Use the existing `DiagnosticsService.log()` JSONL sink. Add a narrow performance helper that records integer duration milliseconds and allowlisted metadata. Startup timing begins in `AppState.create`; Project and Chat timings live at their domain boundaries.
+
+## Acceptance criteria
+
+- With 20+ persisted Worktrees, `AppState.create()` no longer runs one watcher Git lookup per Worktree before Renderer load.
+- `openProject()` resolves before an injected slow watcher promise.
+- A cold Chat attach opens an exact session without invoking adapter list.
+- A snapshot with more than 2,000 projected items returns exactly the newest 2,000.
+- Duplicate initialization tests prove one in-flight operation is shared where caching is introduced.
+- `pnpm check` passes.
+
+## Concerns
+
+- **Concern: PiServices reuse.** `DefaultResourceLoader`, extension runtimes, and bound UI contexts are session-scoped. Reusing them across Chat sessions risks cross-session state and cleanup corruption. The implementation must prefer caching discovery data or in-flight pure work instead of reusing session-bound services.
+- **Concern: watcher readiness.** Filesystem/Git invalidations may be missed during the short interval before the background watcher becomes ready. Initial file tree and Git status reads remain authoritative; watcher readiness is eventual.
+- **Concern: snapshot truncation.** Truncating projected UI items must not modify persisted Pi context or SessionManager entries. It affects only the Renderer snapshot DTO.

electron/appState.test.ts

@@ -1,6 +1,6 @@
 // @vitest-environment node
-import { afterEach, describe, expect, it } from "vitest";
-import { applyMemoryConfig } from "./appState.js";
+import { afterEach, describe, expect, it, vi } from "vitest";
+import { applyMemoryConfig, BackgroundWatcherScheduler } from "./appState.js";
 
 const originalModel = process.env.PI_MEMORY_EXTRACT_MODEL;
 const originalPhase2Model = process.env.PI_MEMORY_PHASE2_MODEL;
@@ -21,6 +21,81 @@ afterEach(() => {
   else process.env.PI_MEMORY_PHASE2_THINKING = originalPhase2Thinking;
 });
 
+describe("BackgroundWatcherScheduler", () => {
+  it("prioritizes the active worktree and bounds concurrency", async () => {
+    const started: string[] = [];
+    const releases = new Map<string, () => void>();
+    let running = 0;
+    let maxRunning = 0;
+    const scheduler = new BackgroundWatcherScheduler(
+      ({ id }) =>
+        new Promise<void>((resolve) => {
+          started.push(id);
+          running += 1;
+          maxRunning = Math.max(maxRunning, running);
+          releases.set(id, () => {
+            running -= 1;
+            resolve();
+          });
+        }),
+      () => undefined,
+      () => undefined,
+      2,
+    );
+
+    scheduler.schedule(
+      [
+        { id: "one", path: "/one" },
+        { id: "two", path: "/two" },
+        { id: "active", path: "/active" },
+        { id: "three", path: "/three" },
+      ],
+      "active",
+    );
+    expect(started).toEqual(["active", "one"]);
+    releases.get("active")?.();
+    await vi.waitFor(() => expect(started).toEqual(["active", "one", "two"]));
+    expect(maxRunning).toBe(2);
+    releases.get("one")?.();
+    releases.get("two")?.();
+    await Promise.resolve();
+    await Promise.resolve();
+    releases.get("three")?.();
+    scheduler.dispose();
+  });
+
+  it("returns immediately, contains failures, and cancels queued work", async () => {
+    const failures: unknown[] = [];
+    let release: () => void = () => undefined;
+    const started: string[] = [];
+    const scheduler = new BackgroundWatcherScheduler(
+      ({ id }) => {
+        started.push(id);
+        if (id === "running")
+          return new Promise<void>((resolve) => {
+            release = resolve;
+          });
+        return Promise.reject(new Error("private path must not escape"));
+      },
+      (error) => failures.push(error),
+      () => undefined,
+      1,
+    );
+
+    scheduler.schedule([
+      { id: "running", path: "/running" },
+      { id: "cancelled", path: "/cancelled" },
+      { id: "failed", path: "/failed" },
+    ]);
+    expect(started).toEqual(["running"]);
+    scheduler.cancel("cancelled");
+    release();
+    await vi.waitFor(() => expect(started).toEqual(["running", "failed"]));
+    await vi.waitFor(() => expect(failures).toHaveLength(1));
+    scheduler.dispose();
+  });
+});
+
 describe("applyMemoryConfig", () => {
   it("clears Memory overrides for an unconfigured clean install", () => {
     process.env.PI_MEMORY_EXTRACT_MODEL = "old/model";