(sidebar): fetch projects once and skip unchanged sidebar renders

.ai/contexts/ipc-bridge.md

@@ -18,7 +18,7 @@ This file is the **canonical inventory** of the IPC surface. When you add a new
 
 | IPC | Args | Returns | Notes |
 |---|---|---|---|
-| `get-projects` | `(showArchived)` | `Project[]` | Sidebar payload. Reads from cache. |
+| `get-projects` | `()` | `{ projects: Project[], allProjects: Project[] }` | Sidebar payload, archived sessions hidden / shown, from one build. Reads from cache. See `session-cache.md` ("One build, two views"). |
 | `get-active-sessions` | — | `{sessionId, busy}[]` | Currently open PTY sessions plus each one's live `_cliBusy` flag — see "Busy-state reconciliation" below. |
 | `get-active-terminals` | — | `Terminal[]` | Active PTY identifiers |
 | `open-terminal` | `(id, projectPath, isNew, sessionOptions)` | `{ok, error?, mcpActive}` | Spawn or attach a PTY. |

.ai/contexts/session-cache.md

@@ -29,7 +29,7 @@ From `session-cache.js`:
 - `init(ctx)` — wire main process → cache (mainWindow ref for IPC events)
 - `refreshFolder(folder, opts)` — opts `{files: Set<string>}` for targeted refresh (watcher payload). Defaults to full folder walk.
 - `populateCacheFromFilesystem()` / `populateCacheViaWorker()` — initial scan / re-scan. The worker (`workers/scan-projects.js`) streams one `{type:'folder', result, current, total}` message per on-disk folder (plus a final `{type:'done'}`) instead of buffering the whole tree, so each folder is written to the DB and pushed to the renderer as soon as it's read — a large history no longer leaves the sidebar empty for the entire scan.
-- `buildProjectsFromCache(showArchived)` — produces the sidebar payload (sorted, grouped by project, missing flag computed here). It also injects every live plain-terminal PTY from `activeSessions` as a synthetic row, so a terminal sorts among the cached sessions even though it has no JSONL; the row's `summary` is the hard-coded string `Terminal`, which is what the sidebar displays — the renderer's own session object is never the one shown. A panel shell is a plain terminal too and is excluded here by `isPanelShellSession` (`panel-terminal-target.js`); see `.ai/contexts/panel-terminal.md` for the full list of places that have to skip one.
+- `buildProjectViewsFromCache()` — returns `{ projects, allProjects }` (archived hidden / shown) from one read of the cache; this is what `get-projects` serves (see "One build, two views"). `buildProjectsFromCache(showArchived)` builds one of the two — produces the sidebar payload (sorted, grouped by project, missing flag computed here). It also injects every live plain-terminal PTY from `activeSessions` as a synthetic row, so a terminal sorts among the cached sessions even though it has no JSONL; the row's `summary` is the hard-coded string `Terminal`, which is what the sidebar displays — the renderer's own session object is never the one shown. A panel shell is a plain terminal too and is excluded here by `isPanelShellSession` (`panel-terminal-target.js`); see `.ai/contexts/panel-terminal.md` for the full list of places that have to skip one.
 - `notifyRendererProjectsChanged()` — throttled (~1.5s leading-edge) push to renderer
 - `sendIndexingProgress()` (internal) — emits the `indexing-progress` IPC event, gated on `coldStart` (captured once at the top of `populateCacheViaWorker()` via `!isInitialScanComplete()`) and throttled to ~4 events/s (the first event and every `done:true` always pass). Feeds the renderer's first-run banner; see `.ai/contexts/ipc-bridge.md`. A `done:true` payload carrying `error` keeps the banner visible with the failure message instead of hiding it.
 
@@ -76,6 +76,39 @@ From `derive-project-path.js`: `deriveProjectPath(folderPath)`, `resolveWorktree
 
 - **Neither the working-set restore nor the reload path resumes a session that is live in another process.** `runRestore` and the post-`loadProjects` re-open of `sessionStorage.activeSessionId` call `openSession(..., { automatic: true })`, which skips the session without a prompt when `guardResume` reports it live elsewhere; the skipped entry is not activated, stays in the persisted working set at its saved position, and is reported by a one-line notice. See `.ai/contexts/cli-session-state.md` ("Live elsewhere").
 
+## Sidebar refresh cost
+
+Measured with 6 live Claude sessions: the renderer burned ~34% of a core and main ~17% (spikes to 110%). Every JSONL append reaches `notifyRendererProjectsChanged` (1.5 s throttle), then the renderer's `onProjectsChanged` (900 ms debounce), then `loadProjects`, which rebuilt the whole sidebar (~22k-node detached tree + morphdom) about once a second. Two changes remove most of that.
+
+### One build, two views
+
+The renderer keeps two lists: `cachedProjects` (archived sessions hidden) and `cachedAllProjects` (everything; used by the archive toggle, search, the status-bar totals and pending-session reconciliation). `loadProjects` used to fetch them with two `get-projects` calls (`showArchived` false and true), so every refresh paid two `getAllCached()` (`SELECT *` over every row), two `getAllMeta()`, two `readdirSync` of the projects dirs plus the per-project `existsSync` probes, two `reconcileCacheFromFilesystem()` stat sweeps and two structured clones over IPC.
+
+`get-projects` now takes no argument and returns `{ projects, allProjects }` from a single `buildProjectViewsFromCache()`:
+
+- `gatherProjectInputs()` does all the reading once: cache rows, meta, folder meta, the empty-dir listing (including the `deriveProjectPath` / `setFolderMeta` backfill), the plain-terminal rows. It builds each session object once and tags it `hiddenUnlessArchived` (archived itself, or a subagent of an archived parent — the same two filters as before).
+- `assembleProjects(inputs, showArchived)` is the old grouping/sorting pass, run twice on the same inputs. `existsSync` is memoised per build, so the `missing` probe runs once per path.
+- The two views share their session objects. Structured clone keeps shared references within one message, so the payload carries each session once, and the renderer's `dedup()` already expected the two lists to point at the same objects.
+
+The views are not derived from each other in the renderer on purpose: a project whose every session is archived appears in the hidden-archived view only through the on-disk empty-dir pass (with that pass's `folder`/`missing` values), which the renderer cannot reproduce without the directory listing. `buildProjectsFromCache(showArchived)` stays as a one-view wrapper over the same two functions; `test/sidebar-refresh-single-fetch.test.js` checks the two views equal the old per-flag builds.
+
+Not done: memoising the built views in main between calls. With one call per refresh there is no second caller in the same tick to share it with, and invalidating it correctly (cache writes, meta writes, hidden projects, terminals, remote descriptors) would cost more risk than it saves.
+
+### Skipping an unchanged sidebar render
+
+`renderProjects(projects, resort, { skipIfUnchanged })` returns `false` without touching the DOM when the signature of what it would render equals the one recorded after the last real render (from any caller). Only the `projects-changed` reload passes `skipIfUnchanged: true`; every other caller (toggles, archive, rename, remote-host refresh, tab return, `resort`) still renders unconditionally, because some of them patch the DOM by hand and count on the re-render to put it back. When the render is skipped, `refreshSidebar` runs `refreshSessionTimeLabels()` (the 30 s ticker's body) so the time labels still pick up the new `modified`.
+
+The signature (`sidebarRenderSignature` in `public/sidebar.js`) is recorded *after* rendering, because rendering itself writes state (`paintSessionIcon` creates and updates `localPtyStates`, `seedRemoteActivity` marks remote rows busy). It covers:
+
+- every field of every project and session, except `firstPrompt` and `created` (the sidebar never reads them) and with `modified` rounded down to the minute;
+- projects and sessions in a canonical order (by content), not the order main sends. Main sorts by exact recency, so a live session's append reorders the list on every flush, but the renderer keeps its own order (`sortedOrder`) for items it has already shown. The places where it does follow the data order are kept in the signature: the members of each slug group and the subagents of each parent, in data order, for every group with more than one member;
+- renderer state the build reads: the filters, search sets, `visibleSessionCount`, `sessionMaxAgeDays`, `activeSessionId`, `activePtyIds`, `pendingSessions`, `activeSubagentsByParent`, and every state snapshot in `localPtyStates` / `remoteSessionStates` / `localTranscriptStates` except `lastActivityAt` / `lastActivitySource`;
+- the current minute, so anything that depends on the clock (the age cutoff that moves sessions into "older", the stale-project auto-collapse, the "today" filter, slug-group header times, status ages) is re-rendered at least once a minute while updates keep coming.
+
+So a live session that only bumps its `modified` costs one render per minute instead of one per second, plus a label refresh. A change to any displayed field, a new or removed session, a different running/busy state or a minute boundary still renders. If you add something the sidebar renders, it is in the signature automatically when it is a field on the project or session object; renderer state it reads from elsewhere has to be added to `sidebarRenderSignature` by hand.
+
+The signature costs ~30 ms for ~5k sessions in jsdom (measured), against seconds for the render it replaces there.
+
 ## Remote SSH hosts (issue #201)
 
 A declared SSH host's `~/.claude/projects` is mirrored into

main.js

@@ -453,7 +453,7 @@ sessionCache.init({
   },
 });
 const { readSessionFile, readFolderFromFilesystem, refreshFolder, reconcileCacheFromFilesystem,
-        buildProjectsFromCache, notifyRendererProjectsChanged, sendStatus, populateCacheViaWorker,
+        buildProjectViewsFromCache, notifyRendererProjectsChanged, sendStatus, populateCacheViaWorker,
         scanFoldersViaWorker, setRemoteRoots, resolveFolderDir } = sessionCache;
 const { resolveJsonlPath, enumerateSessionFiles } = require('./read-session-file');
 
@@ -1048,7 +1048,7 @@ ipcMain.handle('rebuild-cache', async () => {
   }
 });
 
-ipcMain.handle('get-projects', async (_event, showArchived) => {
+ipcMain.handle('get-projects', async () => {
   try {
     // "Cache has rows" is NOT enough to call the start warm: the scan worker
     // streams one DB write per folder, so killing the app mid-scan leaves
@@ -1074,7 +1074,7 @@ ipcMain.handle('get-projects', async (_event, showArchived) => {
       // (see populateCacheViaWorker in session-cache.js), and emits
       // `indexing-progress` events the renderer turns into a one-time banner.
       // This immediate response returns whatever's cached right now (empty
-      // session rows on a true first run, but buildProjectsFromCache still
+      // session rows on a true first run, but buildProjectViewsFromCache still
       // lists every on-disk project directory synchronously below).
       populateCacheViaWorker();
     } else {
@@ -1094,10 +1094,15 @@ ipcMain.handle('get-projects', async (_event, showArchived) => {
       reconcileCacheFromFilesystem();
     }
 
-    return annotateRemoteAttachable(mergePlaceholderSessions(buildProjectsFromCache(showArchived)));
+    // see .ai/contexts/session-cache.md ("One build, two views")
+    const views = buildProjectViewsFromCache();
+    return {
+      projects: annotateRemoteAttachable(mergePlaceholderSessions(views.projects)),
+      allProjects: annotateRemoteAttachable(mergePlaceholderSessions(views.allProjects)),
+    };
   } catch (err) {
     console.error('Error listing projects:', err);
-    return [];
+    return { projects: [], allProjects: [] };
   }
 });
 

preload.js

@@ -12,7 +12,7 @@ contextBridge.exposeInMainWorld('api', {
   getWorkFiles: () => ipcRenderer.invoke('get-work-files'),
   readWorkFile: (filePath) => ipcRenderer.invoke('read-work-file', filePath),
   deleteWorkFile: (filePath) => ipcRenderer.invoke('delete-work-file', filePath),
-  getProjects: (showArchived) => ipcRenderer.invoke('get-projects', showArchived),
+  getProjects: () => ipcRenderer.invoke('get-projects'),
   rebuildCache: () => ipcRenderer.invoke('rebuild-cache'),
   getActiveSessions: () => ipcRenderer.invoke('get-active-sessions'),
   getSessionLiveElsewhere: (id) => ipcRenderer.invoke('session-live-elsewhere', id),

public/app.js

@@ -562,7 +562,7 @@ window.api.onCliBusyState((sessionId, busy) => {
 // --- Single entry point for all sidebar renders ---
 // resort=true: re-sort items by priority+time (use for user-initiated actions)
 // resort=false (default): preserve existing DOM order, new items go to top
-function refreshSidebar({ resort = false } = {}) {
+function refreshSidebar({ resort = false, skipIfUnchanged = false } = {}) {
   // When searching, always use all projects (search ignores archive filter)
   let projects = (searchMatchIds !== null)
     ? cachedAllProjects
@@ -581,7 +581,10 @@ function refreshSidebar({ resort = false } = {}) {
     }).filter(Boolean);
   }
 
-  renderProjects(projects, resort);
+  if (!renderProjects(projects, resort, { skipIfUnchanged })) {
+    refreshSessionTimeLabels();
+    return;
+  }
   pruneRemoteActivityTimers();
   pruneLocalTranscriptTimers();
 }
@@ -984,8 +987,7 @@ function updatePtyTitle() {
 
 scheduleActiveSessionsPoll();
 
-// Refresh sidebar timeago labels every 30s so "just now" ticks forward
-setInterval(() => {
+function refreshSessionTimeLabels() {
   for (const [sessionId, session] of sessionMap) {
     if (!session.modified) continue;
     const item = document.getElementById('si-' + sessionId);
@@ -995,7 +997,10 @@ setInterval(() => {
     const msgSuffix = session.messageCount ? ' \u00b7 ' + session.messageCount + ' msgs' : '';
     timeEl.textContent = formatDate(new Date(session.modified)) + msgSuffix;
   }
-}, 30000);
+}
+
+// Refresh sidebar timeago labels every 30s so "just now" ticks forward
+setInterval(refreshSessionTimeLabels, 30000);
 
 // Shared session map so all caches reference the same objects
 const sessionMap = new Map();
@@ -1014,17 +1019,15 @@ function dedup(projects) {
   }
 }
 
-async function loadProjects({ resort = false } = {}) {
+async function loadProjects({ resort = false, skipIfUnchanged = false } = {}) {
   const wasEmpty = cachedProjects.length === 0;
   if (wasEmpty) {
     loadingStatus.textContent = 'Loading\u2026';
     loadingStatus.className = 'active';
     loadingStatus.style.display = '';
   }
-  const [defaultProjects, allProjects] = await Promise.all([
-    window.api.getProjects(false),
-    window.api.getProjects(true),
-  ]);
+  // see .ai/contexts/session-cache.md ("One build, two views")
+  const { projects: defaultProjects, allProjects } = await window.api.getProjects();
   cachedProjects = defaultProjects;
   cachedAllProjects = allProjects;
   loadingStatus.style.display = 'none';
@@ -1076,7 +1079,7 @@ async function loadProjects({ resort = false } = {}) {
   } catch {}
 
   await pollActiveSessions();
-  refreshSidebar({ resort });
+  refreshSidebar({ resort, skipIfUnchanged });
   renderDefaultStatus();
 }
 
@@ -1466,7 +1469,7 @@ window.api.onProjectsChanged(() => {
   // sidebar redraws at most ~1×/sec.
   projectsChangedTimer = setTimeout(() => {
     projectsChangedTimer = null;
-    loadProjects().then(() => maybeRetryRestoreWorkingSet());
+    loadProjects({ skipIfUnchanged: true }).then(() => maybeRetryRestoreWorkingSet());
   }, 900);
 });
 

public/sidebar.js

@@ -573,8 +573,78 @@ function buildSlugGroup(slug, sessions, subagentIndex) {
   return group;
 }
 
-function renderProjects(projects, resort) {
+// see .ai/contexts/session-cache.md ("Skipping an unchanged sidebar render")
+let lastSidebarRenderSignature = null;
+
+function sortedKeys(collection) {
+  if (!collection) return null;
+  return [...(typeof collection.keys === 'function' ? collection.keys() : collection)].map(String).sort();
+}
+
+const SIGNATURE_SKIPPED_FIELDS = new Set(['sessions', 'firstPrompt', 'created', 'lastActivityAt', 'lastActivitySource']);
+
+function signatureOfFields(obj) {
+  let out = '';
+  for (const key in obj) {
+    if (SIGNATURE_SKIPPED_FIELDS.has(key)) continue;
+    let value = obj[key];
+    if (key === 'modified') {
+      const t = Date.parse(value);
+      if (Number.isFinite(t)) value = Math.floor(t / 60000);
+    }
+    out += key + '\u0001' + (value !== null && typeof value === 'object' ? JSON.stringify(value) : String(value)) + '\u0002';
+  }
+  return out;
+}
+
+function sidebarRenderSignature(projects) {
+  try {
+    const projectParts = [];
+    for (const p of projects) {
+      const sessionParts = [];
+      const groups = new Map();
+      for (const s of p.sessions) {
+        sessionParts.push(signatureOfFields(s));
+        const key = s.parentSessionId ? 'p:' + s.parentSessionId : (s.slug ? 's:' + s.slug : null);
+        if (!key) continue;
+        if (!groups.has(key)) groups.set(key, []);
+        groups.get(key).push(s.sessionId);
+      }
+      const order = [];
+      for (const [key, ids] of groups) if (ids.length > 1) order.push(key + '=' + ids.join(','));
+      projectParts.push(signatureOfFields(p) + '\u0003' + sessionParts.sort().join('\u0004') + '\u0003' + order.sort().join('\u0004'));
+    }
+    const activity = [];
+    const stateMaps = [
+      ['l', typeof localPtyStates !== 'undefined' ? localPtyStates : null],
+      ['r', typeof remoteSessionStates !== 'undefined' ? remoteSessionStates : null],
+      ['t', typeof localTranscriptStates !== 'undefined' ? localTranscriptStates : null],
+    ];
+    for (const [tag, map] of stateMaps) {
+      if (!map) continue;
+      for (const [id, st] of map) activity.push(tag + ':' + id + ':' + signatureOfFields(st.snapshot()));
+    }
+    const subagents = [];
+    for (const [parentId, agents] of activeSubagentsByParent) subagents.push(parentId + ':' + sortedKeys(agents).join(','));
+    const state = JSON.stringify({
+      now: Math.floor(Date.now() / 60000),
+      starred: showStarredOnly, running: showRunningOnly, today: showTodayOnly,
+      search: sortedKeys(searchMatchIds), searchProjects: sortedKeys(searchMatchProjectPaths),
+      visible: visibleSessionCount, maxAge: sessionMaxAgeDays, active: activeSessionId,
+      pty: sortedKeys(activePtyIds), pending: sortedKeys(pendingSessions), subagents: subagents.sort(),
+    });
+    return state + '\u0005' + activity.sort().join('\u0004') + '\u0005' + projectParts.sort().join('\u0005');
+  } catch {
+    return null;
+  }
+}
+
+function renderProjects(projects, resort, { skipIfUnchanged = false } = {}) {
   pruneStaleSubagents();
+  const renderedProjects = projects;
+  if (skipIfUnchanged && !resort && lastSidebarRenderSignature !== null
+      && sidebarRenderSignature(projects) === lastSidebarRenderSignature) return false;
+  lastSidebarRenderSignature = null;
   pendingSubagentRest.clear();
   // see .ai/contexts/session-cache.md ("Remote hosts — busy spinner (issue #242)")
   for (const project of projects) {
@@ -1036,6 +1106,8 @@ function renderProjects(projects, resort) {
   if (activeSessionId && openSessions.has(activeSessionId) && !isUserTyping) {
     openSessions.get(activeSessionId).terminal.focus();
   }
+  lastSidebarRenderSignature = sidebarRenderSignature(renderedProjects);
+  return true;
 }
 
 function rebindSidebarEvents(projects) {