Add spreadsheet-style datasheet tools to Data mode

.claude/skills/app-qa/SKILL.md

@@ -26,6 +26,7 @@ One spec per critical local flow lives in `scripts/screenshot/specs/`:
 | spec | flow (capture prefix) |
 |---|---|
 | `app-shell` | start, data grid, table design, canvas, inline record, SQL query, form builder, preview, close (`app-qa-*`, feeds the docs images in `manifest.mjs`) |
+| `datasheet` | Data mode tools: multi-column sort, filter by selection, find and replace, totals row, hidden and frozen columns, paste from a spreadsheet (`datasheet-*`) |
 | `schema` | create table with keys/FK/unique/check/index, table designer, impact preview, relationship diagram (`schema-*`) |
 | `query` | visual builder with join, aggregate, parameter, filter, sort, preview; generated SQL (`query-*`) |
 | `forms` | generate CRUD forms, keyboard grid resize, validation rule (`forms-*`); Runtime list, create with validation and relationship selector, master/detail, edit (`runtime-*`) |

docs/decisions/README.md

@@ -24,6 +24,7 @@ every risk-retirement spike. This index maps each spike to its record.
 | [Action queries](./action-queries.md) | none. Saved queries that change rows: DuckDB SQL through a writable attach, per-row triggers | accepted |
 | [Access import](./access-import.md) | none. One-way import of Access databases and templates. Format spec in [`docs/access-format.md`](../access-format.md) | accepted |
 | [Data export](./data-export.md) | none. CSV, XLSX and JSON export streamed from DuckDB (PRD Phase 6) | accepted |
+| [Datasheet tools](./datasheet-tools.md) | none. Filter by selection, find and replace, frozen and hidden columns, totals row, spreadsheet paste (PRD Phase 6) | accepted |
 
 The Phase 0 item "signed personalized bundle and envelope-encryption threat
 model" is covered by the [cloud security model](./cloud-security-model.md)

docs/decisions/datasheet-tools.md

@@ -0,0 +1,66 @@
+# Datasheet tools
+
+Status: accepted. Implements the PRD Phase 6 datasheet tools: filter by
+selection, multi-column sort, find and replace, freeze and hide columns, a
+totals row, and multi-cell paste from spreadsheets.
+
+## Context
+
+Access users expect the table datasheet to behave like a spreadsheet. The
+tools have to keep the existing rules: reads go through DuckDB, record writes
+go through `src/lib/records.ts` so triggers fire, and definition edits go
+through the config store.
+
+## Decision
+
+The Data mode grid moved from `DatabaseWorkbench` into `src/data/sheet/`.
+
+- **Filters.** Filter by selection and filter excluding selection turn the
+  focused cell into an `eq`, `ne`, `is_null` or `is_not_null` filter
+  (`filters.ts`). They combine with the search box as one AND list, which the
+  page read, the totals row and export all use. Filters are session state per
+  table and are not saved in the document.
+- **Sort.** Clicking a header sorts by it. Shift-click adds the column as the
+  next sort key (`sorts.ts`). The column menu also sorts.
+- **Find and replace** (`FindReplace.tsx`, `find.ts`) matches the text a cell
+  shows, skipping nulls and blobs. Find next walks the filtered, sorted table
+  one page at a time, wraps once, and moves the grid to the match. Replace all
+  reads every matching row, asks for confirmation, and sends one
+  `writeRecordBatch` with `expected` values.
+- **Hidden and frozen columns and the totals row** are saved per table in
+  `navigationState.datasheetLayouts` through `useDocumentConfig().update`, like
+  the relationship layout (`layout.ts`). Each change is one undo step. Frozen
+  columns are the leftmost visible columns, made `position: sticky` at offsets
+  measured from the rendered headers. Hiding a column never reorders columns.
+- **Totals** come from the `read_table_totals` command
+  (`src-tauri/src/data/totals.rs`). It runs `sum`, `avg`, `count`, `min`,
+  `max`, `stddev_samp` or `var_samp` in DuckDB over the same FROM and WHERE as
+  the page query, so totals cover every filtered row, not only the page. Sum,
+  average, standard deviation and variance need a numeric column.
+- **Paste** (`paste.ts`) takes over a paste that spans more than one cell.
+  It parses tab-separated clipboard text with quoted fields, maps it onto the
+  visible editable columns from the focused cell, and sends one
+  `writeRecordBatch`. Rows that land on existing records update them (an empty
+  cell becomes null). Rows past the last record, or pasted into the new-record
+  row, are inserted (an empty cell keeps the default). A cell that does not
+  parse rejects the whole paste before anything is written.
+
+## Consequences
+
+- Paste and replace all are one batch: a failure writes nothing, and a
+  failing trigger after commit reports a committed write as usual.
+- A paste that starts on existing records and runs past the current page is
+  refused, because the rows after the page are not on screen.
+- Find and replace reads the whole table when replacing all, one page of up
+  to 1000 rows at a time.
+- Filter by selection on text is case-sensitive (DuckDB `=`), unlike the
+  search box, which uses `ILIKE`.
+
+## Evidence
+
+- `src-tauri/src/data/totals_tests.rs`: aggregates over filtered rows, type
+  checks, unknown columns.
+- `tests/unit/datasheet-tools.test.ts`: clipboard parsing, paste plans and
+  writes, find and replace matching, selection filters, layout storage, sort.
+- `tests/integration/datasheet-tools.test.tsx`: each tool end to end against
+  the Rust bridge.

scripts/screenshot/specs/datasheet.spec.tsx

@@ -0,0 +1,158 @@
+import { screen, waitFor, within } from "@testing-library/react";
+import { expect, it, vi } from "vitest";
+import { captureDocument } from "../capture";
+import { createTable, insert, LONG, refreshDatabase, renderNewDocument } from "./fixtures";
+
+const ORDERS: Array<[number, string, string, number, number, string]> = [
+  [1, "Acme Ltd", "North", 4, 120.5, "Open"],
+  [2, "Bolt Ltd", "South", 2, 80, "Shipped"],
+  [3, "Core Inc", "North", 10, 640, "Open"],
+  [4, "Delta Ltd", "East", 1, 15.25, "Cancelled"],
+  [5, "Echo Inc", "South", 7, 210, "Open"],
+  [6, "Fox Ltd", "North", 3, 99.99, "Shipped"],
+];
+
+const names = () =>
+  screen
+    .getAllByRole("textbox", { name: /^customer, row \d+$/ })
+    .map((input) => (input as HTMLInputElement).value);
+
+it("uses the datasheet tools on a table", async () => {
+  const user = await renderNewDocument();
+  await createTable("orders", [
+    { name: "id", declaredType: "INTEGER", primaryKeyPosition: 1 },
+    { name: "customer", declaredType: "TEXT" },
+    { name: "region", declaredType: "TEXT" },
+    { name: "qty", declaredType: "INTEGER" },
+    { name: "amount", declaredType: "REAL" },
+    { name: "status", declaredType: "TEXT" },
+  ]);
+  for (const [id, customer, region, qty, amount, status] of ORDERS)
+    await insert("orders", { id, customer, region, qty, amount, status });
+  await refreshDatabase();
+  await user.click(await screen.findByRole("button", { name: /^orders\b/ }, LONG));
+  await screen.findByRole("textbox", { name: "status, row 6" }, LONG);
+
+  await user.click(screen.getByRole("button", { name: /^region/ }));
+  await user.keyboard("{Shift>}");
+  await user.click(screen.getByRole("button", { name: /^qty/ }));
+  await user.keyboard("{/Shift}");
+  await waitFor(() => expect(names()[0]).toBe("Delta Ltd"), LONG);
+  expect(names().slice(1, 4)).toEqual(["Fox Ltd", "Acme Ltd", "Core Inc"]);
+  await captureDocument(document, {
+    name: "datasheet-01-toolbar-and-multi-sort",
+    expectations: [
+      "A Datasheet tools bar above the grid shows Filter by selection, Filter excluding selection, Find and Totals.",
+      "The region and qty headers show sort arrows with small 1 and 2 markers, and rows are ordered by region then qty.",
+      "Each column header has a small options (⋮) button next to its name.",
+    ],
+  });
+
+  await user.click(screen.getByRole("textbox", { name: "region, row 2" }));
+  await user.click(screen.getByRole("button", { name: "Filter by selection" }));
+  await waitFor(() => expect(names()).toEqual(["Fox Ltd", "Acme Ltd", "Core Inc"]), LONG);
+  expect(screen.getByRole("list", { name: "Active filters" })).toHaveTextContent("region = North");
+  await captureDocument(document, {
+    name: "datasheet-02-filter-by-selection",
+    expectations: [
+      "A green 'region = North' filter chip with a remove (×) button and a 'Clear filters' button sits at the right of the tools bar.",
+      "Only the three North rows are listed and the record count reads 3 records.",
+    ],
+  });
+  await user.click(screen.getByRole("button", { name: "Clear filters" }));
+  await waitFor(() => expect(names()).toHaveLength(6), LONG);
+
+  await user.click(screen.getByRole("button", { name: "Find" }));
+  const panel = screen.getByRole("dialog", { name: "Find and replace" });
+  await user.type(within(panel).getByRole("textbox", { name: "Find" }), "ltd");
+  await user.type(within(panel).getByRole("textbox", { name: "Replace with" }), "LLC");
+  await user.click(within(panel).getByRole("button", { name: "Find next" }));
+  await waitFor(
+    () => expect(screen.getByRole("textbox", { name: "customer, row 1" })).toHaveFocus(),
+    LONG,
+  );
+  await captureDocument(document, {
+    name: "datasheet-03-find",
+    expand: ".data-grid",
+    expectations: [
+      "A Find and replace panel under the tools bar shows Find 'ltd', Replace with 'LLC', Look in, Whole field, Match case, and Find next, Replace and Replace all buttons.",
+      "The first matching cell (customer, row 1, 'Delta Ltd') is focused with a green focus outline.",
+    ],
+  });
+  const confirm = vi.spyOn(window, "confirm").mockReturnValue(true);
+  await user.click(within(panel).getByRole("button", { name: "Replace all" }));
+  expect(await within(panel).findByRole("status", {}, LONG)).toHaveTextContent(
+    "Replaced 4 matches in 4 records.",
+  );
+  await waitFor(() => expect(names()).toContain("Acme LLC"), LONG);
+  confirm.mockRestore();
+  await captureDocument(document, {
+    name: "datasheet-04-replace-all",
+    expand: ".data-grid",
+    expectations: [
+      "The panel reports 'Replaced 4 matches in 4 records.'",
+      "Every customer that ended in 'Ltd' now ends in 'LLC'; 'Core Inc' and 'Echo Inc' are unchanged.",
+    ],
+  });
+  await user.click(within(panel).getByRole("button", { name: "Close find and replace" }));
+
+  await user.click(screen.getByRole("button", { name: "Totals" }));
+  await user.selectOptions(
+    await screen.findByRole("combobox", { name: "Total for qty" }, LONG),
+    "sum",
+  );
+  await user.selectOptions(screen.getByRole("combobox", { name: "Total for amount" }), "avg");
+  const totalCell = (column: string) =>
+    within(screen.getByRole("combobox", { name: `Total for ${column}` }).closest("td")!);
+  await waitFor(() => expect(totalCell("qty").getByRole("status")).toHaveTextContent("27"), LONG);
+  await waitFor(
+    () => expect(totalCell("amount").getByRole("status")).toHaveTextContent("194.29"),
+    LONG,
+  );
+  await captureDocument(document, {
+    name: "datasheet-05-totals-row",
+    expand: ".data-grid",
+    expectations: [
+      "A grey Total row sits under the last record with a picker in every column.",
+      "qty shows Sum 27 and amount shows Average 194.29; other columns read None.",
+      "The Totals button in the tools bar is highlighted as pressed.",
+    ],
+  });
+
+  await user.click(screen.getByRole("button", { name: "Column options for status" }));
+  await user.click(await screen.findByRole("menuitem", { name: "Hide column" }));
+  await waitFor(
+    () => expect(screen.queryByRole("textbox", { name: "status, row 1" })).toBeNull(),
+    LONG,
+  );
+  await user.click(screen.getByRole("button", { name: "Column options for customer" }));
+  await user.click(await screen.findByRole("menuitem", { name: "Freeze through this column" }));
+  const header = (column: string) =>
+    screen.getByRole("button", { name: `Column options for ${column}` }).closest("th")!;
+  await waitFor(() => expect(header("customer")).toHaveClass("frozen", "frozen-edge"), LONG);
+  expect(header("id")).toHaveClass("frozen");
+  expect(header("region")).not.toHaveClass("frozen");
+  await captureDocument(document, {
+    name: "datasheet-06-hidden-and-frozen",
+    expand: ".data-grid",
+    expectations: [
+      "The status column is gone and the tools bar shows a '1 hidden' button and an 'Unfreeze columns' button.",
+      "The id and customer columns are frozen, with a darker divider line after customer.",
+    ],
+  });
+
+  await user.click(screen.getByRole("textbox", { name: "New id" }));
+  await user.paste("7\tGolf LLC\tWest\t5\t50\r\n8\tHotel Inc\tWest\t2\t20.5\r\n");
+  expect(await screen.findByText("Added 2 records.", {}, LONG)).toHaveAttribute("role", "status");
+  await screen.findByRole("textbox", { name: "customer, row 8" }, LONG);
+  await waitFor(() => expect(totalCell("qty").getByRole("status")).toHaveTextContent("34"), LONG);
+  await captureDocument(document, {
+    name: "datasheet-07-paste-from-spreadsheet",
+    expand: ".data-grid",
+    expectations: [
+      "A notice under the tools bar reads 'Added 2 records.'",
+      "Rows 7 and 8 (Golf LLC and Hotel Inc, region West) appear at the end of the grid.",
+      "The totals row updates to qty Sum 34.",
+    ],
+  });
+});

src-tauri/src/data/mod.rs

@@ -25,6 +25,9 @@ pub mod sqltext;
 pub mod support;
 #[cfg(test)]
 mod tests;
+pub mod totals;
+#[cfg(test)]
+mod totals_tests;
 pub mod values;
 pub mod write;
 

src-tauri/src/data/totals.rs

@@ -0,0 +1,118 @@
+//! Datasheet totals row: one aggregate per column over every filtered row.
+use super::{duck_value, q, read::ReadRuntime, DataValue, Filter, LogicalType};
+use serde::{Deserialize, Serialize};
+
+/// The aggregates a datasheet totals row offers (Access's Total row).
+#[derive(Debug, Clone, Copy, Deserialize, Serialize, PartialEq)]
+#[serde(rename_all = "snake_case")]
+pub enum TotalFunction {
+    Sum,
+    Avg,
+    Count,
+    Min,
+    Max,
+    Stdev,
+    Var,
+}
+
+#[derive(Debug, Clone, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct TotalSpec {
+    pub column: String,
+    pub function: TotalFunction,
+}
+
+fn numeric(t: &LogicalType) -> bool {
+    matches!(
+        t,
+        LogicalType::Integer | LogicalType::Real | LogicalType::Decimal { .. }
+    )
+}
+
+fn orderable(t: &LogicalType) -> bool {
+    !matches!(
+        t,
+        LogicalType::Blob | LogicalType::Json | LogicalType::Boolean
+    )
+}
+
+impl ReadRuntime {
+    /// One value per spec, aggregated over the rows `filters` select. Sum,
+    /// average, standard deviation and variance need a numeric column; min and
+    /// max keep the column's type; count counts non-null values.
+    pub fn totals(
+        &self,
+        table: &str,
+        filters: &[Filter],
+        specs: &[TotalSpec],
+    ) -> Result<Vec<DataValue>, String> {
+        if specs.is_empty() {
+            return Ok(vec![]);
+        }
+        let _gate = self.read_gate()?;
+        let plan = self.page_plan(table, &[], filters)?;
+        let mut exprs = vec![];
+        for spec in specs {
+            let column = plan
+                .columns
+                .iter()
+                .find(|c| c.name == spec.column)
+                .ok_or_else(|| format!("Unknown column {:?}", spec.column))?;
+            let t = &column.logical_type;
+            let col = match t {
+                LogicalType::Uuid => format!("CAST({} AS VARCHAR)", q(&column.name)),
+                _ => q(&column.name),
+            };
+            let allowed = match spec.function {
+                TotalFunction::Count => true,
+                TotalFunction::Min | TotalFunction::Max => orderable(t),
+                _ => numeric(t),
+            };
+            if !allowed {
+                return Err(format!(
+                    "{:?} total is not available for {} column {:?}",
+                    spec.function, t, column.name
+                ));
+            }
+            exprs.push(match spec.function {
+                TotalFunction::Sum => format!("sum({col})"),
+                TotalFunction::Avg => format!("avg({col})"),
+                TotalFunction::Count => format!("count({col})"),
+                TotalFunction::Min => format!("min({col})"),
+                TotalFunction::Max => format!("max({col})"),
+                TotalFunction::Stdev => format!("stddev_samp({col})"),
+                TotalFunction::Var => format!("var_samp({col})"),
+            });
+        }
+        // The count query is `SELECT count(*) FROM <source> WHERE <filters>`: aggregate over its FROM.
+        let from_where = plan
+            .count_sql
+            .strip_prefix("SELECT count(*) ")
+            .ok_or("Unexpected page query shape")?;
+        let sql = format!("SELECT {} {from_where}", exprs.join(", "));
+        let raw = self
+            .connection
+            .query_row(&sql, duckdb::params_from_iter(plan.binds.iter()), |r| {
+                (0..specs.len())
+                    .map(|i| r.get::<_, duckdb::types::Value>(i))
+                    .collect::<duckdb::Result<Vec<_>>>()
+            })
+            .map_err(|e| e.to_string())?;
+        Ok(raw
+            .into_iter()
+            .zip(specs)
+            .map(|(v, spec)| {
+                let v = duck_value(v);
+                match spec.function {
+                    TotalFunction::Min | TotalFunction::Max => plan
+                        .columns
+                        .iter()
+                        .find(|c| c.name == spec.column)
+                        .map(|c| c.logical_type.coerce_read(v.clone()))
+                        .unwrap_or(v),
+                    _ => v,
+                }
+            })
+            .collect())
+    }
+}