diff --git a/docs/decisions/access-import.md b/docs/decisions/access-import.md index e7b6867e..ab0e0d4e 100644 --- a/docs/decisions/access-import.md +++ b/docs/decisions/access-import.md @@ -64,6 +64,14 @@ staged copy of the data, in this order: (`translate::dml`). A make-table target the file lacks is created by migration 002 once the data is in. Data-definition and pass-through queries keep their Access SQL in `settings.accessImport.actionQueries`. + Built-ins without a DuckDB function of the same name (`StrConv`, + `MonthName`, `FormatCurrency`, `Partition`, `InStrRev`, `Eval` of a + literal and others) are written out in `translate::builtins`, and custom + `Format` patterns become `strftime` or `format` calls there. Domain + functions (`translate::domain`) whose criteria are built from the row, + such as `DLookup("Price", "Products", "ID=" & [ProductID])`, become + correlated subqueries: the text pieces are parsed as one WHERE clause and + each computed piece is the outer SQL, not text. 4. Reports (`reports.rs`), macros (`macros.rs`), then forms (`forms.rs`, `controls.rs`) convert, in that order so buttons only point at objects that exist. Expressions go through `translate::expr` into the ixtable expression @@ -85,6 +93,17 @@ Settings › Assets previews. The conversion report, with a status and notes for every object, is stored in `settings.accessImport.report` and returned to the wizard. +**Migration report.** Every report item names the object it became +(`target`: form, report, action or query, matched by name), and the +document keeps the items, reader warnings and import time in +`settings.accessImport`. Settings › Access migration, shown only for +imported documents, lists them by status, kind and text, opens the +converted object, and records review marks in +`settings.accessImport.reviewed` (`kind:name` keys) through the config store, +so they save and undo like any edit. It exports Markdown or CSV, rendered in +`src/access/migration.ts` and written by `write_access_report`, which accepts +only `.md`, `.csv` and `.txt` paths and refuses restricted runtime roles. + **Commands and UI.** `inspect_access_file` returns an inventory (tables, row counts, objects, compiled objects, warnings) without changing anything. `import_access_file` runs the conversion. Both run on a blocking thread. The @@ -102,7 +121,11 @@ whether to import data, import, read the report, open the document. - Access SQL is translated, not emulated. A call to a VBA function defined in a module yields an empty column, with a note. A built-in function with no DuckDB equivalent makes the query fail translation, and the report names - it. + it. `Format` follows Access for dates and numbers in US English; the + weekday and month names are English whatever the user's locale. +- A domain function's criteria must be text pieces joined with values. A + criteria held in a variable or built by a VBA function still fails, and the + report names the query. - Text comparison in translated queries is case-insensitive, as in Access, through `lower()`. This costs index use in large tables. - An Access rule the data already breaks is not enforced after import. The @@ -115,14 +138,16 @@ whether to import data, import, read the report, open the document. - `src-tauri/src/access/tests/`: SaveAsText parsing (`text_format.rs`), template reading and the fixture package in `tests/fixtures/access/template/` - (`accdt.rs`), page-file decoding of `orders.accdb`, `orders.mdb`, and the + (`accdt.rs`), the wider SQL and expression coverage run in DuckDB and + through `src/expr` (`translate_coverage.rs`, + `tests/unit/access-expressions.test.ts`), page-file decoding of `orders.accdb`, `orders.mdb`, and the Jackcess `complex-data.accdb`, and refusal of password-protected and encrypted copies of the fixtures (`jet.rs`), SQL and expression translation (`translate.rs`), and full conversions into working documents (`convert.rs`). -- `tests/unit/access-summary.test.ts` and - `tests/integration/access-import.test.tsx`: the wizard against the real - command through the test bridge. +- `tests/unit/access-summary.test.ts`, `tests/unit/access-migration.test.ts` + and `tests/integration/access-import.test.tsx`: the wizard and the + migration tab against the real commands through the test bridge. - Optional corpus tests: `node scripts/access/fetch-templates.mjs ` downloads the 30 featured templates, and `IXTABLE_ACCESS_TEMPLATES_DIR= cargo test --lib access::` imports each one and fails on any config validation error. @@ -143,3 +168,8 @@ whether to import data, import, read the report, open the document. action-queries record). The VBA asset has a preview in Settings › Assets. - 2026-10-09: password-protected and encrypted files moved out of scope. Jet 3/4 page decryption removed, Jet database passwords now refused. +- 2026-10-10: wider query and expression coverage (built-ins, `Format` + patterns, row-based domain criteria, `Like` search boxes, `Switch`, + `Choose`), and the migration report tab with links, review marks and + export. The template corpus was not re-run: its host is unreachable from + the cloud build environment. diff --git a/scripts/screenshot/specs/access-import.spec.tsx b/scripts/screenshot/specs/access-import.spec.tsx index 3ab97c41..dc20f6b0 100644 --- a/scripts/screenshot/specs/access-import.spec.tsx +++ b/scripts/screenshot/specs/access-import.spec.tsx @@ -31,7 +31,7 @@ it("imports an Access template with its forms, report, and data", async () => { await captureDocument(document, { name: "access-02-report", expectations: [ - "The report says how many tables and rows the new document has.", + "The report says how many tables and rows the new document has, and that Settings › Access migration keeps it.", "A table counts converted, partly converted, and not converted objects per kind.", "The expanded notes explain each loss, such as the VBA module kept as an asset.", ], @@ -79,4 +79,50 @@ it("imports an Access template with its forms, report, and data", async () => { "Preview changes reports how many rows it would update, and that nothing was changed.", ], }); + + await user.click(screen.getByRole("button", { name: "Settings" })); + await user.click(await screen.findByRole("tab", { name: "Access migration" }, LONG)); + const panel = await screen.findByRole("tabpanel"); + expect(within(panel).getByRole("table", { name: "Migration items" })).toBeVisible(); + expect(within(panel).getByText(/Imported from Order Desk\.accdt, an Access template/)).toBeInTheDocument(); + await captureDocument(document, { + name: "access-06-migration", + expectations: [ + "Settings has an Access migration tab naming the source file and how many items still need review.", + "A summary table counts converted, partly converted and not converted objects per kind.", + "The item list shows only what needs attention, worst first, with notes, Open links and Reviewed checkboxes.", + ], + }); + + const summary = within(panel).getByText(/still needs? review/); + const before = summary.textContent; + await user.selectOptions(within(panel).getByRole("combobox", { name: /Show/ }), "all"); + await user.selectOptions(within(panel).getByRole("combobox", { name: /Kind/ }), "form"); + const forms = within(panel).getByRole("table", { name: "Migration items" }); + const box = within(forms) + .getAllByRole("checkbox", { name: /^Reviewed form / }) + .find((b) => !b.closest("tr")?.textContent?.includes("Converted"))!; + await user.click(box); + await waitFor(() => expect(summary.textContent).not.toBe(before)); + expect(box).toBeChecked(); + await captureDocument(document, { + name: "access-07-migration-filtered", + expectations: [ + "Show is set to Everything and Kind to Forms, so only forms are listed, converted ones included.", + "The form marked reviewed has its checkbox ticked and the review count above went down by one.", + "Each converted form has an Open form link.", + ], + }); + + const open = within(forms).getAllByRole("button", { name: /^Open form / })[0]; + const name = open.getAttribute("aria-label")!.replace("Open form ", ""); + await user.click(open); + await screen.findByDisplayValue(name, {}, LONG); + await captureDocument(document, { + name: "access-08-open-form", + expectations: [ + "Open form switched to Design mode with the converted Access form selected.", + "The form's name field shows the Access form's name.", + ], + }); }); diff --git a/src-tauri/src/access/convert/mod.rs b/src-tauri/src/access/convert/mod.rs index 44f620d4..bb604540 100644 --- a/src-tauri/src/access/convert/mod.rs +++ b/src-tauri/src/access/convert/mod.rs @@ -373,8 +373,12 @@ pub fn create( check_queries(m, window, &config, &mut report); actions::check_statements(m, window, &config, &mut report)?; // The report stays with the document (Settings › YAML shows it). + report.link_targets(&config); config.settings["accessImport"]["report"] = serde_json::to_value(&report.items).unwrap_or_default(); + config.settings["accessImport"]["warnings"] = serde_json::json!(report.warnings); + config.settings["accessImport"]["importedAt"] = + serde_json::json!(chrono::Utc::now().format("%Y-%m-%dT%H:%M:%SZ").to_string()); m.update_config(window, config) })(); match result { diff --git a/src-tauri/src/access/convert/report.rs b/src-tauri/src/access/convert/report.rs index 7d9ae06f..ee04f700 100644 --- a/src-tauri/src/access/convert/report.rs +++ b/src-tauri/src/access/convert/report.rs @@ -20,6 +20,16 @@ pub struct Item { pub name: String, pub status: Status, pub notes: Vec, + /// The ixtable object it became, for links from the migration report. + #[serde(skip_serializing_if = "Option::is_none")] + pub target: Option, +} + +/// An object of the document: `kind` is form, report, action or query. +#[derive(Debug, Clone, PartialEq, Serialize)] +pub struct Target { + pub kind: &'static str, + pub id: String, } #[derive(Debug, Default, Clone, Serialize)] @@ -41,9 +51,70 @@ impl ImportReport { name: name.into(), status, notes, + target: None, }); } + /// Points each item at the object it became, matched by name. + pub fn link_targets(&mut self, config: &crate::archive::DocumentConfig) { + let find = |list: Vec<(&str, &str)>, name: &str| { + list.into_iter() + .find(|(n, _)| n.eq_ignore_ascii_case(name)) + .map(|(_, id)| id.to_string()) + }; + for item in &mut self.items { + let (kind, id) = match item.kind.as_str() { + "form" => ( + "form", + find( + config + .design + .forms + .iter() + .map(|f| (f.name.as_str(), f.id.as_str())) + .collect(), + &item.name, + ), + ), + "report" => ( + "report", + find( + config + .reports + .iter() + .map(|r| (r.name.as_str(), r.id.as_str())) + .collect(), + &item.name, + ), + ), + "macro" => ( + "action", + find( + config + .actions + .iter() + .map(|a| (a.name.as_str(), a.id.as_str())) + .collect(), + &item.name, + ), + ), + "query" => ( + "query", + find( + config + .saved_queries + .iter() + .map(|q| (q.name.as_str(), q.id.as_str())) + .collect(), + &item.name, + ), + ), + _ => continue, + }; + item.target = id.map(|id| Target { kind, id }); + } + } + /// Adds a note to an existing item (and downgrades a converted item to partial). pub fn note(&mut self, kind: &str, name: &str, note: impl Into) { let note = note.into(); diff --git a/src-tauri/src/access/mod.rs b/src-tauri/src/access/mod.rs index 62c9a06d..8d697ab1 100644 --- a/src-tauri/src/access/mod.rs +++ b/src-tauri/src/access/mod.rs @@ -147,5 +147,32 @@ pub async fn import_access_file( .await } +/// Saves the migration report text the Studio renders (Markdown or CSV). +#[tauri::command] +pub fn write_access_report( + window_label: String, + path: String, + text: String, +) -> Result<(), AppError> { + crate::manager()?.state(&window_label)?; + crate::authz::require_unrestricted(&window_label, "export the migration report")?; + write_report_text(Path::new(&path), &text) +} + +/// Writes report text to a `.md`, `.csv` or `.txt` file. +pub fn write_report_text(path: &Path, text: &str) -> Result<(), AppError> { + let ext = path + .extension() + .map(|e| e.to_string_lossy().to_ascii_lowercase()) + .unwrap_or_default(); + if !["md", "csv", "txt"].contains(&ext.as_str()) { + return Err(AppError::new( + "VALIDATION_ERROR", + "the migration report is saved as a .md, .csv or .txt file", + )); + } + std::fs::write(path, text).map_err(|e| AppError::new("IO_ERROR", e)) +} + #[cfg(test)] mod tests; diff --git a/src-tauri/src/access/tests/convert.rs b/src-tauri/src/access/tests/convert.rs index bddd5856..0e4bff25 100644 --- a/src-tauri/src/access/tests/convert.rs +++ b/src-tauri/src/access/tests/convert.rs @@ -400,7 +400,18 @@ fn binary_databases_get_generated_forms_and_working_queries() { .iter() .find(|i| i.name == "Unpaid Orders") .unwrap(); - assert_eq!(unpaid.status, Status::Skipped); + // DLookup with criteria built from the row is a correlated subquery. + assert_eq!(unpaid.status, Status::Converted, "{:?}", unpaid.notes); + let unpaid = config + .saved_queries + .iter() + .find(|q| q.name == "Unpaid Orders") + .unwrap(); + rows( + m, + &window, + &format!("SELECT count(*) FROM ({})", unpaid.sql), + ); m.close(&window, true).unwrap(); } } @@ -589,3 +600,45 @@ fn action_queries_convert_and_run(m: &DocumentManager, config: &crate::archive:: assert!(!notes.contains("DuckDB rejects"), "{notes}"); } } + +#[test] +fn migration_report_links_objects_and_saves_as_text() { + let path = super::fixtures::template(); + let (report, errs) = import(&path, "mig"); + assert_eq!(errs, Vec::::new()); + let config = manager().config("mig").unwrap(); + let stored = &config.settings["accessImport"]; + assert!(stored["importedAt"] + .as_str() + .is_some_and(|t| t.ends_with('Z'))); + assert!(stored["warnings"].is_array()); + // Forms, reports, macros and queries point at what they became. + for item in report.items.iter().filter(|i| i.status != Status::Skipped) { + let Some(target) = &item.target else { + assert!( + !["form", "report", "query"].contains(&item.kind.as_str()), + "{item:?}" + ); + continue; + }; + let found = match target.kind { + "form" => config.design.forms.iter().any(|f| f.id == target.id), + "report" => config.reports.iter().any(|r| r.id == target.id), + "action" => config.actions.iter().any(|a| a.id == target.id), + _ => config.saved_queries.iter().any(|q| q.id == target.id), + }; + assert!(found, "{item:?}"); + } + let linked = stored["report"].as_array().unwrap(); + assert!(linked.iter().any(|i| i["target"]["kind"] == "form")); + let dir = std::env::temp_dir().join(format!("ixtable-mig-{}", uuid::Uuid::new_v4())); + std::fs::create_dir_all(&dir).unwrap(); + crate::access::write_report_text(&dir.join("report.md"), "# Report").unwrap(); + assert_eq!( + std::fs::read_to_string(dir.join("report.md")).unwrap(), + "# Report" + ); + let err = crate::access::write_report_text(&dir.join("report.exe"), "x").unwrap_err(); + assert_eq!(err.code, "VALIDATION_ERROR"); + manager().close("mig", true).unwrap(); +} diff --git a/src-tauri/src/access/tests/mod.rs b/src-tauri/src/access/tests/mod.rs index 8a3ef5c5..ca173fab 100644 --- a/src-tauri/src/access/tests/mod.rs +++ b/src-tauri/src/access/tests/mod.rs @@ -4,3 +4,4 @@ mod fixtures; mod jet; mod text_format; mod translate; +mod translate_coverage; diff --git a/src-tauri/src/access/tests/translate.rs b/src-tauri/src/access/tests/translate.rs index 54ab0b54..4c775383 100644 --- a/src-tauri/src/access/tests/translate.rs +++ b/src-tauri/src/access/tests/translate.rs @@ -5,7 +5,7 @@ use crate::access::translate::expr::{translate, Target}; use crate::access::translate::format::format_pattern; use crate::access::translate::sql::{Dialect, Kind, Schema, SqlWriter}; -struct Fixture; +pub(super) struct Fixture; impl Schema for Fixture { fn columns(&self, source: &str) -> Option> { @@ -99,7 +99,7 @@ fn functions_and_date_arithmetic() { let sql = duck("SELECT IIf(IsNull(Status),Nz(Amount),Len(Status)) AS a, DateAdd(\"m\",1,[Order Date]) AS b, DateDiff(\"d\",[Order Date],Date()) AS c, [Order Date]+7 AS d, Format([Order Date],\"yyyy\") AS e, Int(Now()) AS f FROM Orders"); assert_eq!( sql, - "SELECT CASE WHEN (\"Status\" IS NULL) THEN coalesce(\"Amount\", 0) ELSE length(\"Status\") END AS \"a\", (\"Order Date\" + to_months(CAST(1 AS INTEGER))) AS \"b\", date_diff('day', \"Order Date\", CAST(CAST(now() AS TIMESTAMP) AS DATE)) AS \"c\", (CAST(\"Order Date\" AS TIMESTAMP) + to_seconds(CAST(round((7) * 86400) AS BIGINT))) AS \"d\", strftime(\"Order Date\", '%Y') AS \"e\", CAST(CAST(now() AS TIMESTAMP) AS DATE) AS \"f\" FROM \"Orders\"" + "SELECT CASE WHEN (\"Status\" IS NULL) THEN coalesce(\"Amount\", 0) ELSE length(\"Status\") END AS \"a\", (\"Order Date\" + to_months(CAST(1 AS INTEGER))) AS \"b\", date_diff('day', \"Order Date\", CAST(CAST(now() AS TIMESTAMP) AS DATE)) AS \"c\", (CAST(\"Order Date\" AS TIMESTAMP) + to_seconds(CAST(round((7) * 86400) AS BIGINT))) AS \"d\", strftime(CAST(\"Order Date\" AS TIMESTAMP), '%Y') AS \"e\", CAST(CAST(now() AS TIMESTAMP) AS DATE) AS \"f\" FROM \"Orders\"" ); assert_eq!( duck("SELECT DLookup(\"Company\",\"Customers\",\"ID=1\") AS c FROM Orders"), diff --git a/src-tauri/src/access/tests/translate_coverage.rs b/src-tauri/src/access/tests/translate_coverage.rs new file mode 100644 index 00000000..e27907fb --- /dev/null +++ b/src-tauri/src/access/tests/translate_coverage.rs @@ -0,0 +1,216 @@ +//! Wider Access SQL coverage: built-in functions, `Format` patterns and +//! domain functions with criteria built from the row, each run in DuckDB. +use super::translate::Fixture; +use crate::access::translate::ast::parse_statement; +use crate::access::translate::builtins::{number_spec, strftime_pattern}; +use crate::access::translate::sql::{Dialect, SqlWriter}; + +fn db() -> duckdb::Connection { + let db = duckdb::Connection::open_in_memory().unwrap(); + db.execute_batch( + "CREATE TABLE orders(\"ID\" INTEGER PRIMARY KEY, \"Customer\" INTEGER, \"Order Date\" TIMESTAMP, \"Amount\" DOUBLE, \"Paid\" BOOLEAN, \"Status\" VARCHAR); + CREATE TABLE customers(\"ID\" INTEGER PRIMARY KEY, \"Company\" VARCHAR, \"Logo\" BLOB); + INSERT INTO customers VALUES (1, 'Contoso', NULL), (2, 'O''Brien & Co', NULL); + INSERT INTO orders VALUES (1, 1, TIMESTAMP '2024-03-05 14:07:09', 1234.5, false, 'open'), + (2, 2, TIMESTAMP '2024-11-20 08:00:00', 0.25, true, 'closed'), + (3, 2, TIMESTAMP '2024-12-01 09:30:00', 40, false, 'open');", + ) + .unwrap(); + db +} + +fn translate(access: &str) -> String { + let st = parse_statement(access).unwrap_or_else(|e| panic!("{access}: {e}")); + let mut w = SqlWriter::new(Dialect::DuckDb, &Fixture); + w.statement(&st).unwrap_or_else(|e| panic!("{access}: {e}")) +} + +fn untranslated(access: &str) -> String { + let st = parse_statement(access).unwrap(); + SqlWriter::new(Dialect::DuckDb, &Fixture) + .statement(&st) + .unwrap_err() +} + +/// The value of an expression for order 1, as text. +fn value(db: &duckdb::Connection, expr: &str) -> Option { + let sql = translate(&format!("SELECT {expr} AS V FROM Orders WHERE ID = 1")); + db.query_row( + &format!("SELECT CAST(\"V\" AS VARCHAR) FROM ({sql})"), + [], + |r| r.get(0), + ) + .unwrap_or_else(|e| panic!("{expr} → {sql}: {e}")) +} + +fn column(db: &duckdb::Connection, access: &str) -> Vec> { + let sql = translate(access); + let mut st = db + .prepare(&format!( + "SELECT CAST(\"V\" AS VARCHAR) FROM ({sql}) ORDER BY \"ID\"" + )) + .unwrap_or_else(|e| panic!("{sql}: {e}")); + st.query_map([], |r| r.get(0)) + .unwrap() + .map(Result::unwrap) + .collect() +} + +#[test] +fn text_number_and_date_builtins_run_in_duckdb() { + let db = db(); + let cases = [ + ("StrConv(\"hello big world\", 3)", "Hello Big World"), + ("StrConv(Status, 1)", "OPEN"), + ("StrConv(\"ABC\", 2)", "abc"), + ("MonthName(3)", "March"), + ("MonthName(Month([Order Date]), True)", "Mar"), + ("WeekdayName(1)", "Sunday"), + ("WeekdayName(Weekday([Order Date]), True)", "Tue"), + ("FormatCurrency(Amount)", "$1,234.50"), + ("FormatNumber(Amount, 0)", "1,235"), + ("FormatPercent(0.256, 1)", "25.6%"), + ("FormatDateTime([Order Date], 2)", "3/5/2024"), + ("FormatDateTime([Order Date], 4)", "14:07"), + ("Partition(Amount, 0, 5000, 1000)", "1000:1999"), + ("Partition(15, 0, 99, 10)", " 10: 19"), + ("Partition(150, 0, 99, 10)", "100: "), + ("Hex(255)", "FF"), + ("Oct(8)", "10"), + ("InStrRev(\"a-b-c\", \"-\")", "4"), + ("InStrRev(\"abc\", \"x\")", "0"), + ("StrReverse(\"abc\")", "cba"), + ("StrComp(\"abc\", \"ABC\")", "0"), + ("StrComp(\"abc\", \"abd\")", "-1"), + ("StrComp(\"b\", \"a\", 0)", "1"), + ("TimeSerial(14, 30, 5)", "14:30:05"), + ("Round(Atn(1) * 4, 4)", "3.1416"), + ("Eval(\"1 + 2\")", "3"), + ]; + for (expr, expected) in cases { + assert_eq!(value(&db, expr).as_deref(), Some(expected), "{expr}"); + } + let r: f64 = value(&db, "Rnd()").unwrap().parse().unwrap(); + assert!((0.0..1.0).contains(&r)); + assert!(untranslated("SELECT StrConv(Status, 64) FROM Orders").contains("StrConv")); +} + +#[test] +fn format_patterns_run_in_duckdb() { + let db = db(); + let cases = [ + ("Format([Order Date], \"yyyy-mm-dd\")", "2024-03-05"), + ("Format([Order Date], \"mmm d, yyyy\")", "Mar 5, 2024"), + ("Format([Order Date], \"dddd\")", "Tuesday"), + ("Format([Order Date], \"hh:nn:ss\")", "14:07:09"), + ("Format([Order Date], \"h:mm AM/PM\")", "2:07 PM"), + ("Format([Order Date], \"mmmm yyyy\")", "March 2024"), + ( + "Format([Order Date], \"\\Q\\t\\r \"\"x\"\" yy\")", + "Qtr x 24", + ), + ("Format([Order Date], \"Long Time\")", "2:07:09 PM"), + ("Format(Amount, \"#,##0.00\")", "1,234.50"), + ("Format(Amount, \"$#,##0\")", "$1,235"), + ("Format(Amount, \"0.0\")", "1234.5"), + ("Format(ID, \"0000\")", "0001"), + ("Format(0.125, \"0.0%\")", "12.5%"), + ("Format(0.5, \"#.00\")", ".50"), + ("Format(ID, \"\"\"No. \"\"0\")", "No. 1"), + ("Format(Paid, \"Yes/No\")", "No"), + ("Format(Status, \">\")", "OPEN"), + ("Format(Amount, \"Standard\")", "1,234.50"), + ("Format(Amount)", "1234.5"), + ]; + for (expr, expected) in cases { + assert_eq!(value(&db, expr).as_deref(), Some(expected), "{expr}"); + } + assert_eq!( + strftime_pattern("m/d/yy h:nn").unwrap(), + "%-m/%-d/%y %-H:%M" + ); + assert_eq!(strftime_pattern("hh:mm").unwrap(), "%H:%M"); + assert_eq!(strftime_pattern("mm:ss").unwrap(), "%M:%S"); + assert_eq!(number_spec("$#,##0.00").decimals, 2); + assert!( + untranslated("SELECT Format([Order Date], \"q\\/yyyy\") FROM Orders").contains("quarter") + ); +} + +#[test] +fn domain_functions_with_criteria_from_the_row_are_correlated() { + let db = db(); + assert_eq!( + column( + &db, + "SELECT ID, DLookUp(\"Company\", \"Customers\", \"ID=\" & [Customer]) AS V FROM Orders" + ), + [ + Some("Contoso".into()), + Some("O'Brien & Co".into()), + Some("O'Brien & Co".into()) + ] + ); + // Quotes around the value, and the domain is the query's own table. + assert_eq!( + column(&db, "SELECT ID, DCount(\"*\", \"Orders\", \"Status='\" & [Status] & \"'\") AS V FROM Orders"), + [Some("2".into()), Some("1".into()), Some("2".into())] + ); + assert_eq!( + column(&db, "SELECT O.ID, DSum(\"Amount\", \"Orders\", \"Customer = \" & O.Customer & \" AND ID <> \" & O.ID) AS V FROM Orders AS O"), + [None, Some("40.0".into()), Some("0.25".into())] + ); + assert_eq!( + column(&db, "SELECT ID, DMax(\"[Order Date]\", \"Orders\", \"[Order Date] < #\" & [Order Date] & \"#\") AS V FROM Orders"), + [None, Some("2024-03-05 14:07:09".into()), Some("2024-11-20 08:00:00".into())] + ); + // A text value with a quote in it stays one value. + assert_eq!( + column(&db, "SELECT ID, DCount(\"*\", \"Customers\", \"Company = '\" & DLookup(\"Company\", \"Customers\", \"ID=\" & [Customer]) & \"'\") AS V FROM Orders"), + [Some("1".into()), Some("1".into()), Some("1".into())] + ); + // DCount with a literal criteria (was translated as a VBA function before). + assert_eq!( + value(&db, "DCount(\"ID\", \"Orders\", \"Paid = False\")").as_deref(), + Some("2") + ); +} + +/// Form and report expressions; tests/unit/access-expressions.test.ts +/// evaluates the same ixtable expressions. +#[test] +fn more_access_expressions_become_ixtable_expressions() { + use crate::access::translate::expr::{field, translate, Target}; + let fields = |n: &str| { + ["Amount", "Due", "Status", "Find", "Priority"] + .iter() + .find(|f| f.eq_ignore_ascii_case(n)) + .map(|f| field("record", f)) + }; + let form = + |e: &str| translate(e, Target::Form, &fields).unwrap_or_else(|err| panic!("{e}: {err}")); + let cases = [ + ("[Status] Like \"*\" & [Find] & \"*\"", "contains(record.Status, record.Find)"), + ("[Status] Like [Find] & \"*\"", "startswith(record.Status, record.Find)"), + ("[Status] Like \"*\" & [Find]", "endswith(record.Status, record.Find)"), + ("StrConv([Status], 1)", "upper(record.Status)"), + ("MonthName(Month([Due]))", "format(date(2000, month(record.Due), 1), 'MMMM')"), + ("MonthName(3, True)", "format(date(2000, 3, 1), 'MMM')"), + ("DatePart(\"q\", [Due])", "floor((month(record.Due) - 1) / 3) + 1"), + ("DatePart(\"yyyy\", [Due])", "year(record.Due)"), + ( + "Switch([Amount] > 100, \"High\", [Amount] > 10, \"Mid\", True, \"Low\")", + "if(record.Amount > 100, 'High', if(record.Amount > 10, 'Mid', if(true, 'Low', null)))", + ), + ( + "Choose([Priority], \"Low\", \"Normal\", \"High\")", + "if(record.Priority = 1, 'Low', if(record.Priority = 2, 'Normal', if(record.Priority = 3, 'High', null)))", + ), + ("Sgn([Amount])", "if(record.Amount > 0, 1, if(record.Amount < 0, -1, 0))"), + ]; + for (access, expected) in cases { + assert_eq!(form(access), expected, "{access}"); + } + assert!(translate("[Status] Like [Find]", Target::Form, &fields).is_err()); + assert!(translate("StrConv([Status], 3)", Target::Form, &fields).is_err()); +} diff --git a/src-tauri/src/access/translate/builtins.rs b/src-tauri/src/access/translate/builtins.rs new file mode 100644 index 00000000..1058c499 --- /dev/null +++ b/src-tauri/src/access/translate/builtins.rs @@ -0,0 +1,510 @@ +//! Access built-in functions in DuckDB SQL beyond the core set in +//! `functions.rs`: text case and names, number and date formatting, +//! `Partition`, and the less common math and string functions. +use super::ast::*; +use super::sql::{string_literal, SqlWriter}; + +fn arg_int(args: &[Expr], i: usize) -> Option { + match args.get(i) { + Some(Expr::Num(n)) => n.parse().ok(), + Some(Expr::Neg(x)) => match &**x { + Expr::Num(n) => n.parse::().ok().map(|v| -v), + _ => None, + }, + Some(Expr::Bool(b)) => Some(if *b { -1 } else { 0 }), + _ => None, + } +} + +/// True when the argument is absent or a literal false/0. +fn is_false(args: &[Expr], i: usize) -> bool { + args.get(i).is_none() || arg_int(args, i) == Some(0) +} + +/// Upper-cases the first letter of each word. +fn proper(s: &str) -> String { + format!("array_to_string(list_transform(string_split(lower({s}), ' '), w -> concat(upper(left(w, 1)), substring(w, 2))), ' ')") +} + +/// Translates `lower` (a lower-case Access function name) when DuckDB can +/// express it. `a` holds the translated arguments. None: not a known built-in. +pub fn duck_call( + w: &mut SqlWriter, + lower: &str, + a: &[String], + args: &[Expr], +) -> Result, String> { + let need = |k: usize| { + if a.len() < k { + Err(format!("{lower} needs {k} arguments")) + } else { + Ok(()) + } + }; + let sql = match lower { + "strconv" => { + need(2)?; + match arg_int(args, 1) { + Some(1) => format!("upper({})", a[0]), + Some(2) => format!("lower({})", a[0]), + Some(3) => proper(&a[0]), + _ => { + return Err( + "StrConv supports only vbUpperCase, vbLowerCase and vbProperCase".into(), + ) + } + } + } + "monthname" => { + need(1)?; + let f = if is_false(args, 1) { "%B" } else { "%b" }; + format!( + "strftime(make_date(2000, CAST({} AS INTEGER), 1), '{f}')", + a[0] + ) + } + "weekdayname" => { + need(1)?; + if !is_false(args, 2) && arg_int(args, 2) != Some(1) { + return Err("WeekdayName with a first day other than Sunday".into()); + } + let f = if is_false(args, 1) { "%A" } else { "%a" }; + // 2000-01-01 was a Saturday, so day 1 (Sunday) is the 2nd. + format!( + "strftime(make_date(2000, 1, 1 + CAST({} AS INTEGER)), '{f}')", + a[0] + ) + } + "formatcurrency" | "formatnumber" | "formatpercent" => { + need(1)?; + let digits = match args.get(1) { + None => 2, + Some(_) => arg_int(args, 1) + .filter(|d| (0..=10).contains(d)) + .ok_or_else(|| format!("{lower} needs literal digits"))?, + }; + let (prefix, scale, suffix) = match lower { + "formatcurrency" => ("$", "", ""), + "formatpercent" => ("", " * 100", "%"), + _ => ("", "", ""), + }; + // fmt rounds half to even; Access rounds half away from zero. + let core = format!( + "format('{{:,.{digits}f}}', round(CAST(({}){scale} AS DOUBLE), {digits}))", + a[0] + ); + wrap(prefix, core, suffix) + } + "formatdatetime" => { + need(1)?; + let f = match arg_int(args, 1).unwrap_or(0) { + 0 => "%-m/%-d/%Y %-I:%M:%S %p", + 1 => "%A, %B %-d, %Y", + 2 => "%-m/%-d/%Y", + 3 => "%-I:%M:%S %p", + 4 => "%H:%M", + _ => return Err("FormatDateTime needs a literal named format".into()), + }; + format!("strftime(CAST({} AS TIMESTAMP), '{f}')", a[0]) + } + "partition" => { + need(4)?; + partition(a) + } + "hex" => format!("upper(to_hex(CAST({} AS BIGINT)))", a[0]), + "oct" => format!("printf('%o', CAST({} AS BIGINT))", a[0]), + "rnd" => "random()".into(), + "timer" => { + "(epoch(CAST(now() AS TIMESTAMP)) - epoch(CAST(CAST(now() AS TIMESTAMP) AS DATE)))" + .into() + } + "timeserial" => { + need(3)?; + format!( + "(TIME '00:00:00' + to_seconds(CAST({} AS BIGINT) * 3600 + CAST({} AS BIGINT) * 60 + CAST({} AS BIGINT)))", + a[0], a[1], a[2] + ) + } + "instrrev" => { + need(2)?; + if a.len() > 2 { + return Err("InStrRev with a start position".into()); + } + // Position of the last match: 0 when absent. + format!( + "(CASE WHEN instr({0}, {1}) = 0 THEN 0 ELSE length({0}) - instr(reverse({0}), reverse({1})) - length({1}) + 2 END)", + a[0], a[1] + ) + } + "strreverse" => format!("reverse({})", a[0]), + "strcomp" => { + need(2)?; + // Text compare (the Access default in queries) ignores case. + let binary = arg_int(args, 2) == Some(0); + let (l, r) = if binary { + (a[0].clone(), a[1].clone()) + } else { + (format!("lower({})", a[0]), format!("lower({})", a[1])) + }; + format!("(CASE WHEN {l} IS NULL OR {r} IS NULL THEN NULL WHEN {l} < {r} THEN -1 WHEN {l} > {r} THEN 1 ELSE 0 END)") + } + "atn" => format!("atan({})", a[0]), + "sin" | "cos" | "tan" => format!("{lower}({})", a[0]), + "iserror" => "FALSE".into(), + "eval" => match args.first() { + Some(Expr::Str(text)) => { + let e = parse_expression(text).map_err(|e| format!("Eval: {e}"))?; + w.expr(&e)? + } + _ => return Err("Eval with a computed expression".into()), + }, + _ => return Ok(None), + }; + Ok(Some(sql)) +} + +fn wrap(prefix: &str, core: String, suffix: &str) -> String { + match (prefix.is_empty(), suffix.is_empty()) { + (true, true) => core, + _ => format!( + "concat({}, {core}, {})", + string_literal(prefix), + string_literal(suffix) + ), + } +} + +/// `Partition(n, start, stop, interval)`: the range label `" 10: 19"`. +fn partition(a: &[String]) -> String { + let n = &format!("CAST(floor({}) AS BIGINT)", a[0]); + let (start, stop, step) = (&a[1], &a[2], &a[3]); + // Access pads both numbers to the width of the larger bound. + let width = format!("CAST(greatest(length(CAST({stop} + 1 AS VARCHAR)), length(CAST({start} - 1 AS VARCHAR))) AS INTEGER)"); + let pad = |x: String| format!("lpad(CAST({x} AS VARCHAR), {width}, ' ')"); + let lo = format!("({start} + (({n}) - {start}) // {step} * {step})"); + let hi = format!("least({lo} + {step} - 1, {stop})"); + format!( + "(CASE WHEN {} IS NULL THEN NULL WHEN {n} < {start} THEN concat({}, ':', {}) WHEN {n} > {stop} THEN concat({}, ':', {}) ELSE concat({}, ':', {}) END)", + a[0], + pad("''".into()), + pad(format!("{start} - 1")), + pad(format!("{stop} + 1")), + pad("''".into()), + pad(lo.clone()), + pad(hi) + ) +} + +/// `Format(value, "pattern")` with a named or custom Access format. +pub fn format_sql(value: &str, pattern: Option<&Expr>) -> Result { + let Some(pattern) = pattern else { + return Ok(format!("CAST({value} AS VARCHAR)")); + }; + let Expr::Str(fmt) = pattern else { + return Err("Format needs a literal format".into()); + }; + let named = match fmt.trim().to_ascii_lowercase().as_str() { + "short date" => Some("%-m/%-d/%Y"), + "medium date" => Some("%d-%b-%y"), + "long date" => Some("%A, %B %-d, %Y"), + "general date" => Some("%-m/%-d/%Y %-I:%M:%S %p"), + "short time" => Some("%H:%M"), + "medium time" => Some("%-I:%M %p"), + "long time" => Some("%-I:%M:%S %p"), + "ww" => return Ok(format!("CAST(week({value}) AS VARCHAR)")), + "q" => return Ok(format!("CAST(quarter({value}) AS VARCHAR)")), + "y" => return Ok(format!("CAST(dayofyear({value}) AS VARCHAR)")), + "w" => return Ok(format!("CAST(dayofweek({value}) + 1 AS VARCHAR)")), + "currency" | "euro" => return Ok(number_sql(value, &number_spec("$#,##0.00"))), + "fixed" => return Ok(number_sql(value, &number_spec("0.00"))), + "standard" => return Ok(number_sql(value, &number_spec("#,##0.00"))), + "percent" => return Ok(number_sql(value, &number_spec("0.00%"))), + "general number" | "general" | "" => return Ok(format!("CAST({value} AS VARCHAR)")), + "yes/no" => return Ok(format!("CASE WHEN {value} THEN 'Yes' ELSE 'No' END")), + "true/false" => return Ok(format!("CASE WHEN {value} THEN 'True' ELSE 'False' END")), + "on/off" => return Ok(format!("CASE WHEN {value} THEN 'On' ELSE 'Off' END")), + ">" => return Ok(format!("upper(CAST({value} AS VARCHAR))")), + "<" => return Ok(format!("lower(CAST({value} AS VARCHAR))")), + _ => None, + }; + if let Some(f) = named { + return Ok(format!("strftime(CAST({value} AS TIMESTAMP), '{f}')")); + } + let section = first_section(fmt); + if section.contains(['0', '#']) && !is_date_pattern(§ion) { + return Ok(number_sql(value, &number_spec(§ion))); + } + if is_date_pattern(§ion) { + let f = strftime_pattern(§ion)?; + return Ok(format!( + "strftime(CAST({value} AS TIMESTAMP), {})", + string_literal(&f) + )); + } + Err(format!("the format \"{fmt}\" is not supported")) +} + +fn first_section(s: &str) -> String { + let mut out = String::new(); + let mut quoted = false; + for c in s.chars() { + if c == '"' { + quoted = !quoted; + } + if c == ';' && !quoted { + break; + } + out.push(c); + } + out +} + +fn is_date_pattern(s: &str) -> bool { + let mut quoted = false; + let mut escaped = false; + for c in s.chars() { + if escaped { + escaped = false; + continue; + } + match c { + '"' => quoted = !quoted, + '\\' => escaped = true, + 'y' | 'Y' | 'm' | 'M' | 'd' | 'D' | 'h' | 'H' | 'n' | 'N' | 's' | 'S' if !quoted => { + return true + } + _ => {} + } + } + false +} + +/// Access date tokens → strftime. `m` after an hour means minutes. +pub fn strftime_pattern(p: &str) -> Result { + let chars: Vec = p.chars().collect(); + let mut out = String::new(); + let mut i = 0; + let mut after_hour = false; + let run = |i: usize, c: char| { + chars[i..] + .iter() + .take_while(|x| x.eq_ignore_ascii_case(&c)) + .count() + }; + let lit = |out: &mut String, c: char| { + if c == '%' { + out.push_str("%%"); + } else { + out.push(c); + } + }; + while i < chars.len() { + let c = chars[i]; + let rest: String = chars[i..].iter().collect::().to_ascii_lowercase(); + if rest.starts_with("am/pm") { + out.push_str("%p"); + i += 5; + continue; + } + if rest.starts_with("a/p") { + out.push_str("%p"); + i += 3; + continue; + } + if rest.starts_with("ampm") { + out.push_str("%p"); + i += 4; + continue; + } + match c.to_ascii_lowercase() { + 'y' => { + let n = run(i, 'y'); + out.push_str(match n { + 1 => "%j", + 2 => "%y", + _ => "%Y", + }); + i += n; + } + 'm' => { + let n = run(i, 'm'); + let minutes = after_hour && n <= 2 || next_is_seconds(&chars, i + n); + out.push_str(match (minutes, n) { + (true, 1) => "%-M", + (true, _) => "%M", + (false, 1) => "%-m", + (false, 2) => "%m", + (false, 3) => "%b", + _ => "%B", + }); + after_hour = false; + i += n; + } + 'd' => { + let n = run(i, 'd'); + out.push_str(match n { + 1 => "%-d", + 2 => "%d", + 3 => "%a", + 4 => "%A", + 5 => "%-m/%-d/%Y", + _ => "%A, %B %-d, %Y", + }); + i += n; + } + 'h' => { + let n = run(i, 'h'); + let twelve = chars[i..].iter().collect::().to_ascii_lowercase(); + let twelve = + twelve.contains("am/pm") || twelve.contains("a/p") || twelve.contains("ampm"); + out.push_str(match (twelve, n) { + (true, 1) => "%-I", + (true, _) => "%I", + (false, 1) => "%-H", + (false, _) => "%H", + }); + after_hour = true; + i += n; + } + 'n' => { + let n = run(i, 'n'); + out.push_str(if n == 1 { "%-M" } else { "%M" }); + i += n; + } + 's' => { + let n = run(i, 's'); + out.push_str(if n == 1 { "%-S" } else { "%S" }); + i += n; + } + 'q' => return Err("the quarter in a date format".into()), + 'w' => return Err("the week in a date format".into()), + '"' => { + i += 1; + while i < chars.len() && chars[i] != '"' { + lit(&mut out, chars[i]); + i += 1; + } + i += 1; + } + '\\' => { + if let Some(&n) = chars.get(i + 1) { + lit(&mut out, n); + } + i += 2; + } + _ => { + lit(&mut out, c); + i += 1; + } + } + } + Ok(out) +} + +fn next_is_seconds(chars: &[char], from: usize) -> bool { + chars[from..] + .iter() + .find(|c| c.is_ascii_alphabetic()) + .is_some_and(|c| c.eq_ignore_ascii_case(&'s')) +} + +#[derive(Debug, PartialEq)] +pub struct NumberSpec { + pub prefix: String, + pub suffix: String, + pub grouping: bool, + pub decimals: usize, + /// Zeros before the decimal point: the minimum integer digits. + pub zeros: usize, + pub percent: bool, +} + +/// Reads `$#,##0.00`, `0.0%`, `000`, `"Qty: "0`. +pub fn number_spec(p: &str) -> NumberSpec { + let mut spec = NumberSpec { + prefix: String::new(), + suffix: String::new(), + grouping: false, + decimals: 0, + zeros: 0, + percent: false, + }; + let chars: Vec = p.chars().collect(); + let mut i = 0; + let mut seen_digit = false; + let mut after_point = false; + while i < chars.len() { + let c = chars[i]; + let text = if seen_digit { + &mut spec.suffix + } else { + &mut spec.prefix + }; + match c { + '0' | '#' => { + // Digits after a suffix started are part of the number again. + seen_digit = true; + if after_point { + if c == '0' { + spec.decimals += 1; + } + } else if c == '0' { + spec.zeros += 1; + } + } + ',' if seen_digit && !after_point => spec.grouping = true, + '.' if seen_digit || chars.get(i + 1).is_some_and(|n| *n == '0' || *n == '#') => { + seen_digit = true; + after_point = true; + } + '%' => { + spec.percent = true; + text.push('%'); + } + '"' => { + i += 1; + while i < chars.len() && chars[i] != '"' { + text.push(chars[i]); + i += 1; + } + } + '\\' => { + if let Some(&n) = chars.get(i + 1) { + text.push(n); + } + i += 1; + } + _ => text.push(c), + } + i += 1; + } + spec +} + +pub fn number_sql(value: &str, spec: &NumberSpec) -> String { + let d = spec.decimals; + // fmt rounds half to even; Access rounds half away from zero. + let scaled = if spec.percent { + format!("round(CAST(({value}) * 100 AS DOUBLE), {d})") + } else { + format!("round(CAST({value} AS DOUBLE), {d})") + }; + let core = if spec.grouping { + format!("format('{{:,.{d}f}}', {scaled})") + } else if spec.zeros > 1 { + let width = spec.zeros + if d > 0 { d + 1 } else { 0 }; + format!("format('{{:0{width}.{d}f}}', {scaled})") + } else { + format!("format('{{:.{d}f}}', {scaled})") + }; + let core = if spec.zeros == 0 && !spec.grouping { + // `#.00` drops the leading zero of a fraction. + format!("regexp_replace({core}, '^(-?)0\\.', '\\1.')") + } else { + core + }; + format!( + "CASE WHEN {value} IS NULL THEN NULL ELSE {} END", + wrap(&spec.prefix, core, &spec.suffix) + ) +} diff --git a/src-tauri/src/access/translate/domain.rs b/src-tauri/src/access/translate/domain.rs new file mode 100644 index 00000000..0c593d37 --- /dev/null +++ b/src-tauri/src/access/translate/domain.rs @@ -0,0 +1,148 @@ +//! Domain aggregate functions (`DLookup`, `DCount`, `DSum`...) → scalar subqueries. +//! +//! Access builds the criteria as text at run time, usually from the current +//! row: `DLookup("Price", "Products", "ID=" & [ProductID])`. The text pieces +//! are joined with placeholders for the computed pieces, parsed as one WHERE +//! clause, and each placeholder becomes the outer SQL, so the subquery is +//! correlated with the row instead of rebuilt from text. +use super::ast::*; +use super::sql::SqlWriter; + +pub fn is_domain(name: &str) -> bool { + matches!( + name, + "dlookup" | "dcount" | "dsum" | "davg" | "dmin" | "dmax" | "dfirst" | "dlast" + ) +} + +fn arg_str(args: &[Expr], i: usize) -> Option<&str> { + match args.get(i) { + Some(Expr::Str(s)) => Some(s), + Some(Expr::Paren(inner)) => arg_str(std::slice::from_ref(inner), 0), + _ => None, + } +} + +/// The pieces of a `&` (or `+` on text) chain, left to right. +fn concat_pieces<'e>(e: &'e Expr, out: &mut Vec<&'e Expr>) { + match e { + Expr::Bin(BinOp::Concat | BinOp::Add, l, r) + if is_texty(l) || is_texty(r) || matches!(e, Expr::Bin(BinOp::Concat, ..)) => + { + concat_pieces(l, out); + concat_pieces(r, out); + } + Expr::Paren(inner) if matches!(**inner, Expr::Bin(BinOp::Concat, ..)) => { + concat_pieces(inner, out) + } + other => out.push(other), + } +} + +fn is_texty(e: &Expr) -> bool { + matches!(e, Expr::Str(_) | Expr::Bin(BinOp::Concat, ..)) +} + +/// Criteria text with `[__outer_N]` placeholders, and the SQL of each one. +fn criteria_text(w: &mut SqlWriter, e: &Expr) -> Result<(String, Vec<(String, String)>), String> { + let mut pieces = vec![]; + concat_pieces(e, &mut pieces); + let mut text = String::new(); + let mut outer = vec![]; + let mut i = 0; + while i < pieces.len() { + match pieces[i] { + Expr::Str(s) => text.push_str(s), + computed => { + let name = format!("__outer_{}", w.outer.len() + outer.len()); + w.qualify = true; + let sql = w.expr(computed); + w.qualify = false; + let mut sql = sql?; + // `"Name='" & [Name] & "'"`: the quotes wrap a value, not text to parse. + let next = match pieces.get(i + 1) { + Some(Expr::Str(s)) => Some(s.as_str()), + _ => None, + }; + let quote_char = ['\'', '"', '#'] + .into_iter() + .find(|q| text.ends_with(*q) && next.is_some_and(|n| n.starts_with(*q))); + if let Some(q) = quote_char { + text.pop(); + if q != '#' { + sql = format!("CAST({sql} AS VARCHAR)"); + } + text.push_str(&format!("[{name}]")); + outer.push((name, sql)); + text.push_str(&next.unwrap_or_default()[1..]); + i += 2; + continue; + } + text.push_str(&format!("[{name}]")); + outer.push((name, sql)); + } + } + i += 1; + } + Ok((text, outer)) +} + +/// `DLookup("expr", "domain", criteria)` → a scalar subquery. +pub fn domain_sql(w: &mut SqlWriter, f: &str, args: &[Expr]) -> Result { + let expr = arg_str(args, 0).ok_or_else(|| format!("{f} needs a literal expression"))?; + let domain = arg_str(args, 1).ok_or_else(|| format!("{f} needs a literal domain"))?; + let (criteria, outer) = match args.get(2) { + None => (None, vec![]), + Some(e) => match arg_str(args, 2) { + Some(s) => (Some(s.to_string()), vec![]), + None => { + let (text, outer) = criteria_text(w, e)?; + (Some(text), outer) + } + }, + }; + let agg = match f { + "dlookup" => None, + "dcount" => Some("Count"), + "dsum" => Some("Sum"), + "davg" => Some("Avg"), + "dmin" => Some("Min"), + "dmax" => Some("Max"), + "dfirst" => Some("First"), + _ => Some("Last"), + }; + let domain = domain.trim().trim_start_matches('[').trim_end_matches(']'); + let column = match agg { + Some(a) => format!("{a}({expr})"), + None => expr.to_string(), + }; + // The same table in the outer query: alias the domain so names stay apart. + let shadowed = !outer.is_empty() + && w.scopes.iter().any(|s| { + s.sources + .iter() + .any(|(name, _)| name.eq_ignore_ascii_case(domain)) + }); + let mut sql = format!("SELECT {column} FROM [{domain}]"); + if shadowed { + sql.push_str(" AS [__domain]"); + } + if let Some(c) = criteria.filter(|c| !c.trim().is_empty()) { + sql.push_str(&format!(" WHERE {c}")); + } + let select = match parse_statement(&sql) { + Ok(Statement::Select(s)) => s, + Ok(_) => return Err("a domain function needs a SELECT".into()), + Err(e) => return Err(format!("{f} criteria could not be read ({e})")), + }; + let added = outer.len(); + w.outer.extend(outer); + let inner = w.select(&select); + let keep = w.outer.len() - added; + w.outer.truncate(keep); + let mut inner = inner?; + if agg.is_none() { + inner.push_str(" LIMIT 1"); + } + Ok(format!("({inner})")) +} diff --git a/src-tauri/src/access/translate/expr.rs b/src-tauri/src/access/translate/expr.rs index 68ea8b7c..63d0ef64 100644 --- a/src-tauri/src/access/translate/expr.rs +++ b/src-tauri/src/access/translate/expr.rs @@ -202,7 +202,7 @@ impl<'a> ExprWriter<'a> { fn like(&mut self, expr: &Expr, pattern: &Expr) -> Result { let e = self.write(expr)?; let Expr::Str(p) = pattern else { - return Err("Like with a computed pattern".into()); + return self.computed_like(&e, pattern); }; let inner = p.trim_matches('*'); let plain = !inner.contains(['*', '?', '#', '[']); @@ -231,6 +231,26 @@ impl<'a> ExprWriter<'a> { } } + /// `Like "*" & [Find] & "*"`: a search box's pattern around a value. + fn computed_like(&mut self, e: &str, pattern: &Expr) -> Result { + let star = |x: &Expr| matches!(x, Expr::Str(s) if s == "*"); + let (lead, middle, trail) = match pattern { + Expr::Bin(BinOp::Concat, l, r) => match (&**l, &**r) { + (Expr::Bin(BinOp::Concat, a, b), c) if star(a) && star(c) => (true, &**b, true), + (a, b) if star(a) => (true, b, false), + (a, b) if star(b) => (false, a, true), + _ => return Err("Like with a computed pattern".into()), + }, + _ => return Err("Like with a computed pattern".into()), + }; + let v = self.write(middle)?; + Ok(match (lead, trail) { + (true, true) => format!("contains({e}, {v})"), + (true, false) => format!("endswith({e}, {v})"), + _ => format!("startswith({e}, {v})"), + }) + } + fn aggregate(&mut self, f: &str, args: &[Expr]) -> Result { if self.target != Target::Report || self.in_item { return Err(format!("{f}() is only supported in reports")); @@ -281,6 +301,59 @@ impl<'a> ExprWriter<'a> { }; return Ok(format!("{lower}('{unit}', {n}, {})", a[1])); } + "datepart" => { + let Some(Expr::Str(code)) = args.first() else { + return Err("DatePart needs a literal interval".into()); + }; + let d = self.write(args.get(1).ok_or("DatePart needs 2 arguments")?)?; + return Ok(match code.to_ascii_lowercase().as_str() { + "yyyy" => format!("year({d})"), + "q" => format!("floor((month({d}) - 1) / 3) + 1"), + "m" => format!("month({d})"), + "d" => format!("day({d})"), + "w" => format!("weekday({d})"), + "h" => format!("hour({d})"), + "n" => format!("minute({d})"), + other => return Err(format!("DatePart \"{other}\" is not supported")), + }); + } + "strconv" => { + let v = self.write(args.first().ok_or("StrConv needs 2 arguments")?)?; + return match args.get(1) { + Some(Expr::Num(n)) if n == "1" => Ok(format!("upper({v})")), + Some(Expr::Num(n)) if n == "2" => Ok(format!("lower({v})")), + _ => Err("StrConv supports only vbUpperCase and vbLowerCase".into()), + }; + } + "monthname" => { + let m = self.write(args.first().ok_or("MonthName needs a month")?)?; + let abbreviated = matches!(args.get(1), Some(Expr::Bool(true))) + || matches!(args.get(1), Some(Expr::Num(n)) if n != "0"); + let p = if abbreviated { "MMM" } else { "MMMM" }; + return Ok(format!("format(date(2000, {m}, 1), '{p}')")); + } + "switch" => { + if args.len() < 2 || args.len() % 2 != 0 { + return Err("Switch needs condition/value pairs".into()); + } + let mut out = "null".to_string(); + for pair in args.chunks(2).rev() { + out = format!( + "if({}, {}, {out})", + self.write(&pair[0])?, + self.write(&pair[1])? + ); + } + return Ok(out); + } + "choose" => { + let index = self.write(args.first().ok_or("Choose needs an index")?)?; + let mut out = "null".to_string(); + for (i, v) in args.iter().enumerate().skip(1).rev() { + out = format!("if({index} = {i}, {}, {out})", self.write(v)?); + } + return Ok(out); + } "format" => { let value = self.write(args.first().ok_or("Format needs a value")?)?; let Some(Expr::Str(f)) = args.get(1) else { @@ -347,6 +420,8 @@ impl<'a> ExprWriter<'a> { "cdbl" | "csng" | "ccur" | "cdec" | "val" => format!("number({})", a[0]), "cdate" | "datevalue" => a[0].clone(), "coalesce" => format!("coalesce({})", a.join(", ")), + "sgn" => format!("if({0} > 0, 1, if({0} < 0, -1, 0))", a[0]), + "iserror" => "false".into(), _ => return Err(format!("{name}() is not supported")), }) } diff --git a/src-tauri/src/access/translate/functions.rs b/src-tauri/src/access/translate/functions.rs index fb9114a8..5d91476c 100644 --- a/src-tauri/src/access/translate/functions.rs +++ b/src-tauri/src/access/translate/functions.rs @@ -1,6 +1,6 @@ //! Access (VBA) built-in functions in SQL, plus `Like` patterns and date literals. use super::ast::*; -use super::sql::{string_literal, Dialect, SqlWriter}; +use super::sql::{Dialect, SqlWriter}; /// True when an expression uses an aggregate (outside subqueries). pub fn contains_aggregate(e: &Expr) -> bool { @@ -222,7 +222,7 @@ pub fn sql_call( } "datevalue" | "cdate" if duck => format!("CAST({} AS TIMESTAMP)", a[0]), "timevalue" if duck => format!("CAST({} AS TIME)", a[0]), - "format" if duck => format_sql(&a, args)?, + "format" if duck => super::builtins::format_sql(&a[0], args.get(1))?, "left" => format!("{}({}, 1, {})", if duck { "substring" } else { "substr" }, a[0], a[1]), "right" if duck => format!("right({}, {})", a[0], a[1]), "right" => format!("substr({}, -({}))", a[0], a[1]), @@ -298,12 +298,16 @@ pub fn sql_call( } "coalesce" => format!("coalesce({})", a.join(", ")), "plaintext" if duck => format!("replace(regexp_replace({}, '<[^>]*>', '', 'g'), ' ', ' ')", a[0]), - // A VBA function: the column stays, without values. - _ if duck && !is_builtin(lower) => { - w.out.notes.push(format!("{name}() is a VBA function, so its column is empty")); - "NULL".into() - } - "dlookup" | "dcount" | "dsum" | "davg" | "dmin" | "dmax" | "dfirst" | "dlast" if duck => domain_sql(w, lower, args)?, + _ if duck && super::domain::is_domain(lower) => super::domain::domain_sql(w, lower, args)?, + _ if duck => match super::builtins::duck_call(w, lower, &a, args)? { + Some(sql) => sql, + // A VBA function: the column stays, without values. + None if !is_builtin(lower) => { + w.out.notes.push(format!("{name}() is a VBA function, so its column is empty")); + "NULL".into() + } + None => return Err(format!("{name}() is not supported")), + }, _ => return Err(format!("{name}() is not supported")), }) } @@ -339,73 +343,6 @@ fn is_builtin(name: &str) -> bool { .contains(&name) } -/// `DLookup("expr", "domain", "criteria")` with literal arguments → a scalar subquery. -fn domain_sql(w: &mut SqlWriter, f: &str, args: &[Expr]) -> Result { - let expr = arg_str(args, 0).ok_or_else(|| format!("{f} needs literal arguments"))?; - let domain = arg_str(args, 1).ok_or_else(|| format!("{f} needs literal arguments"))?; - let criteria = match args.get(2) { - None => None, - Some(Expr::Str(s)) => Some(s.as_str()), - Some(_) => return Err(format!("{f} with a computed criteria")), - }; - let agg = match f { - "dlookup" => None, - "dcount" => Some("Count"), - "dsum" => Some("Sum"), - "davg" => Some("Avg"), - "dmin" => Some("Min"), - "dmax" => Some("Max"), - "dfirst" => Some("First"), - _ => Some("Last"), - }; - let domain = domain.trim().trim_start_matches('[').trim_end_matches(']'); - let column = match agg { - Some(a) => format!("{a}({expr})"), - None => expr.to_string(), - }; - let mut sql = format!("SELECT {column} FROM [{domain}]"); - if let Some(c) = criteria.filter(|c| !c.trim().is_empty()) { - sql.push_str(&format!(" WHERE {c}")); - } - let select = match parse_statement(&sql)? { - Statement::Select(s) => s, - _ => return Err("a domain function needs a SELECT".into()), - }; - let mut inner = w.select(&select)?; - if agg.is_none() { - inner.push_str(" LIMIT 1"); - } - Ok(format!("({inner})")) -} - -fn format_sql(a: &[String], args: &[Expr]) -> Result { - let fmt = arg_str(args, 1).ok_or("Format needs a literal format")?; - let strf = match fmt.to_ascii_lowercase().as_str() { - "yyyy" => "%Y", - "yy" => "%y", - "mm" => "%m", - "m" => "%-m", - "mmm" => "%b", - "mmmm" => "%B", - "dd" => "%d", - "d" => "%-d", - "ww" => return Ok(format!("CAST(week({}) AS VARCHAR)", a[0])), - "q" => return Ok(format!("CAST(quarter({}) AS VARCHAR)", a[0])), - "short date" => "%-m/%-d/%Y", - "medium date" => "%d-%b-%y", - "long date" => "%A, %B %-d, %Y", - "mmmm yyyy" => "%B %Y", - "mmm yyyy" => "%b %Y", - "yyyy-mm-dd" => "%Y-%m-%d", - "mm/dd/yyyy" => "%m/%d/%Y", - "currency" => return Ok(format!("printf('$%.2f', {})", a[0])), - "percent" => return Ok(format!("printf('%.2f%%', ({}) * 100)", a[0])), - "fixed" | "standard" => return Ok(format!("printf('%.2f', {})", a[0])), - _ => return Ok(format!("CAST({} AS VARCHAR)", a[0])), - }; - Ok(format!("strftime({}, {})", a[0], string_literal(strf))) -} - pub enum LikePattern { Like(String), Regex(String), diff --git a/src-tauri/src/access/translate/mod.rs b/src-tauri/src/access/translate/mod.rs index 4ab4d617..66b8a69e 100644 --- a/src-tauri/src/access/translate/mod.rs +++ b/src-tauri/src/access/translate/mod.rs @@ -1,6 +1,8 @@ //! Translation of Access SQL, expressions and formats (`docs/access-format.md` §6). pub mod ast; +pub mod builtins; mod dml; +mod domain; pub mod expr; pub mod format; pub mod functions; diff --git a/src-tauri/src/access/translate/sql.rs b/src-tauri/src/access/translate/sql.rs index c55483be..7eac80d1 100644 --- a/src-tauri/src/access/translate/sql.rs +++ b/src-tauri/src/access/translate/sql.rs @@ -79,6 +79,11 @@ pub struct SqlWriter<'a> { inlining: Vec, /// True while translating a column DEFAULT (SQLite keywords for now). pub in_default: bool, + /// Placeholder names standing for outer-query SQL in a domain function's + /// criteria (`DLookup(..., "ID=" & [ID])`): (name, SQL). + pub(super) outer: Vec<(String, String)>, + /// Qualify bare column names with their source (outer references). + pub(super) qualify: bool, } pub fn quote(name: &str) -> String { @@ -139,6 +144,8 @@ impl<'a> SqlWriter<'a> { table_columns: vec![], inlining: vec![], in_default: false, + outer: vec![], + qualify: false, } } @@ -220,7 +227,8 @@ impl<'a> SqlWriter<'a> { "sum" | "avg" | "count" | "dcount" | "dsum" | "davg" | "year" | "month" | "day" | "hour" | "minute" | "second" | "weekday" | "datediff" | "datepart" | "len" | "instr" | "val" | "int" | "fix" | "abs" | "round" - | "cint" | "clng" | "cdbl" | "csng" | "ccur" | "cdec" | "sgn" | "sqr" => { + | "cint" | "clng" | "cdbl" | "csng" | "ccur" | "cdec" | "sgn" | "sqr" + | "instrrev" | "strcomp" | "rnd" | "timer" | "atn" | "sin" | "cos" | "tan" => { Kind::Number } "min" | "max" | "first" | "last" | "nz" | "dmin" | "dmax" | "dfirst" @@ -234,7 +242,9 @@ impl<'a> SqlWriter<'a> { _ => Kind::Other, }, "format" | "left" | "right" | "mid" | "trim" | "ltrim" | "rtrim" | "ucase" - | "lcase" | "cstr" | "replace" | "plaintext" => Kind::Text, + | "lcase" | "cstr" | "replace" | "plaintext" | "strconv" | "monthname" + | "weekdayname" | "formatcurrency" | "formatnumber" | "formatpercent" + | "formatdatetime" | "partition" | "hex" | "oct" | "strreverse" => Kind::Text, _ => Kind::Other, } } @@ -280,8 +290,11 @@ impl<'a> SqlWriter<'a> { match parts { [one] => { for scope in self.scopes.iter().rev() { - for (_, cols) in &scope.sources { + for (src, cols) in &scope.sources { if let Some((c, k)) = find(cols, &one.text) { + if self.qualify { + return Some((format!("{}.{}", quote(src), quote(&c)), k)); + } return Some((quote(&c), k)); } } @@ -418,6 +431,11 @@ impl<'a> SqlWriter<'a> { .join(".") )); } + if let [one] = parts { + if let Some((_, sql)) = self.outer.iter().find(|(n, _)| *n == one.text) { + return Ok(sql.clone()); + } + } let original: Vec = parts.iter().map(|p| p.text.clone()).collect(); let first = parts[0].text.to_ascii_lowercase(); if first == "me" && parts.len() == 2 { diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index dc0d3cfc..cae90d67 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -462,6 +462,7 @@ pub fn run() { // access commands access::inspect_access_file, access::import_access_file, + access::write_access_report, // templates commands templates::list_templates, templates::read_template_config, diff --git a/src/access/AccessImportWizard.tsx b/src/access/AccessImportWizard.tsx index 37848556..1e4405a7 100644 --- a/src/access/AccessImportWizard.tsx +++ b/src/access/AccessImportWizard.tsx @@ -222,6 +222,7 @@ function ReportView({ result }: { result: AccessImport }) {

Created a document with {plural(report.tables, "table")} and {plural(report.rows, "row")}.

+

Settings › Access migration keeps this report, with links, review marks and export.

diff --git a/src/access/MigrationTab.tsx b/src/access/MigrationTab.tsx new file mode 100644 index 00000000..373b7a0f --- /dev/null +++ b/src/access/MigrationTab.tsx @@ -0,0 +1,253 @@ +import { Download } from "lucide-react"; +import { useId, useState } from "react"; +import { asTauriError, type TauriError } from "../lib/api"; +import { useDocumentConfig } from "../lib/config-store"; +import { type RevealTarget, useShell } from "../shell/context"; +import { writeAccessReport } from "./api"; +import { chooseReportDestination } from "./dialog"; +import { + filterItems, + itemKey, + type MigrationFilter, + openCount, + readMigration, + type StatusFilter, + statusLabel, + toCsv, + toMarkdown, + withReviewed, +} from "./migration"; +import { FORMAT_LABELS, KIND_ORDER, kindLabel, statusCounts } from "./summary"; +import type { ImportTarget } from "./types"; +import "./access.css"; + +const revealFor = (t: ImportTarget): RevealTarget | null => { + switch (t.kind) { + case "form": + return { mode: "design", objectId: t.id }; + case "report": + return { mode: "reports", objectId: t.id }; + case "action": + return { mode: "automation", tab: "actions", objectId: t.id }; + default: + return null; + } +}; + +const OPEN_LABEL: Record = { + form: "Open form", + report: "Open report", + action: "Open action", + query: "", +}; + +/** + * Settings › Access migration: everything an Access import converted, with + * what each object lost, links to the result, review marks, and export + * (docs/decisions/access-import.md, "Migration report"). + */ +export function AccessMigrationTab() { + const { config, update } = useDocumentConfig(); + const { requestReveal } = useShell(); + const migration = readMigration(config.settings); + const searchId = useId(); + const [filter, setFilter] = useState({ + status: "attention", + kind: "", + search: "", + hideReviewed: false, + }); + const [error, setError] = useState(null); + const [notice, setNotice] = useState(""); + + if (!migration) { + return ( +
+

Access migration

+

This document was not imported from Microsoft Access.

+
+ ); + } + const items = filterItems(migration, filter); + const reviewed = new Set(migration.reviewed); + const counts = statusCounts({ items: migration.report, warnings: [], tables: 0, rows: 0 }); + const kinds = KIND_ORDER.filter((k) => migration.report.some((i) => i.kind === k)); + const open = openCount(migration); + + const mark = (key: string, on: boolean) => + update( + (draft) => ({ ...draft, settings: withReviewed(draft.settings, key, on) }), + on ? "Mark Access item reviewed" : "Unmark Access item reviewed", + ); + + const exportAs = async (format: "md" | "csv") => { + setError(null); + setNotice(""); + const base = `${config.name || "Access"} migration report`; + const path = await chooseReportDestination(base, format); + if (typeof path !== "string") return; + try { + await writeAccessReport( + path, + format === "md" ? toMarkdown(migration, config.name) : toCsv(migration), + ); + setNotice(`Saved ${path.split(/[\\/]/).pop()}`); + } catch (reason) { + setError(asTauriError(reason)); + } + }; + + return ( +
+

Access migration

+

+ Imported from {migration.source || "an Access file"},{" "} + {(FORMAT_LABELS[migration.format] ?? migration.format).replace(/^Access/, "an Access")} + {migration.importedAt ? ` on ${migration.importedAt.slice(0, 10)}` : ""}.{" "} + {open === 0 + ? "Every item that needs attention is reviewed." + : `${open} item${open === 1 ? "" : "s"} still need${open === 1 ? "s" : ""} review.`} +

+
+ + + + + + + + + + {counts.map((c) => ( + + + + + + + ))} + +
ObjectsConvertedPartly convertedNot converted
{kindLabel(c.kind)}{c.converted}{c.partial}{c.skipped}
+
+ + + + setFilter({ ...filter, search: e.target.value })} + /> + + + +
+ {notice &&

{notice}

} + {error && ( +
+ {error.code} {error.message} +
+ )} + {items.length === 0 ? ( +

No items match.

+ ) : ( + + + + + + + + + + + {items.map((item) => { + const key = itemKey(item); + const target = item.target ? revealFor(item.target) : null; + return ( + + + + + + + ); + })} + +
ObjectStatusNotesReviewed
+ {item.name} {item.kind} + {item.target && target && ( +
+ +
+ )} + 		{statusLabel(item.status)} + {item.notes.length > 0 && ( +
    + {item.notes.map((n) => ( +
  • {n}
    • + ))} +
+ )} + 		+ mark(key, e.target.checked)} + /> +
+ )} + {migration.warnings.length > 0 && ( +
    + {migration.warnings.map((w) => ( +
  • {w}
    • + ))} +
+ )} + + ); +} diff --git a/src/access/access.css b/src/access/access.css index a91e98e6..735448c3 100644 --- a/src/access/access.css +++ b/src/access/access.css @@ -37,3 +37,28 @@ margin: 0; padding-left: 16px; } +.access-migration td ul { + margin: 0; + padding-left: 16px; +} +.access-migration-filters { + flex-wrap: wrap; + align-items: center; + gap: 8px; +} +.access-migration-filters input[type="search"] { + background: #fff; + border: 1px solid #d3d7d1; + border-radius: 5px; + padding: 4px 8px; + min-width: 160px; +} +.access-migration-filters button { + display: inline-flex; + align-items: center; + gap: 6px; +} +.access-migration-filters button svg { + width: 16px; + height: 16px; +} diff --git a/src/access/api.ts b/src/access/api.ts index f43011dc..2ed744d1 100644 --- a/src/access/api.ts +++ b/src/access/api.ts @@ -8,3 +8,7 @@ export const inspectAccessFile = (path: string) => /** A new untitled document converted from an Access file, with the import report. */ export const importAccessFile = (path: string, options: AccessImportOptions) => call("import_access_file", { path, options }); + +/** Saves migration report text (Markdown or CSV) to `path`. */ +export const writeAccessReport = (path: string, text: string) => + call("write_access_report", { path, text }); diff --git a/src/access/dialog.ts b/src/access/dialog.ts index d1bf2e27..41c4039b 100644 --- a/src/access/dialog.ts +++ b/src/access/dialog.ts @@ -1,4 +1,4 @@ -import { open } from "@tauri-apps/plugin-dialog"; +import { open, save } from "@tauri-apps/plugin-dialog"; export const chooseAccessFile = () => open({ @@ -7,3 +7,14 @@ export const chooseAccessFile = () => directory: false, filters: [{ name: "Access databases and templates", extensions: ["accdb", "mdb", "accdt"] }], }); + +export const chooseReportDestination = (name: string, format: "md" | "csv") => + save({ + title: "Export migration report", + defaultPath: `${name}.${format}`, + filters: [ + format === "md" + ? { name: "Markdown", extensions: ["md"] } + : { name: "CSV", extensions: ["csv"] }, + ], + }); diff --git a/src/access/index.tsx b/src/access/index.tsx index 75a4a078..51012c2a 100644 --- a/src/access/index.tsx +++ b/src/access/index.tsx @@ -1 +1,2 @@ export { AccessImportWizard } from "./AccessImportWizard"; +export { AccessMigrationTab } from "./MigrationTab"; diff --git a/src/access/migration.ts b/src/access/migration.ts new file mode 100644 index 00000000..baa0c3d8 --- /dev/null +++ b/src/access/migration.ts @@ -0,0 +1,161 @@ +import { FORMAT_LABELS, KIND_ORDER, kindLabel } from "./summary"; +import type { AccessFormat, AccessMigration, ImportItem, ItemStatus } from "./types"; + +/** + * The migration report kept in `settings.accessImport` of an imported + * document (docs/decisions/access-import.md): reading it, filtering it, + * marking items reviewed, and rendering it as Markdown or CSV. + */ + +const STATUS_LABELS: Record = { + converted: "Converted", + partial: "Partly converted", + skipped: "Not converted", +}; + +export const statusLabel = (status: ItemStatus) => STATUS_LABELS[status]; + +const isRecord = (v: unknown): v is Record => + typeof v === "object" && v !== null && !Array.isArray(v); + +const strings = (v: unknown) => + Array.isArray(v) ? v.filter((x): x is string => typeof x === "string") : []; + +/** The stored format name; the importer writes "accdt" for templates. */ +const formatOf = (v: unknown): AccessFormat => + v === "jet3" || v === "jet4" || v === "ace" ? v : "template"; + +/** The migration report of a document, or null when it was not imported from Access. */ +export function readMigration(settings: unknown): AccessMigration | null { + const access = isRecord(settings) ? settings.accessImport : null; + if (!isRecord(access) || !Array.isArray(access.report)) return null; + const report = access.report.filter( + (i): i is ImportItem => isRecord(i) && typeof i.kind === "string" && typeof i.name === "string", + ); + return { + source: typeof access.source === "string" ? access.source : "", + format: formatOf(access.format), + report: report.map((i) => ({ ...i, notes: strings(i.notes) })), + warnings: strings(access.warnings), + importedAt: typeof access.importedAt === "string" ? access.importedAt : null, + reviewed: strings(access.reviewed), + }; +} + +export const itemKey = (item: Pick) => `${item.kind}:${item.name}`; + +/** Settings with `key` marked reviewed (or not); other settings are kept. */ +export function withReviewed(settings: unknown, key: string, reviewed: boolean): unknown { + const base = isRecord(settings) ? settings : {}; + const access = isRecord(base.accessImport) ? base.accessImport : {}; + const current = new Set(strings(access.reviewed)); + if (reviewed) current.add(key); + else current.delete(key); + return { ...base, accessImport: { ...access, reviewed: [...current].sort() } }; +} + +export type StatusFilter = "attention" | "all" | ItemStatus; + +export interface MigrationFilter { + status: StatusFilter; + kind: string; + search: string; + hideReviewed: boolean; +} + +const rank = (s: ItemStatus) => (s === "skipped" ? 0 : s === "partial" ? 1 : 2); +const kindRank = (kind: string) => { + const i = KIND_ORDER.indexOf(kind); + return i < 0 ? KIND_ORDER.length : i; +}; + +/** Items matching the filter, worst first, then by kind and name. */ +export function filterItems(m: AccessMigration, f: MigrationFilter): ImportItem[] { + const search = f.search.trim().toLowerCase(); + const reviewed = new Set(m.reviewed); + return m.report + .filter((i) => + f.status === "all" + ? true + : f.status === "attention" + ? i.status !== "converted" + : i.status === f.status, + ) + .filter((i) => !f.kind || i.kind === f.kind) + .filter((i) => !f.hideReviewed || !reviewed.has(itemKey(i))) + .filter( + (i) => + !search || + i.name.toLowerCase().includes(search) || + i.notes.some((n) => n.toLowerCase().includes(search)), + ) + .sort( + (a, b) => + rank(a.status) - rank(b.status) || + kindRank(a.kind) - kindRank(b.kind) || + a.name.localeCompare(b.name), + ); +} + +/** Items still needing attention that nobody marked reviewed. */ +export function openCount(m: AccessMigration): number { + const reviewed = new Set(m.reviewed); + return m.report.filter((i) => i.status !== "converted" && !reviewed.has(itemKey(i))).length; +} + +const mdCell = (s: string) => s.replace(/\|/g, "\\|").replace(/\r?\n/g, " "); + +/** The report as Markdown: a summary table, then every item that lost something. */ +export function toMarkdown(m: AccessMigration, title: string): string { + const reviewed = new Set(m.reviewed); + const lines = [`# Access migration report: ${title}`, ""]; + lines.push(`Source: ${m.source || "unknown"} (${FORMAT_LABELS[m.format] ?? m.format})`); + if (m.importedAt) lines.push(`Imported: ${m.importedAt}`); + lines.push("", "| Objects | Converted | Partly converted | Not converted |", "|---|---|---|---|"); + const kinds = [...new Set(m.report.map((i) => i.kind))].sort((a, b) => kindRank(a) - kindRank(b)); + for (const kind of kinds) { + const items = m.report.filter((i) => i.kind === kind); + const n = (s: ItemStatus) => items.filter((i) => i.status === s).length; + lines.push(`| ${kindLabel(kind)} | ${n("converted")} | ${n("partial")} | ${n("skipped")} |`); + } + const attention = filterItems(m, { + status: "attention", + kind: "", + search: "", + hideReviewed: false, + }); + if (attention.length) { + lines.push("", "## What did not convert fully", ""); + for (const item of attention) { + const mark = reviewed.has(itemKey(item)) ? " (reviewed)" : ""; + lines.push(`### ${mdCell(item.name)} (${item.kind}): ${statusLabel(item.status)}${mark}`, ""); + for (const note of item.notes) lines.push(`- ${mdCell(note)}`); + lines.push(""); + } + } + if (m.warnings.length) { + lines.push("", "## Reading problems", ""); + for (const w of m.warnings) lines.push(`- ${mdCell(w)}`); + } + return `${lines.join("\n").trimEnd()}\n`; +} + +const csvCell = (s: string) => (/[",\r\n]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s); + +/** One row per object and note: kind, name, status, reviewed, note. */ +export function toCsv(m: AccessMigration): string { + const reviewed = new Set(m.reviewed); + const rows = [["Kind", "Name", "Status", "Reviewed", "Note"]]; + const all = filterItems(m, { status: "all", kind: "", search: "", hideReviewed: false }); + for (const item of all) { + const base = [ + item.kind, + item.name, + statusLabel(item.status), + reviewed.has(itemKey(item)) ? "yes" : "no", + ]; + if (item.notes.length === 0) rows.push([...base, ""]); + for (const note of item.notes) rows.push([...base, note]); + } + return `${rows.map((r) => r.map(csvCell).join(",")).join("\r\n")}\r\n`; +} diff --git a/src/access/types.ts b/src/access/types.ts index 4ae2ad66..da070a8a 100644 --- a/src/access/types.ts +++ b/src/access/types.ts @@ -18,18 +18,36 @@ export interface AccessInventory { reports: string[]; macros: string[]; modules: string[]; - /** Objects a binary file stores compiled: [kind, name]. */ + // Objects a binary file stores compiled: [kind, name]. compiled: [string, string][]; warnings: string[]; } export type ItemStatus = "converted" | "partial" | "skipped"; +/** The ixtable object an Access object became. */ +export interface ImportTarget { + kind: "form" | "report" | "action" | "query"; + id: string; +} + export interface ImportItem { kind: string; name: string; status: ItemStatus; notes: string[]; + target?: ImportTarget; +} + +/** `settings.accessImport` of an imported document: the migration report. */ +export interface AccessMigration { + source: string; + format: AccessFormat; + report: ImportItem[]; + warnings: string[]; + importedAt: string | null; + // `kind:name` keys of items the developer marked as reviewed. + reviewed: string[]; } export interface AccessImportReport { diff --git a/src/shell/AppSettings.tsx b/src/shell/AppSettings.tsx index 4782cb7d..353a460d 100644 --- a/src/shell/AppSettings.tsx +++ b/src/shell/AppSettings.tsx @@ -1,11 +1,14 @@ import { type KeyboardEvent, useEffect, useState } from "react"; import { appInfo } from "../lib/api"; -import { type SettingsTabId, settingsTabs } from "./settings-tabs"; +import { useDocumentConfig } from "../lib/config-store"; +import { type SettingsTabId, settingsTabs as allTabs } from "./settings-tabs"; /** The `app` mode: application settings as a tab strip over `settingsTabs`. */ export function AppSettings() { const [active, setActive] = useState("assets"); const [runtime, setRuntime] = useState(""); + const { config } = useDocumentConfig(); + const settingsTabs = allTabs.filter((tab) => !tab.visible || tab.visible(config)); useEffect(() => { appInfo() .then((info) => setRuntime(`${info.name} · ${info.runtime} runtime`)) diff --git a/src/shell/settings-tabs.tsx b/src/shell/settings-tabs.tsx index 1bd28d76..c751e1a6 100644 --- a/src/shell/settings-tabs.tsx +++ b/src/shell/settings-tabs.tsx @@ -1,4 +1,6 @@ import type { ComponentType } from "react"; +import { AccessMigrationTab } from "../access"; +import { readMigration } from "../access/migration"; import { CloudTab } from "../cloud"; import { FileSourcesTab } from "../import"; import { AssetsTab, LogsTab } from "../persistence"; @@ -7,9 +9,11 @@ import { RolesTab } from "../runtime"; import { MigrationsTab } from "../migrations"; import { DatasourceTab, EntitiesTab } from "../schema"; import { ProblemsTab } from "./ProblemsTab"; +import type { DocumentConfig } from "../lib/types"; import { YamlTab } from "./YamlTab"; export type SettingsTabId = + | "access" | "assets" | "release" | "cloud" @@ -26,6 +30,8 @@ export interface SettingsTabDefinition { id: SettingsTabId; label: string; Component: ComponentType; + // Shown only for documents it applies to (default: always). + visible?: (config: DocumentConfig) => boolean; } /** Tabs of the `app` mode, in display order. Each Component lives in its owning feature dir. */ @@ -41,4 +47,10 @@ export const settingsTabs: readonly SettingsTabDefinition[] = [ { id: "yaml", label: "YAML", Component: YamlTab }, { id: "problems", label: "Problems", Component: ProblemsTab }, { id: "logs", label: "Logs", Component: LogsTab }, + { + id: "access", + label: "Access migration", + Component: AccessMigrationTab, + visible: (config) => readMigration(config.settings) !== null, + }, ]; diff --git a/tests/integration/access-import.test.tsx b/tests/integration/access-import.test.tsx index 8a6d8f46..5b3be095 100644 --- a/tests/integration/access-import.test.tsx +++ b/tests/integration/access-import.test.tsx @@ -1,7 +1,7 @@ import { mkdtempSync, readFileSync, writeFileSync } from "node:fs"; import { join, resolve } from "node:path"; import { gunzipSync } from "node:zlib"; -import { render, screen, within } from "@testing-library/react"; +import { render, screen, waitFor, within } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; import { beforeEach, expect, it } from "vitest"; import App from "../../src/App"; @@ -119,3 +119,49 @@ it("imports a template whose VBA is viewable and whose action queries run", asyn }); expect(run.changed).toBe((await readPage("Orders")).total); }); + +it("keeps a migration report in Settings with links, review marks and export", async () => { + const dialog = await openWizard(); + const dir = mkdtempSync(join(process.env.IXTABLE_STATE_DIR!, "access-")); + dialogMock.open.mockResolvedValueOnce(packTemplate(join(dir, "Order Desk.accdt"))); + await user.click(within(dialog).getByRole("button", { name: /Choose file/ })); + await within(dialog).findByRole("table", { name: "Access tables" }, LONG); + await user.click(within(dialog).getByRole("button", { name: "Import" })); + await within(dialog).findByRole("region", { name: "Access import report" }, LONG); + expect( + within(dialog).getByText(/Settings › Access migration keeps this report/), + ).toBeInTheDocument(); + await user.click(within(dialog).getByRole("button", { name: "Open document" })); + + await user.click(await screen.findByRole("button", { name: "Settings" }, LONG)); + await user.click(await screen.findByRole("tab", { name: "Access migration" }, LONG)); + const panel = await screen.findByRole("tabpanel"); + expect(within(panel).getByText(/Imported from Order Desk\.accdt, an Access template/)).toBeInTheDocument(); + const items = within(panel).getByRole("table", { name: "Migration items" }); + const first = within(items).getAllByRole("row")[1]; + const box = within(first).getByRole("checkbox", { name: /^Reviewed / }); + const summary = within(panel).getByText(/still needs? review/); + const before = summary.textContent; + await user.click(box); + await waitFor(() => expect(summary.textContent).not.toBe(before), LONG); + expect(box).toBeChecked(); + + await user.selectOptions(within(panel).getByRole("combobox", { name: /Show/ }), "all"); + await user.selectOptions(within(panel).getByRole("combobox", { name: /Kind/ }), "form"); + const forms = within(panel).getByRole("table", { name: "Migration items" }); + expect(within(forms).getAllByRole("row").length).toBeGreaterThan(1); + + const target = join(dir, "report.md"); + dialogMock.save.mockResolvedValueOnce(target); + await user.click(within(panel).getByRole("button", { name: /Export Markdown/ })); + await within(panel).findByText("Saved report.md", {}, LONG); + const md = readFileSync(target, "utf8"); + expect(md).toMatch(/^# Access migration report: /); + expect(md).toContain("| Forms |"); + expect(md).toContain("(reviewed)"); + + const open = within(forms).getAllByRole("button", { name: /^Open form / })[0]; + const name = open.getAttribute("aria-label")!.replace("Open form ", ""); + await user.click(open); + expect(await screen.findByDisplayValue(name, {}, LONG)).toBeInTheDocument(); +}); diff --git a/tests/unit/access-expressions.test.ts b/tests/unit/access-expressions.test.ts new file mode 100644 index 00000000..e26ee237 --- /dev/null +++ b/tests/unit/access-expressions.test.ts @@ -0,0 +1,28 @@ +import { describe, expect, it } from "vitest"; +import { evaluate } from "../../src/expr"; + +const record = { Amount: 42, Due: "2024-08-15", Status: "Open order", Find: "order", Priority: 2 }; + +describe("translated Access expressions", () => { + it.each([ + ["contains(record.Status, record.Find)", true], + ["startswith(record.Status, record.Find)", false], + ["endswith(record.Status, record.Find)", true], + ["upper(record.Status)", "OPEN ORDER"], + ["format(date(2000, month(record.Due), 1), 'MMMM')", "August"], + ["format(date(2000, 3, 1), 'MMM')", "Mar"], + ["floor((month(record.Due) - 1) / 3) + 1", 3], + ["year(record.Due)", 2024], + [ + "if(record.Amount > 100, 'High', if(record.Amount > 10, 'Mid', if(true, 'Low', null)))", + "Mid", + ], + [ + "if(record.Priority = 1, 'Low', if(record.Priority = 2, 'Normal', if(record.Priority = 3, 'High', null)))", + "Normal", + ], + ["if(record.Amount > 0, 1, if(record.Amount < 0, -1, 0))", 1], + ])("%s", (source, expected) => { + expect(evaluate(source, { record })).toBe(expected); + }); +}); diff --git a/tests/unit/access-migration.test.ts b/tests/unit/access-migration.test.ts new file mode 100644 index 00000000..b32f16be --- /dev/null +++ b/tests/unit/access-migration.test.ts @@ -0,0 +1,96 @@ +import { describe, expect, it } from "vitest"; +import { + filterItems, + itemKey, + openCount, + readMigration, + toCsv, + toMarkdown, + withReviewed, +} from "../../src/access/migration"; + +const settings = { + theme: "keep", + accessImport: { + source: "Orders.accdb", + format: "accdt", + importedAt: "2026-10-10T12:00:00Z", + warnings: ["page 9 could not be read"], + report: [ + { kind: "table", name: "Orders", status: "converted", notes: [] }, + { + kind: "form", + name: "Order Entry", + status: "partial", + notes: ["the On Load VBA procedure does not run"], + target: { kind: "form", id: "f1" }, + }, + { + kind: "query", + name: 'Pass, "Through"', + status: "skipped", + notes: ["pass-through | server", "Access SQL: SELECT 1"], + }, + { kind: "macro", name: "AutoExec", status: "partial", notes: ["Beep was left out"] }, + { kind: "bogus" }, + ], + }, +}; + +const all = { status: "all", kind: "", search: "", hideReviewed: false } as const; + +describe("Access migration report", () => { + it("reads the report from settings and ignores anything malformed", () => { + const m = readMigration(settings)!; + expect(m.source).toBe("Orders.accdb"); + expect(m.format).toBe("template"); + expect(m.report).toHaveLength(4); + expect(m.reviewed).toEqual([]); + expect(readMigration({})).toBeNull(); + expect(readMigration(null)).toBeNull(); + expect(readMigration({ accessImport: { report: "x" } })).toBeNull(); + }); + + it("filters worst first and searches names and notes", () => { + const m = readMigration(settings)!; + expect(filterItems(m, { ...all, status: "attention" }).map((i) => i.name)).toEqual([ + 'Pass, "Through"', + "Order Entry", + "AutoExec", + ]); + expect(filterItems(m, { ...all, kind: "form" }).map((i) => i.name)).toEqual(["Order Entry"]); + expect(filterItems(m, { ...all, search: "beep" }).map((i) => i.name)).toEqual(["AutoExec"]); + expect(filterItems(m, { ...all, status: "converted" }).map((i) => i.name)).toEqual(["Orders"]); + }); + + it("marks items reviewed without touching other settings", () => { + const key = itemKey({ kind: "macro", name: "AutoExec" }); + const next = withReviewed(settings, key, true) as typeof settings & { + accessImport: { reviewed: string[] }; + }; + expect(next.theme).toBe("keep"); + expect(next.accessImport.reviewed).toEqual(["macro:AutoExec"]); + const m = readMigration(next)!; + expect(openCount(m)).toBe(2); + expect(filterItems(m, { ...all, status: "attention", hideReviewed: true })).toHaveLength(2); + const cleared = readMigration(withReviewed(next, key, false))!; + expect(cleared.reviewed).toEqual([]); + }); + + it("renders Markdown and CSV", () => { + const m = readMigration(withReviewed(settings, "form:Order Entry", true))!; + const md = toMarkdown(m, "Order Desk"); + expect(md).toContain("# Access migration report: Order Desk"); + expect(md).toContain("| Forms | 0 | 1 | 0 |"); + expect(md).toContain("### Order Entry (form): Partly converted (reviewed)"); + expect(md).toContain("- pass-through \\| server"); + expect(md).toContain("## Reading problems"); + const csv = toCsv(m).split("\r\n"); + expect(csv[0]).toBe("Kind,Name,Status,Reviewed,Note"); + expect(csv).toContain('query,"Pass, ""Through""",Not converted,no,pass-through | server'); + expect(csv).toContain( + "form,Order Entry,Partly converted,yes,the On Load VBA procedure does not run", + ); + expect(csv).toContain("table,Orders,Converted,no,"); + }); +});