Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 35 additions & 5 deletions docs/decisions/access-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ staged copy of the data, in this order:
(`translate::dml`). A make-table target the file lacks is created by
migration 002 once the data is in. Data-definition and pass-through
queries keep their Access SQL in `settings.accessImport.actionQueries`.
Built-ins without a DuckDB function of the same name (`StrConv`,
`MonthName`, `FormatCurrency`, `Partition`, `InStrRev`, `Eval` of a
literal and others) are written out in `translate::builtins`, and custom
`Format` patterns become `strftime` or `format` calls there. Domain
functions (`translate::domain`) whose criteria are built from the row,
such as `DLookup("Price", "Products", "ID=" & [ProductID])`, become
correlated subqueries: the text pieces are parsed as one WHERE clause and
each computed piece is the outer SQL, not text.
4. Reports (`reports.rs`), macros (`macros.rs`), then forms (`forms.rs`,
`controls.rs`) convert, in that order so buttons only point at objects that
exist. Expressions go through `translate::expr` into the ixtable expression
Expand All @@ -85,6 +93,17 @@ Settings › Assets previews. The conversion report, with a status and notes
for every object, is stored in `settings.accessImport.report` and returned to
the wizard.

**Migration report.** Every report item names the object it became
(`target`: form, report, action or query, matched by name), and the
document keeps the items, reader warnings and import time in
`settings.accessImport`. Settings › Access migration, shown only for
imported documents, lists them by status, kind and text, opens the
converted object, and records review marks in
`settings.accessImport.reviewed` (`kind:name` keys) through the config store,
so they save and undo like any edit. It exports Markdown or CSV, rendered in
`src/access/migration.ts` and written by `write_access_report`, which accepts
only `.md`, `.csv` and `.txt` paths and refuses restricted runtime roles.

**Commands and UI.** `inspect_access_file` returns an inventory (tables, row
counts, objects, compiled objects, warnings) without changing anything.
`import_access_file` runs the conversion. Both run on a blocking thread. The
Expand All @@ -102,7 +121,11 @@ whether to import data, import, read the report, open the document.
- Access SQL is translated, not emulated. A call to a VBA function defined in a
module yields an empty column, with a note. A built-in function with no
DuckDB equivalent makes the query fail translation, and the report names
it.
it. `Format` follows Access for dates and numbers in US English; the
weekday and month names are English whatever the user's locale.
- A domain function's criteria must be text pieces joined with values. A
criteria held in a variable or built by a VBA function still fails, and the
report names the query.
- Text comparison in translated queries is case-insensitive, as in Access,
through `lower()`. This costs index use in large tables.
- An Access rule the data already breaks is not enforced after import. The
Expand All @@ -115,14 +138,16 @@ whether to import data, import, read the report, open the document.

- `src-tauri/src/access/tests/`: SaveAsText parsing (`text_format.rs`), template
reading and the fixture package in `tests/fixtures/access/template/`
(`accdt.rs`), page-file decoding of `orders.accdb`, `orders.mdb`, and the
(`accdt.rs`), the wider SQL and expression coverage run in DuckDB and
through `src/expr` (`translate_coverage.rs`,
`tests/unit/access-expressions.test.ts`), page-file decoding of `orders.accdb`, `orders.mdb`, and the
Jackcess `complex-data.accdb`, and refusal of password-protected and
encrypted copies of the fixtures (`jet.rs`), SQL and expression translation
(`translate.rs`), and full conversions into working documents
(`convert.rs`).
- `tests/unit/access-summary.test.ts` and
`tests/integration/access-import.test.tsx`: the wizard against the real
command through the test bridge.
- `tests/unit/access-summary.test.ts`, `tests/unit/access-migration.test.ts`
and `tests/integration/access-import.test.tsx`: the wizard and the
migration tab against the real commands through the test bridge.
- Optional corpus tests: `node scripts/access/fetch-templates.mjs <dir>` downloads
the 30 featured templates, and `IXTABLE_ACCESS_TEMPLATES_DIR=<dir> cargo test
--lib access::` imports each one and fails on any config validation error.
Expand All @@ -143,3 +168,8 @@ whether to import data, import, read the report, open the document.
action-queries record). The VBA asset has a preview in Settings › Assets.
- 2026-10-09: password-protected and encrypted files moved out of scope. Jet
3/4 page decryption removed, Jet database passwords now refused.
- 2026-10-10: wider query and expression coverage (built-ins, `Format`
patterns, row-based domain criteria, `Like` search boxes, `Switch`,
`Choose`), and the migration report tab with links, review marks and
export. The template corpus was not re-run: its host is unreachable from
the cloud build environment.
48 changes: 47 additions & 1 deletion scripts/screenshot/specs/access-import.spec.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ it("imports an Access template with its forms, report, and data", async () => {
await captureDocument(document, {
name: "access-02-report",
expectations: [
"The report says how many tables and rows the new document has.",
"The report says how many tables and rows the new document has, and that Settings › Access migration keeps it.",
"A table counts converted, partly converted, and not converted objects per kind.",
"The expanded notes explain each loss, such as the VBA module kept as an asset.",
],
Expand Down Expand Up @@ -79,4 +79,50 @@ it("imports an Access template with its forms, report, and data", async () => {
"Preview changes reports how many rows it would update, and that nothing was changed.",
],
});

await user.click(screen.getByRole("button", { name: "Settings" }));
await user.click(await screen.findByRole("tab", { name: "Access migration" }, LONG));
const panel = await screen.findByRole("tabpanel");
expect(within(panel).getByRole("table", { name: "Migration items" })).toBeVisible();
expect(within(panel).getByText(/Imported from Order Desk\.accdt, an Access template/)).toBeInTheDocument();
await captureDocument(document, {
name: "access-06-migration",
expectations: [
"Settings has an Access migration tab naming the source file and how many items still need review.",
"A summary table counts converted, partly converted and not converted objects per kind.",
"The item list shows only what needs attention, worst first, with notes, Open links and Reviewed checkboxes.",
],
});

const summary = within(panel).getByText(/still needs? review/);
const before = summary.textContent;
await user.selectOptions(within(panel).getByRole("combobox", { name: /Show/ }), "all");
await user.selectOptions(within(panel).getByRole("combobox", { name: /Kind/ }), "form");
const forms = within(panel).getByRole("table", { name: "Migration items" });
const box = within(forms)
.getAllByRole("checkbox", { name: /^Reviewed form / })
.find((b) => !b.closest("tr")?.textContent?.includes("Converted"))!;
await user.click(box);
await waitFor(() => expect(summary.textContent).not.toBe(before));
expect(box).toBeChecked();
await captureDocument(document, {
name: "access-07-migration-filtered",
expectations: [
"Show is set to Everything and Kind to Forms, so only forms are listed, converted ones included.",
"The form marked reviewed has its checkbox ticked and the review count above went down by one.",
"Each converted form has an Open form link.",
],
});

const open = within(forms).getAllByRole("button", { name: /^Open form / })[0];
const name = open.getAttribute("aria-label")!.replace("Open form ", "");
await user.click(open);
await screen.findByDisplayValue(name, {}, LONG);
await captureDocument(document, {
name: "access-08-open-form",
expectations: [
"Open form switched to Design mode with the converted Access form selected.",
"The form's name field shows the Access form's name.",
],
});
});
4 changes: 4 additions & 0 deletions src-tauri/src/access/convert/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -373,8 +373,12 @@ pub fn create(
check_queries(m, window, &config, &mut report);
actions::check_statements(m, window, &config, &mut report)?;
// The report stays with the document (Settings › YAML shows it).
report.link_targets(&config);
config.settings["accessImport"]["report"] =
serde_json::to_value(&report.items).unwrap_or_default();
config.settings["accessImport"]["warnings"] = serde_json::json!(report.warnings);
config.settings["accessImport"]["importedAt"] =
serde_json::json!(chrono::Utc::now().format("%Y-%m-%dT%H:%M:%SZ").to_string());
m.update_config(window, config)
})();
match result {
Expand Down
71 changes: 71 additions & 0 deletions src-tauri/src/access/convert/report.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ pub struct Item {
pub name: String,
pub status: Status,
pub notes: Vec<String>,
/// The ixtable object it became, for links from the migration report.
#[serde(skip_serializing_if = "Option::is_none")]
pub target: Option<Target>,
}

/// An object of the document: `kind` is form, report, action or query.
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct Target {
pub kind: &'static str,
pub id: String,
}

#[derive(Debug, Default, Clone, Serialize)]
Expand All @@ -41,9 +51,70 @@ impl ImportReport {
name: name.into(),
status,
notes,
target: None,
});
}

/// Points each item at the object it became, matched by name.
pub fn link_targets(&mut self, config: &crate::archive::DocumentConfig) {
let find = |list: Vec<(&str, &str)>, name: &str| {
list.into_iter()
.find(|(n, _)| n.eq_ignore_ascii_case(name))
.map(|(_, id)| id.to_string())
};
for item in &mut self.items {
let (kind, id) = match item.kind.as_str() {
"form" => (
"form",
find(
config
.design
.forms
.iter()
.map(|f| (f.name.as_str(), f.id.as_str()))
.collect(),
&item.name,
),
),
"report" => (
"report",
find(
config
.reports
.iter()
.map(|r| (r.name.as_str(), r.id.as_str()))
.collect(),
&item.name,
),
),
"macro" => (
"action",
find(
config
.actions
.iter()
.map(|a| (a.name.as_str(), a.id.as_str()))
.collect(),
&item.name,
),
),
"query" => (
"query",
find(
config
.saved_queries
.iter()
.map(|q| (q.name.as_str(), q.id.as_str()))
.collect(),
&item.name,
),
),
_ => continue,
};
item.target = id.map(|id| Target { kind, id });
}
}

/// Adds a note to an existing item (and downgrades a converted item to partial).
pub fn note(&mut self, kind: &str, name: &str, note: impl Into<String>) {
let note = note.into();
Expand Down
27 changes: 27 additions & 0 deletions src-tauri/src/access/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -147,5 +147,32 @@ pub async fn import_access_file(
.await
}

/// Saves the migration report text the Studio renders (Markdown or CSV).
#[tauri::command]
pub fn write_access_report(
window_label: String,
path: String,
text: String,
) -> Result<(), AppError> {
crate::manager()?.state(&window_label)?;
crate::authz::require_unrestricted(&window_label, "export the migration report")?;
write_report_text(Path::new(&path), &text)
}

/// Writes report text to a `.md`, `.csv` or `.txt` file.
pub fn write_report_text(path: &Path, text: &str) -> Result<(), AppError> {
let ext = path
.extension()
.map(|e| e.to_string_lossy().to_ascii_lowercase())
.unwrap_or_default();
if !["md", "csv", "txt"].contains(&ext.as_str()) {
return Err(AppError::new(
"VALIDATION_ERROR",
"the migration report is saved as a .md, .csv or .txt file",
));
}
std::fs::write(path, text).map_err(|e| AppError::new("IO_ERROR", e))
}

#[cfg(test)]
mod tests;
55 changes: 54 additions & 1 deletion src-tauri/src/access/tests/convert.rs
Original file line number Diff line number Diff line change
Expand Up @@ -400,7 +400,18 @@ fn binary_databases_get_generated_forms_and_working_queries() {
.iter()
.find(|i| i.name == "Unpaid Orders")
.unwrap();
assert_eq!(unpaid.status, Status::Skipped);
// DLookup with criteria built from the row is a correlated subquery.
assert_eq!(unpaid.status, Status::Converted, "{:?}", unpaid.notes);
let unpaid = config
.saved_queries
.iter()
.find(|q| q.name == "Unpaid Orders")
.unwrap();
rows(
m,
&window,
&format!("SELECT count(*) FROM ({})", unpaid.sql),
);
m.close(&window, true).unwrap();
}
}
Expand Down Expand Up @@ -589,3 +600,45 @@ fn action_queries_convert_and_run(m: &DocumentManager, config: &crate::archive::
assert!(!notes.contains("DuckDB rejects"), "{notes}");
}
}

#[test]
fn migration_report_links_objects_and_saves_as_text() {
let path = super::fixtures::template();
let (report, errs) = import(&path, "mig");
assert_eq!(errs, Vec::<String>::new());
let config = manager().config("mig").unwrap();
let stored = &config.settings["accessImport"];
assert!(stored["importedAt"]
.as_str()
.is_some_and(|t| t.ends_with('Z')));
assert!(stored["warnings"].is_array());
// Forms, reports, macros and queries point at what they became.
for item in report.items.iter().filter(|i| i.status != Status::Skipped) {
let Some(target) = &item.target else {
assert!(
!["form", "report", "query"].contains(&item.kind.as_str()),
"{item:?}"
);
continue;
};
let found = match target.kind {
"form" => config.design.forms.iter().any(|f| f.id == target.id),
"report" => config.reports.iter().any(|r| r.id == target.id),
"action" => config.actions.iter().any(|a| a.id == target.id),
_ => config.saved_queries.iter().any(|q| q.id == target.id),
};
assert!(found, "{item:?}");
}
let linked = stored["report"].as_array().unwrap();
assert!(linked.iter().any(|i| i["target"]["kind"] == "form"));
let dir = std::env::temp_dir().join(format!("ixtable-mig-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).unwrap();
crate::access::write_report_text(&dir.join("report.md"), "# Report").unwrap();
assert_eq!(
std::fs::read_to_string(dir.join("report.md")).unwrap(),
"# Report"
);
let err = crate::access::write_report_text(&dir.join("report.exe"), "x").unwrap_err();
assert_eq!(err.code, "VALIDATION_ERROR");
manager().close("mig", true).unwrap();
}
1 change: 1 addition & 0 deletions src-tauri/src/access/tests/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ mod fixtures;
mod jet;
mod text_format;
mod translate;
mod translate_coverage;
4 changes: 2 additions & 2 deletions src-tauri/src/access/tests/translate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use crate::access::translate::expr::{translate, Target};
use crate::access::translate::format::format_pattern;
use crate::access::translate::sql::{Dialect, Kind, Schema, SqlWriter};

struct Fixture;
pub(super) struct Fixture;

impl Schema for Fixture {
fn columns(&self, source: &str) -> Option<Vec<(String, Kind)>> {
Expand Down Expand Up @@ -99,7 +99,7 @@ fn functions_and_date_arithmetic() {
let sql = duck("SELECT IIf(IsNull(Status),Nz(Amount),Len(Status)) AS a, DateAdd(\"m\",1,[Order Date]) AS b, DateDiff(\"d\",[Order Date],Date()) AS c, [Order Date]+7 AS d, Format([Order Date],\"yyyy\") AS e, Int(Now()) AS f FROM Orders");
assert_eq!(
sql,
"SELECT CASE WHEN (\"Status\" IS NULL) THEN coalesce(\"Amount\", 0) ELSE length(\"Status\") END AS \"a\", (\"Order Date\" + to_months(CAST(1 AS INTEGER))) AS \"b\", date_diff('day', \"Order Date\", CAST(CAST(now() AS TIMESTAMP) AS DATE)) AS \"c\", (CAST(\"Order Date\" AS TIMESTAMP) + to_seconds(CAST(round((7) * 86400) AS BIGINT))) AS \"d\", strftime(\"Order Date\", '%Y') AS \"e\", CAST(CAST(now() AS TIMESTAMP) AS DATE) AS \"f\" FROM \"Orders\""
"SELECT CASE WHEN (\"Status\" IS NULL) THEN coalesce(\"Amount\", 0) ELSE length(\"Status\") END AS \"a\", (\"Order Date\" + to_months(CAST(1 AS INTEGER))) AS \"b\", date_diff('day', \"Order Date\", CAST(CAST(now() AS TIMESTAMP) AS DATE)) AS \"c\", (CAST(\"Order Date\" AS TIMESTAMP) + to_seconds(CAST(round((7) * 86400) AS BIGINT))) AS \"d\", strftime(CAST(\"Order Date\" AS TIMESTAMP), '%Y') AS \"e\", CAST(CAST(now() AS TIMESTAMP) AS DATE) AS \"f\" FROM \"Orders\""
);
assert_eq!(
duck("SELECT DLookup(\"Company\",\"Customers\",\"ID=1\") AS c FROM Orders"),
Expand Down
Loading
Loading