Repository navigation
OCR: fold a manual upload's subject into the purchase title and a comment - #24
Merged
Merged
Conversation
…ment
Moco drafts arrive by email import or by manual upload, where a staff
member picks a PDF and types a subject. That subject was being thrown
away -- and the draft is deleted after a successful create, so it was
gone for good. On an expense receipt it is also the only statement of
the BUSINESS PURPOSE: "Mittagessen 20.8." on a restaurant slip is the
bookkeeping-relevant fact, while the document itself can only say
"Restaurantbeleg Kueche/Bar".
The subject now rides along to the OCR call as context and comes back
merged into a new `position_title` field, which becomes both the
purchase and the line-item title. It is deliberately NOT folded into
`description` -- that stays a pure reading of the document, which the
Gutschrift and energy-credit-note paths depend on. `_build_create_payload`
falls through position_title -> description -> supplier_name, so a model
that omits the field lands exactly on the old behaviour.
Letting the model merge rather than concatenating by rule is what makes
the file-name case work: roughly half the live manual subjects are
browser file names ("trennscheibenblaetter.pdf"), and a fixed template
would stamp those onto every solar purchase. Confirmed live -- the lunch
receipt booked as "Mittagessen 20.8. - Ligu Lehm, Bern" while
"trennscheibenblaetter.pdf" was dropped in favour of the OCR reading.
The discriminator is `email_from`, NOT `user`: a forwarded email carries
both (solar draft 3213194), so keying on `user` would feed email Subject
headers to the model as if they were operator input.
Also adds the provenance comment the subject deserves: a manual upload
gets a "Manueller Upload" comment (Betreff + uploader) as the mutually
exclusive counterpart to the existing email-source comment, and the
email comment gains the Betreff line it was dropping.
The subject is employee-typed free text landing in a prompt, so it rides
in the user turn (keeping SYSTEM_PROMPT per-request identical), fenced
and labelled as data rather than instruction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WUMTJGKwUmQAYXcgcg4cVX
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…ypes `payment.note` is the remark Bexio shows against a payment, and neither branch produced text worth reading. The IBAN/QR branch read only Moco's `info` -- the QR-bill Zahlungszweck, which is empty on both live IBAN fixtures and on most invoices -- with `"-"` as the fallback. So in practice nearly every QR payment reached Bexio carrying a bare dash. The MANUAL branch composed supplier/Belegnummer/Zahlungszweck but filtered the join on `is not None`, while its parts were `x or ""` and therefore never None. Empty parts survived as dangling separators: an OCR'd card receipt with an unmatched supplier and no Zahlungszweck produced `" - 000047 - "`, which is the commonest shape this flow now generates. Both branches now call `_payment_note`, which leads with the Moco purchase `title`. On an OCR'd purchase that is `position_title` -- the business purpose folded in from a manual upload's operator subject -- and nothing else on the purchase says what the expense was actually for, which is what whoever approves the payment needs. Falls back to the composed supplier/Belegnummer/Zahlungszweck with empty parts properly dropped, then to `"-"`; capped at 80 chars, which the MANUAL branch already applied and the IBAN one did not. Both rules were faithful ports of the n8n workflow, so this is a deliberate divergence from it rather than a bug fix to the port. Left alone after review: `message` / `booking_text` / `reference_no` are payment-instruction fields, not the remark, and the bill block's setting both `message` and `reference_no` for QR (where the outgoing-payment endpoint makes them exclusive) is in the n8n export too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WUMTJGKwUmQAYXcgcg4cVX
Live 400 from POST /4.0/purchase/bills on Moco purchase 4642736: payment.booking_text size must be between 1 and 35 `booking_text` carries the purchase's `receipt_identifier` verbatim. On a Hornbach Kassenbon the OCR flow reads the till-slip transaction line as the invoice number -- "0750 19.08.2026 16:07 0011 000171 004642", 40 chars -- so every retry of that webhook failed. `message` carries the same value, so the cap is applied once at the source rather than per field. `message` may well tolerate more (the Swiss unstructured-message limit is 140), but nothing we put in it is longer than a receipt id, so there's no value in guessing at a second limit Bexio hasn't told us about. Deliberately not applied to two neighbouring fields: - `vendor_ref` is the idempotency key `_find_existing_bill` searches on. Bexio accepted the full 40 chars in the very request that rejected `booking_text`, and truncating it would make a replay miss the bill it should update rather than duplicate. - `reference_no` holds a QR reference (exactly 27 digits, already under the cap). A truncated payment reference is a wrong payment. Same cap applied in `_build_outgoing_payment_payload`, which sets `booking_text` from the same source and would hit the same wall. Note the truncated value is not pretty -- "0750 19.08.2026 16:07 0011 000171 0" is a mangled till-slip line in Bexio's journal. Fixing that properly means teaching the OCR prompt not to treat a transaction line as an invoice number, which is a separate change with idempotency consequences (`receipt_identifier` is the dedupe key on both sides). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WUMTJGKwUmQAYXcgcg4cVX
Every Telegram message in this codebase leads with a status icon -- the chat is scanned, not read -- but the four n8n-ported Bexio ones predated that convention and went out as bare text: Expense in Moco not synced to Bexio: Reason: No company given An audit of every notify() call site found exactly these four without one, so all four are fixed rather than just the reported message: the two not-synced skips and the book/payment soft failure in BexioExpenseSyncService, plus the no-customer skip in BexioInvoiceSyncService. All get warning rather than error: nothing here is a hard failure. The skips are deliberate business-rule gates, and the book/payment step is soft-failed on purpose since the bill itself is the authoritative side effect. Error stays reserved for "the thing was not created at all", which is how index.py's _app_error and the OCR service already use it. The existing skip tests assert the prefix so it can't regress. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WUMTJGKwUmQAYXcgcg4cVX
Bexio rejects an em dash on its text fields, and the position_title separator introduced two commits ago flows straight through to the bill title, the line-item title and the payment remark. The OCR prompt now asks for a plain ASCII hyphen when joining the operator subject to the document description. That alone would be a half-fix: the model writes German prose and reaches for an em dash freely, so the join is the commonest source of one but not the only one. _bexio_text therefore normalises em, en, figure, horizontal-bar and minus characters on every free-text field we send Bexio, collapses the whitespace they leave behind, and truncates last -- in that order, so the length cap counts the characters Bexio actually receives rather than ones that get rewritten afterwards. Moco keeps whatever the model wrote. This is a translation for one downstream system, not a correction of the source data. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WUMTJGKwUmQAYXcgcg4cVX
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moco drafts arrive two ways: email import (
email_fromset, already replayed into a 📧 comment) and manual upload, where a staff member picks a PDF and types a subject. That subject was being discarded — and since the draft is deleted after a successful create, it was gone for good.It's also the only statement of the business purpose. On skyr draft 3216692 the operator typed "Mittagessen 20.8."; the document itself can only say "Restaurantbeleg Küche/Bar, Barzahlung per Debit Mastercard". No amount of reading the PDF recovers the first fact.
What the live data forced
email_fromis the discriminator, notuser. Solar draft3213194is an email Romain forwarded — it carriesemail_fromanduser: Romain Kälin. Keying onuserwould feed email Subject headers to the model as if they were operator input. There's a regression test on exactly this.Roughly half of manual subjects are worthless file names, and it splits by account:
This is why the merge is done by the model rather than by a concatenation rule — a fixed
"<subject> — <description>"template would stamp260731 CKW Meierhofweg10 Rechnung 600 949 594.pdf — Stromrechnungonto every solar upload. Sorting a purpose from a file name needs the document in hand.Design
position_titlefield, not an overloadeddescription.descriptionstays a pure reading of the document, whichEnergyCreditNoteServiceand the Gutschrift path also read._build_create_payloadfalls throughposition_title → description → supplier_name → "OCR-importierte Rechnung", so a model that omits the field lands exactly on today's behaviour.BexioExpenseSyncService._truncateapplies to both the bill title and the bill line title, so nothing gets chopped mid-word downstream.SYSTEM_PROMPTper-request identical. It's employee-typed free text landing in a prompt, so it's fenced in guillemets and explicitly labelled as data, not instruction.Betreff+Hochgeladen von) is the mutually exclusive counterpart of the email-source comment. The email comment also gains theBetreff:line it was dropping — for an email draft the title is the Subject header. A title alone never renders as an email source; withoutemail_fromthat would claim a manual upload as an email.Verification
Live dry-runs through
scripts/test_ocr_create_purchase.pyagainst both real cases:description(document)position_title(booked)Mittagessen 20.8.trennscheibenblätter.pdfThe file-name subject was dropped as intended, while still being recorded in the 📎 comment as provenance.
620 passed— 12 new service tests, 7 new OCR-client tests, 2 new endpoint tests. Since the title is model-generated it can't be asserted verbatim, so the tests pin the plumbing (subject reaches the model, field reaches the payload, fallbacks hold, email subjects never leak in) and the live output is recorded in the spec.Spec:
specs/SPEC_manual_upload_subject.md(committed separately, first).🤖 Generated with Claude Code
https://claude.ai/code/session_01WUMTJGKwUmQAYXcgcg4cVX
Added in
4c697e6— the Bexio payment remarkFollow-on review of
payment.note, the remark Bexio shows against a payment. Neither branch produced text worth reading:info or "-"'-'on both real IBAN fixtures" - ".join(company, receipt, info)cut at 80'Restaurant Beispiel - R-99999 - Geschäftsessen mit Kunde'"-". The remark read Moco'sinfo— the QR-bill Zahlungszweck — which is empty on both live IBAN fixtures and on most invoices.is not None, but its parts werex or ""and so neverNone. An OCR'd card receipt with an unmatched supplier and no Zahlungszweck produced' - 000047 - '— the commonest shape this flow now generates.Both branches now call
_payment_note, which leads with the Moco purchasetitle. On an OCR'd purchase that's theposition_titlethis PR introduces, so the remark states the business purpose — the thing whoever approves the payment actually needs. Falls back to the composed supplier / Belegnummer / Zahlungszweck with empty parts dropped, then to"-"; capped at 80 for both branches (only MANUAL was capped before).Rendered against every fixture:
purchase_with_ibanVisitenkarten und Flyer CH240067780purchase_no_ibanLunch meetingpurchase_digitec_realAircondition - Rechnung 80572997Mittagessen 20.8. — Ligu Lehm, Bern000047-Both old rules were faithful ports of
reference/Sync_expenses_from_Moco_to_Bexio.json, so this is a deliberate divergence from n8n rather than a fix to the port. (The n8n original was worse:receipt_identifier||"" + " - "binds asreceipt_identifier || (" - "), concatenating with no separator at all.)Reviewed and left alone:
message/booking_text/reference_noare payment-instruction fields, not the remark. The bill payment block sets bothmessageandreference_nofor QR while_build_outgoing_payment_payloadmakes them mutually exclusive — that asymmetry is in the n8n export too, so it's upstream intent. The outgoing-payment payload gains nonote: the endpoint has no such field and adding an unvalidated one risks a 400 on an already soft-failed step.627 passed— 7 new tests (notewas asserted nowhere before). Decision recorded as D6 inspecs/SPEC_manual_upload_subject.md.