diff --git a/Repo-docs-connectors.-.md b/Repo-docs-connectors.-.md new file mode 100644 index 0000000..a92c8ea --- /dev/null +++ b/Repo-docs-connectors.-.md @@ -0,0 +1,98 @@ + + +> Mirrored from `/mnt/DATA/git/mousehold/docs/connectors.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Connector Boundary + +Mousehold treats connectors as evidence producers. A connector submits `RawObservation` +payloads; the backend turns them into reviewable `FinancialEvent` candidates and links +the source evidence. + +## Local-First Rule + +Bank and mailbox credentials should stay in the local agent where possible. +The API stores: + +- connection type and display name +- owner user and household +- sync status and last error +- a local secret reference, not the secret itself +- normalized observations + +## Current PoC Routes + +- Manual entry: `POST /observations/manual` +- Generic normalized import: `POST /observations/import` +- Local-agent batch sync: `POST /observations/sync` +- Connector metadata: `POST /connections` +- Local-agent registration: `POST /local-agent/register` +- Browser IMAP folder listing: `POST /agent/imap/folders` +- Browser IMAP scan: `POST /agent/imap/scan` +- PayPal CSV statement parsing: `POST /agent/paypal/statement/parse` +- Klarna CSV statement parsing: `POST /agent/klarna/statement/parse` + +## Data Source Mapping + +- Bank CSV, FinTS, PSD2: confirmed cash movement observations. FinTS fetching is implemented in the local agent. +- PayPal CSV: confirmed PayPal account observations. +- PayPal email: announced or outstanding purchase/payment evidence. +- Klarna email: usually outstanding obligation evidence. +- Order confirmation email: announced purchase evidence. + +Bank debits that fund PayPal, Klarna, or cards should be linked as settlements or +liability payments instead of duplicated as new expenses. + +## Reconciliation + +The reconciliation service computes reviewable suggestions from imported events +instead of linking automatically. Current suggestions cover: + +- PayPal/Klarna/order email evidence matched to PayPal or Klarna statement rows +- PayPal/Klarna statement rows matched to FinTS, bank CSV, or PSD2 cash movement +- exact amount/currency matches with date windows, party overlap, provider names, + and shared references + +Accepting a suggestion creates an event relation, attaches the confirming +observation to the primary event as evidence, and upgrades the primary event to +`confirmed` when the confirming event is confirmed. The confirming event is kept +for auditability; later UI cleanup can decide how aggressively to collapse or +hide duplicate-looking rows. + +## IMAP Flow + +The web app has a `Mail` panel for IMAP proof-of-concept imports. It sends IMAP +credentials to the configured agent route for that request only, selects the +requested folder read-only, scans recent messages, and returns previews plus +normalized observations. The backend stores observations only after the user +clicks import. + +The parser currently recognizes EUR amounts, simple due-date phrases, Klarna +references, PayPal transaction IDs, order numbers, and invoice numbers. Email +evidence is classified as `announced` unless it is a Klarna or due-date driven +promised payment, which becomes `outstanding`. Confirmed cash movement should +still come from FinTS, bank CSV, PSD2, or PayPal/Klarna statement CSV. + +## PayPal Statement Flow + +The active PayPal path is statement and email based. + +The `Statements` panel supports PayPal Activity Download CSV exports. It +recognizes fields such as `Date`, `Name`, `Type`, `Status`, `Currency`, `Gross`, +`Fee`, `Net`, and `Transaction ID`, then imports confirmed `paypal_csv` +observations. + +PayPal emails remain early evidence for announced or outstanding obligations. +Later PayPal statements and bank debits should confirm or settle those events +instead of creating duplicate household expenses. + +## Klarna Statement Flow + +The `Statements` panel also supports Klarna CSV. It handles Klarna Merchant +settlement reports with a summary section followed by a `type`/`amount` +transaction table, and a simpler consumer-style CSV with date, description, +amount, currency, reference, and type columns. Klarna CSV observations use +`source_type=klarna_csv` and are treated as confirmed statement evidence, while +Klarna emails remain the early outstanding/announced signal.