first ruggedy draft

docs/connectors.md

@@ -0,0 +1,91 @@
+# Connector Boundary
+
+Mousehold treats connectors as evidence producers. A connector submits `RawObservation`
+payloads; the backend turns them into reviewable `FinancialEvent` candidates and links
+the source evidence.
+
+## Local-First Rule
+
+Bank and mailbox credentials should stay in the local agent where possible.
+The API stores:
+
+- connection type and display name
+- owner user and household
+- sync status and last error
+- a local secret reference, not the secret itself
+- normalized observations
+
+## Current PoC Routes
+
+- Manual entry: `POST /observations/manual`
+- Generic normalized import: `POST /observations/import`
+- Local-agent batch sync: `POST /observations/sync`
+- Connector metadata: `POST /connections`
+- Local-agent registration: `POST /local-agent/register`
+- Browser IMAP folder listing: `POST /agent/imap/folders`
+- Browser IMAP scan: `POST /agent/imap/scan`
+- PayPal CSV statement parsing: `POST /agent/paypal/statement/parse`
+- Klarna CSV statement parsing: `POST /agent/klarna/statement/parse`
+
+## Data Source Mapping
+
+- Bank CSV, FinTS, PSD2: confirmed cash movement observations. FinTS fetching is implemented in the local agent.
+- PayPal CSV: confirmed PayPal account observations.
+- PayPal email: announced or outstanding purchase/payment evidence.
+- Klarna email: usually outstanding obligation evidence.
+- Order confirmation email: announced purchase evidence.
+
+Bank debits that fund PayPal, Klarna, or cards should be linked as settlements or
+liability payments instead of duplicated as new expenses.
+
+## Reconciliation
+
+The reconciliation service computes reviewable suggestions from imported events
+instead of linking automatically. Current suggestions cover:
+
+- PayPal/Klarna/order email evidence matched to PayPal or Klarna statement rows
+- PayPal/Klarna statement rows matched to FinTS, bank CSV, or PSD2 cash movement
+- exact amount/currency matches with date windows, party overlap, provider names,
+  and shared references
+
+Accepting a suggestion creates an event relation, attaches the confirming
+observation to the primary event as evidence, and upgrades the primary event to
+`confirmed` when the confirming event is confirmed. The confirming event is kept
+for auditability; later UI cleanup can decide how aggressively to collapse or
+hide duplicate-looking rows.
+
+## IMAP Flow
+
+The web app has a `Mail` panel for IMAP proof-of-concept imports. It sends IMAP
+credentials to the configured agent route for that request only, selects the
+requested folder read-only, scans recent messages, and returns previews plus
+normalized observations. The backend stores observations only after the user
+clicks import.
+
+The parser currently recognizes EUR amounts, simple due-date phrases, Klarna
+references, PayPal transaction IDs, order numbers, and invoice numbers. Email
+evidence is classified as `announced` unless it is a Klarna or due-date driven
+promised payment, which becomes `outstanding`. Confirmed cash movement should
+still come from FinTS, bank CSV, PSD2, or PayPal/Klarna statement CSV.
+
+## PayPal Statement Flow
+
+The active PayPal path is statement and email based.
+
+The `Statements` panel supports PayPal Activity Download CSV exports. It
+recognizes fields such as `Date`, `Name`, `Type`, `Status`, `Currency`, `Gross`,
+`Fee`, `Net`, and `Transaction ID`, then imports confirmed `paypal_csv`
+observations.
+
+PayPal emails remain early evidence for announced or outstanding obligations.
+Later PayPal statements and bank debits should confirm or settle those events
+instead of creating duplicate household expenses.
+
+## Klarna Statement Flow
+
+The `Statements` panel also supports Klarna CSV. It handles Klarna Merchant
+settlement reports with a summary section followed by a `type`/`amount`
+transaction table, and a simpler consumer-style CSV with date, description,
+amount, currency, reference, and type columns. Klarna CSV observations use
+`source_type=klarna_csv` and are treated as confirmed statement evidence, while
+Klarna emails remain the early outstanding/announced signal.

docs/fints.md

@@ -0,0 +1,133 @@
+# FinTS Connector
+
+The FinTS connector can run embedded in the main API or in the separate local
+agent. Embedded mode is the default for local/self-hosted installs, so users do
+not need to start a second process.
+
+The privacy tradeoff is explicit:
+
+- Embedded mode: the API process receives PIN/TAN inputs while a FinTS job runs.
+- Separate local-agent mode: PIN/TAN inputs stay in the separate local process.
+
+In both modes, PINs are not stored in the database. Normalized observations are
+stored only when you import them.
+
+## Required Inputs
+
+- `--blz` or `MOUSEHOLD_FINTS_BLZ`: bank identifier/BLZ.
+- `--server` or `MOUSEHOLD_FINTS_SERVER`: FinTS endpoint URL.
+- `--bank-user-id` or `MOUSEHOLD_FINTS_USER_ID`: your bank login ID.
+- PIN: prompted interactively by default, or provided through `MOUSEHOLD_FINTS_PIN`.
+- `--product-id` or `MOUSEHOLD_FINTS_PRODUCT_ID`: required by python-fints 4+.
+  Register a product ID with Deutsche Kreditwirtschaft/ZKA before connecting.
+- Optional `--customer-id`, `--tan-medium`, and `--product-version`.
+
+## comdirect
+
+The web UI has a `comdirect` preset.
+
+Current values:
+
+```text
+BLZ: 20041111
+FinTS endpoint: https://fints.comdirect.de/fints
+Bank user ID: your comdirect Zugangsnummer
+Customer ID: leave empty
+```
+
+If your account uses a different comdirect/Commerzbank GF comdirect BLZ, replace
+the preset BLZ before fetching accounts.
+
+## Local State
+
+The connector stores non-PIN client state in:
+
+```text
+~/.config/mousehold/fints/<profile>.json
+```
+
+This caches bank parameter data, selected TAN mechanism, and selected TAN
+medium so repeated syncs do not ask the same setup questions. The file is
+written with mode `0600`.
+
+Use `--no-state` for a one-off session without reading or writing that cache.
+
+## Commands
+
+The browser UI uses the embedded API route by default:
+
+```text
+http://127.0.0.1:8001/agent/fints/...
+```
+
+To use a separate local-agent HTTP bridge instead:
+
+```sh
+mousehold-agent serve --host 127.0.0.1 --port 8765
+```
+
+Then set `VITE_LOCAL_AGENT_URL=http://127.0.0.1:8765` in
+`apps/web/.env.development`.
+
+List accounts:
+
+```sh
+mousehold-agent fints-accounts \
+  --profile my-bank \
+  --blz <bank-blz> \
+  --server <fints-endpoint-url> \
+  --bank-user-id <bank-login-id> \
+  --with-balances
+```
+
+Display transactions:
+
+```sh
+mousehold-agent fints-transactions \
+  --profile my-bank \
+  --blz <bank-blz> \
+  --server <fints-endpoint-url> \
+  --bank-user-id <bank-login-id> \
+  --iban <iban> \
+  --start-date 2026-06-01 \
+  --end-date 2026-06-27
+```
+
+Sync transactions into Mousehold:
+
+```sh
+mousehold-agent sync-fints \
+  --api-url http://127.0.0.1:8001 \
+  --connection-id <connection-id> \
+  --household-id <household-id> \
+  --owner-user-id <user-id> \
+  --source-account-id <account-id> \
+  --profile my-bank \
+  --blz <bank-blz> \
+  --server <fints-endpoint-url> \
+  --bank-user-id <bank-login-id> \
+  --iban <iban> \
+  --days 30 \
+  --display
+```
+
+## TAN Handling
+
+If the bank returns a TAN challenge, the CLI prints the challenge text and asks
+for the TAN. Decoupled app confirmation flows prompt you to confirm in the
+banking app and press Enter. Binary challenge images are written under:
+
+```text
+~/.config/mousehold/fints/challenges/
+```
+
+## Ledger Mapping
+
+FinTS transactions are imported as `source_type = "fints"` and
+`status = "confirmed"`.
+
+- Positive bank movements become `income`.
+- Ordinary negative bank movements become `expense`.
+- Negative movements mentioning PayPal, Klarna, or credit-card providers become
+  `liability_payment`, so they can later be linked to prior obligations instead
+  of becoming duplicate household expenses.