chore: sync GovOPlaN module split state

docs/CONNECTOR_SOURCE_LIFECYCLE.md

@@ -0,0 +1,159 @@
+# Connector Source Lifecycle
+
+GovOPlaN modules should treat external systems as sources with explicit
+lifecycle state. A connector profile can consume records from a source, publish
+records into a source, or do both. The lifecycle below keeps connectors
+predictable and avoids hidden module imports.
+
+## Source Directions
+
+- `consume`: GovOPlaN reads external records, normalizes them, and exposes them
+  to modules as external references, events, or staged imports.
+- `publish`: GovOPlaN creates or updates external records and stores the external
+  identifiers as immutable references.
+- `bidirectional`: GovOPlaN supports both directions with conflict detection and
+  reconciliation rules.
+
+## Source Data Lifecycle
+
+Connector profiles have operational states, while individual external records
+or source datasets move through a data lifecycle:
+
+1. `discovered`
+   A source, record, file, feed item, webhook event, or remote object is known
+   but not yet trusted for domain use.
+2. `connected`
+   GovOPlaN can authenticate and fetch or publish against the source profile.
+3. `imported`
+   Minimal source data has been staged with external id, version/ETag, source
+   timestamp, and provenance.
+4. `validated`
+   Shape, permissions, freshness, and required fields passed connector and
+   domain validation.
+5. `transformed`
+   A dataflow, workflow, or domain module normalized the staged payload into a
+   domain-specific form.
+6. `published`
+   GovOPlaN exposed or wrote an output through API, RSS, report, export, or a
+   downstream connector.
+7. `archived`
+   The source/output is no longer active but remains available under retention,
+   audit, and external-reference rules.
+8. `deprecated`
+   The source/output remains readable for history but must not be used for new
+   workflows.
+
+Every transition must preserve provenance, permissions context, freshness, and
+audit trace. Domain modules may add stricter states, but they should map back to
+this lifecycle when a connector publishes status.
+
+## Lifecycle States
+
+1. `draft`
+   Profile exists but is not used by runtime jobs.
+2. `configured`
+   Required endpoint and credential references are present.
+3. `tested`
+   A health/test run succeeded and recorded non-secret diagnostics.
+4. `active`
+   Runtime jobs may consume or publish data.
+5. `degraded`
+   The connector is active but health checks or recent jobs show failures.
+6. `paused`
+   Operators intentionally stop scheduled connector activity.
+7. `retiring`
+   The connector is being removed from active workflows while references remain
+   readable.
+8. `retired`
+   No new runtime activity is allowed. Historical references remain available.
+
+## State Transition Gates
+
+| Transition | Required Evidence | Blockers |
+| --- | --- | --- |
+| `draft` -> `configured` | endpoint fields are valid, credential references exist, owner/tenant scope is set | plaintext secret in profile payload, unsupported connector type |
+| `configured` -> `tested` | latest profile test succeeded and diagnostics were redacted | failed auth, unreachable endpoint, TLS/policy error |
+| `tested` -> `active` | operator enabled runtime use, required modules/capabilities are present, schedule/webhook is valid | missing module, missing permission, no idempotency strategy for publish jobs |
+| `active` -> `degraded` | health check or job telemetry reports failures | none; this is automatic diagnostic state |
+| `degraded` -> `active` | health/test succeeds or failed jobs are reconciled | unresolved conflict or repeated failure threshold |
+| any running state -> `paused` | operator pause request or maintenance preflight | active critical transaction that cannot be interrupted |
+| `paused` -> `active` | successful re-test when credentials/endpoints changed | failed profile test |
+| any state -> `retiring` | uninstall/disable plan accepted, schedulers/workers stopped | active domain references that require operator decision |
+| `retiring` -> `retired` | non-destructive retirement complete, references remain readable | destructive retirement requested without provider and backup |
+
+## Consume Flow
+
+1. Discover changes through polling, webhook, batch upload, or manual operator
+   action.
+2. Fetch only the minimal remote data required for the declared use case.
+3. Normalize into a connector-owned staging payload.
+4. Validate shape, required fields, and source trust level.
+5. Emit a core-mediated event such as `connector.record_discovered`.
+6. Let domain modules claim or transform staged data through capabilities, not
+   imports.
+7. Store external references with source system, object type, object id, version
+   or ETag, and last-seen timestamp.
+
+## Publish Flow
+
+1. Domain module requests publish through a core-mediated connector capability.
+2. Connector validates profile state, permission, idempotency key, and payload
+   shape.
+3. Connector sends the remote request.
+4. Connector stores the remote id, version/ETag, and response diagnostics.
+5. Connector emits `connector.record_published` or `connector.publish_failed`.
+6. Domain module stores only the external-reference DTO and any domain result.
+
+## Reconciliation
+
+Every connector that writes to an external system needs a reconciliation story:
+
+- idempotency key for create/update jobs
+- remote object version, ETag, or last-modified value where available
+- conflict state when local and remote records diverge
+- retry policy for temporary failures
+- explicit operator action for destructive overwrite or deletion
+- audit trace from GovOPlaN record to external request and response summary
+
+## Capability Boundary
+
+Domain modules must not import connector implementation packages directly. They
+should ask core for capabilities such as:
+
+- `connectors.catalog`
+- `connectors.profileTester`
+- `connectors.health`
+- `connectors.externalReferences`
+- `connectors.sourceConsumer`
+- `connectors.sourcePublisher`
+
+Connector payloads should be DTOs or protocol objects from kernel/core
+contracts. Protocol-specific clients stay inside the connector module that owns
+them.
+
+## Safety Rules
+
+- Secret values never leave the secret contract and are never stored in test
+  result payloads.
+- Runtime jobs must include profile id, connector type, direction, idempotency
+  key, and triggering principal/system actor.
+- Profile tests must redact tokens, passwords, cookies, authorization headers,
+  and remote personal data not needed for diagnostics.
+- Deactivation must stop schedulers/workers before profile removal.
+- Uninstall defaults to non-destructive retirement; domain data and external
+  references remain readable.
+- Destructive retirement requires a module-owned retirement provider and an
+  explicit operator choice.
+
+## Release Checklist
+
+Before shipping an executable connector type:
+
+- Add catalogue metadata and capability names.
+- Add profile schema validation that rejects plaintext secrets.
+- Add redaction tests for success and failure diagnostics.
+- Add unavailable-optional-module tests for every consuming domain module.
+- Add profile test and health status fixtures.
+- Add external-reference DTO tests.
+- Add lifecycle transition tests for pause, retry, retirement, and uninstall
+  guard behavior.