diff --git a/README.md b/README.md index f738a04..2d3c2ac 100644 --- a/README.md +++ b/README.md @@ -173,6 +173,9 @@ The current executable slice and remaining production gates are documented in [Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md). Same-host replica balancing and the multi-host promotion boundary are documented in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md). +Create, update, pause, resume, verify and remove a local or multi-hypervisor K3s +VM target with the guarded lifecycle documented in +[Kubernetes VM Test Lab](docs/KUBERNETES_TEST_LAB.md). The recovery state machine, migration rollback boundary, and required restore drills are documented in [Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md). diff --git a/docs/ASSISTED_AND_NON_DIGITAL_CHANNELS.md b/docs/ASSISTED_AND_NON_DIGITAL_CHANNELS.md new file mode 100644 index 0000000..40c0415 --- /dev/null +++ b/docs/ASSISTED_AND_NON_DIGITAL_CHANNELS.md @@ -0,0 +1,167 @@ +# Assisted and Non-Digital Channels + +## Purpose + +GovOPlaN must support people who cannot or do not use a self-service portal. +Telephone, paper, in-person service, authorized representation, mobile staff, +interpreters, and temporary offline work are not exceptional side systems. +They are governed channels into the same service, case, workflow, record, and +decision. + +The goal is equivalent institutional treatment, not forced channel identity. +The system preserves which channel was used and which evidence is available +without giving digitally confident users stronger substantive rights. + +The first end-to-end journey is tracked in +[GovOPlaN #42](https://git.add-ideas.de/GovOPlaN/govoplan/issues/42). + +## Actor Model + +Every assisted interaction distinguishes: + +- the affected person or organization; +- the real staff member or external helper entering information; +- the represented party and representation basis; +- an interpreter, witness, guardian, or support person where relevant; +- the responsible institutional function; +- the channel and location; +- the person who reviewed or confirmed the captured information. + +"Entered by" is not "declared by". "Declared by" is not "verified by". +Authentication assurance, representation authority, and evidence quality are +separate fields. + +## Channel-Neutral Intake Contract + +All channels create the same versioned service/form submission contract with +additional provenance: + +- service, form, schema, language, and accessibility version; +- valid and recorded time; +- channel (`portal`, `counter`, `telephone`, `paper`, `email`, `mobile`, + `representative`, `offline_import`, or configured extension); +- affected and represented parties; +- capture actor and responsible function; +- consent, notice, purpose, legal basis, and information source; +- field-level source and confidence where staff transcribed or inferred data; +- attachments, scans, originals, signatures, recordings, and attestations as + governed evidence references; +- read-back/confirmation result and correction path; +- receipt and chosen return channels; +- duplicate/matching assessment and any manual resolution. + +Forms Runtime owns the submission lifecycle. Parties owns procedural capacity +and representation. Identity/Addresses own subject and contact references. +Cases owns the matter. Records owns filing and retention. Audit preserves the +action/effect evidence. + +## Assisted Session + +An assisted session is a resumable work item, not a privileged bypass. It: + +1. selects service, language, channel, affected party, and represented capacity; +2. shows the staff member only fields and evidence relevant to the service; +3. explains why sensitive data is requested and what evidence quality is + required; +4. records source per value when information comes from speech, paper, an + existing register, or staff observation; +5. validates and previews consequences before submission; +6. supports read-back, correction, confirmation, and a second-person check + where policy requires it; +7. generates an accessible receipt through the requested channel; +8. creates follow-up tasks when original documents, signatures, translation, + or verification remain outstanding. + +The helper's normal account and represented function remain in the audit +chain. Assistance never grants access to unrelated records about the person. + +## Paper And Scanning + +- Register receipt before scanning so custody and deadlines do not depend on + successful OCR. +- Store the original scan or external archive reference with digest, pages, + capture device/provider, time, operator, and quality assessment. +- Treat OCR and extracted fields as derived data with confidence and source + coordinates. A person confirms consequential values. +- Support separation, ordering, missing-page, duplicate, malware, and + readability review. +- File the resulting document and submission into the appropriate eAkte; + retain or return the physical original according to policy. +- Produce cover sheets, barcodes, and return instructions through Templates, + not a separate print domain. + +## Telephone And In-Person Handling + +- Show a scripted but adaptable interview from the same Form definition. +- Record how identity and representation were checked; do not equate caller ID + with identity proof. +- Require explicit confirmation of consequential declarations and capture the + method (read-back, signed summary, one-time code, witness, later letter). +- Record call audio only when a lawful, declared profile permits it; an + interaction note is the default. +- Make interrupted sessions resumable without exposing prior answers to an + unauthorized caller or visitor. + +## Offline And Mobile Work + +Offline packages are encrypted, device-bound, time-limited, purpose-limited, +and contain only the required forms/reference data. Synchronization uses +idempotent intents and exposes conflicts rather than last-write-wins. Device +loss, expiry, revocation, duplicate submission, clock drift, and outcome +unknown have explicit recovery paths. + +## Outbound Non-Digital Delivery + +Campaign and Postbox model one delivery intent with channel choices and policy: + +- portal/postbox delivery; +- email; +- print and postal fulfillment through a managed provider or local handoff; +- in-person collection; +- telephone notification followed by durable confirmation; +- accessible or language-specific variants. + +Distribution preferences are purpose- and service-specific, effective-dated, +and may be overridden only by a documented legal or urgent-delivery rule. A +fallback occurs only before a channel has accepted the effect unless policy +explicitly authorizes duplicate delivery. Receipts distinguish creation, +provider acceptance, dispatch, delivery, return, and acknowledgement. + +## Accessibility And Equality + +- The person can request language, easy-language, large-print, screen-reader, + sign-language, relay, interpreter, or representative support without those + preferences becoming a general-purpose profile visible everywhere. +- Staff interfaces support keyboard-only capture, clear focus, error summary, + read-back, and printable/offline alternatives. +- Channel choice and need for assistance must not be used as an adverse risk + signal. +- Reports compare completion, wait, correction, abandonment, and outcome by + channel only under a declared equality/service-quality purpose and with + privacy thresholds. + +## Security And Abuse Controls + +- purpose-aware field access and session timeout; +- current authority checks for every read and effect; +- dual control for high-risk identity, payment, address, or representation + changes; +- immutable source/attestation evidence and correction history; +- rate and anomaly controls that do not silently reject a person; +- explicit safe handling of domestic-abuse, protected-address, witness, or + sealed-record cases; +- no secret answers or full documents in ordinary operational logs. + +## First Reference Journey + +Implement the permit-to-payment/service-to-decision journey through three +equivalent starts: + +1. self-service portal submission; +2. staff-assisted counter/telephone submission; +3. paper receipt, scan, extraction, confirmation, and filing. + +All three must create the same Case and Workflow contract, preserve different +provenance, support correction, produce a receipt, file an eAkte, reach the same +decision rules, and prove accessibility, privacy, recovery, and channel +fallback in browser and operator tests. diff --git a/docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md b/docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md index 7f35f95..835330f 100644 --- a/docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md +++ b/docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md @@ -1,5 +1,11 @@ # GovOPlaN Capability and IT-Infrastructure Fit Assessment +> **Pinned historical evidence:** This document assesses the exact 2026-07-22 +> Campaign composition below. It is intentionally not updated to describe later +> main-branch work. Use [Strategy Status](STRATEGY_STATUS.md) for the current +> cross-product reconciliation and create a new dated fit assessment for a new +> target composition. + ## Assessment record | Field | Value | @@ -19,8 +25,8 @@ Datasources, Dataflow, Search, encryption contracts, and other later main-branch work must not be inferred into this evidence record. The current product direction and implemented-state reconciliation are documented separately in -the -[Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md). +the [Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md) +and [Strategy Status](STRATEGY_STATUS.md). This is a fit assessment, not a production approval or security certification. It deliberately does not infer implementation from a repository, issue, or diff --git a/docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md b/docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md index 9cfae1a..9b535f8 100644 --- a/docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md +++ b/docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md @@ -18,7 +18,8 @@ Read it together with: - the [institutional governance target architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md) - the [selected reference-journey program](REFERENCE_JOURNEY_PROGRAM.md) -- the [current capability and infrastructure fit assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md) +- the [current strategy status](STRATEGY_STATUS.md) +- the [pinned Campaign capability and infrastructure fit assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md) - the [interface pattern language](INTERFACE_PATTERN_LANGUAGE.md) - the [interface surface inventory](INTERFACE_SURFACE_INVENTORY.md) - the [module contract and install model](MODULE_CONTRACTS_AND_INSTALLS.md) @@ -41,8 +42,8 @@ Read it together with: - Use [Near-term portfolio order](#near-term-portfolio-order) for the bridge to implementation and [Product decisions](#product-decisions-to-make-progressively) for choices that can remain deferred. -- Use the [dated snapshot appendix](#snapshot-appendix-2026-07-20) only to - understand which live backlog and release facts informed this revision. +- Use the [dated strategic review](STRATEGIC_REVIEW_2026-08-05.md) to understand + why the current convergence and reference-journey order was chosen. ### Planning ownership @@ -50,7 +51,7 @@ Read it together with: | --- | --- | | What product should GovOPlaN become, for whom, in which configurations, and through which outcome horizons? | This meta roadmap | | Which module owns a capability, which technical wave should deliver it, and what implementation gates apply? | The Core master roadmap and owning-module concepts | -| What is actively planned, blocked, implemented, or closed now? | Gitea issues | +| What is actively planned, blocked, implemented, or closed now? | Gitea issues and the dated reconciliation in `STRATEGY_STATUS.md` | | What can a named composition credibly claim in a target environment? | A dated capability/infrastructure fit assessment | The horizons and near-term order below express product outcomes and portfolio @@ -1351,71 +1352,10 @@ language, what service it configured, who can act, which systems participate, what happens when they fail, how a decision can be reviewed, and where the evidence remains—and the product can prove that explanation at runtime. -## Snapshot appendix: 2026-07-20 +## Dated Context -This appendix records volatile facts that informed this revision. It is not a -second source of truth and should be refreshed or removed when a later roadmap -review uses a new release/backlog snapshot. - -### Composition and release snapshot - -The cross-repository contract scan found 43 module manifest contracts, 29 -provided interface names, 16 requirements, and no contract error across 65 -scanned repositories. That is meaningful composition evidence, but the release -metadata trailed the integrated code: Core, Policy, Poll, and Scheduling -declared `0.1.9` while the whole-product release requirements remained on -module tag `v0.1.8`; the root self-hosted `.env.example` and release smoke -composition did not yet exercise all installed release modules. Other -development compositions already included some of those modules. This was a -release/composition gap, not evidence that the underlying slices did not exist. - -### Backlog snapshot - -The Gitea audit found 206 open issues across 36 of 66 catalogued repositories -and 362 closed issues. Campaign had 51 open issues and Core 44; together they -held 46% of current work. This reflected substantial completed kernel, -security, and platform work and a deliberate concentration on the first usable -vertical, but also risked crowding out production evidence and the shared -process spine. - -The issue workflow needed a reconciliation pass before another delivery -program could be inferred from labels: 119 open issues remained in triage, 116 -had no milestone, and several recently pushed Calendar, Scheduling, Poll, -Campaign, and Files slices still described themselves as local or awaiting -integration. Conversely, 30 repositories had no open issue; for many -later-wave modules this meant no implementation program had been opened, not -that the capability was complete. - -[Poll #2](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/2) was a clear -tracker-drift example: its configurable transition engine, agreed transition -matrix/history, idempotent keyed retries, re-decision audit, archive/unarchive, -and preservation behavior were implemented and pushed while the issue still -reported `needs-info`. - -Issue anchors that informed the bridge from the baseline into this roadmap: - -- [Meta #10](https://git.add-ideas.de/GovOPlaN/govoplan/issues/10) for the - capability/infrastructure assessment and its target proof; -- [Meta #11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) for the - universal interface and focused-view direction; -- [Core #225](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/225) for - guided, safe configuration; -- [Core #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29) for the - backup/restore production gate; -- [Core #263](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/263) and - [Campaign #63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63), - [#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62), - [#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65), and - [#69](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/69) for the - reference interface/delivery vocabulary and behavior; -- [Poll #1](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/1) for the - database-enforced respondent invariant exposed by Scheduling; -- [Connectors #6](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/6) - for the governed connector configuration/simulation foundation; -- [Meta #9](https://git.add-ideas.de/GovOPlaN/govoplan/issues/9) for the first - permit-to-payment reference process; and -- [Meta #12](https://git.add-ideas.de/GovOPlaN/govoplan/issues/12) for the - deliberately deferred, consumer-independent export-control story. - -Live Gitea issue state remains canonical. These dated facts explain the roadmap -sequence only. +The volatile release and backlog appendix that originally accompanied this +roadmap has been removed so the durable direction cannot become a competing +status source. The [Strategic Review 2026-08-05](STRATEGIC_REVIEW_2026-08-05.md) +retains the dated assessment and reasoning. Current reconciliation belongs in +[Strategy Status](STRATEGY_STATUS.md), and live work state belongs in Gitea. diff --git a/docs/FEDERATED_GOVOPLAN_ARCHITECTURE.md b/docs/FEDERATED_GOVOPLAN_ARCHITECTURE.md new file mode 100644 index 0000000..9d1839f --- /dev/null +++ b/docs/FEDERATED_GOVOPLAN_ARCHITECTURE.md @@ -0,0 +1,155 @@ +# Federated GovOPlaN Architecture + +## Purpose + +Federation lets autonomous GovOPlaN installations exchange data, +configuration, work, messages, records, and evidence without sharing a database +or surrendering local policy. It is institution-to-institution cooperation, +not multi-tenancy across an untrusted network. + +The first implementation should prove a bounded exchange between two +installations. A new federation module is not justified until the shared +protocol has at least two independent consumers. Core owns neutral envelopes +and trust contracts; Connectors owns transport providers; domain modules own +the objects and effects they exchange. + +Implementation is tracked in +[GovOPlaN #41](https://git.add-ideas.de/GovOPlaN/govoplan/issues/41). + +## Invariants + +1. Every installation remains authoritative for its tenants, identities, + policies, keys, records, and local mappings. +2. A remote identity or permission never becomes a local authorization claim. +3. Every exchange declares purpose, legal/organizational basis, classification, + minimization, retention expectation, and permitted onward use. +4. Every object reference identifies origin instance, owner tenant, object type, + object ID, exact revision, and source-authority mode. +5. Payloads and receipts are signed; sensitive transports use mutually + authenticated encrypted channels. +6. Acceptance, rejection, outcome unknown, retry, revocation, correction, and + reconciliation are durable states. +7. Local policy may reject or narrow a remote request. It cannot silently claim + to have accepted an effect that did not occur. +8. Federation works asynchronously and can exchange signed offline bundles + where continuous connectivity is unavailable. + +## Trust Domains + +An instance publishes a signed, versioned federation descriptor containing: + +- stable instance and operator identity; +- supported protocol and schema versions; +- signing and transport key identifiers with rotation history; +- accepted object and exchange profiles; +- endpoint locations and size/rate limits; +- support, incident, revocation, and data-protection contacts; +- evidence and conformance references. + +Pairing is a two-sided administrative workflow. Each side verifies the other, +maps the remote institution to a local trusted-party record, selects permitted +profiles and purposes, sets policy ceilings, and records approvals. Trust is +directional and profile-specific; trusting signed Postbox delivery does not +automatically permit case transfer or configuration import. + +## Exchange Envelope + +Every request, response, receipt, correction, and revocation uses one neutral +envelope with: + +- message ID, correlation ID, causation ID, creation and expiry; +- origin and destination instance/institution/tenant references; +- real actor and represented institutional capacity where disclosure is + permitted; +- exchange profile and semantic schema version; +- exact domain object references and content digests; +- purpose, legal basis, classification, data categories, retention expectation, + onward-transfer constraint, and subject notice status; +- requested action and idempotency key; +- encryption recipients and signature chain; +- attachment/object manifests rather than unbounded embedded blobs; +- previous-envelope references for correction, replacement, or revocation. + +The envelope is evidence, not a universal domain object. Each owner validates +and imports or links its own payload. + +## Exchange Profiles + +| Profile | First owners | Behavior | +| --- | --- | --- | +| Postbox delivery | Postbox, Campaign, Notifications | Address or derive a remote function-bound postbox, obtain acceptance receipt, and track acknowledgement where permitted | +| Case handoff | Cases, Parties, Services, Workflow Engine | Offer exact context and evidence; destination accepts into a new local case and returns the mapping | +| Record transfer | Records, Files, DMS, Audit | Transfer or offer a signed record package with file-plan, metadata, content digests, holds, and disposition constraints | +| Decision/evidence reference | Decisions, Committee, Audit | Publish a protected exact outcome or verifiable reference without transferring unrelated case content | +| Data product publication | Datasources, Dataflow, Reporting | Publish immutable governed materializations with schema, quality, freshness, lineage, and use constraints | +| Configuration package | Core, Policy, Views, Workflow, Forms, Templates | Exchange signed definitions; destination assesses compatibility, maps values, derives locally, and never imports secrets | +| Search discovery | Search and domain providers | Return permission-filtered metadata or a handoff link; never expose raw remote indexes as local authority | + +## State Machine + +```text +draft -> authorized -> queued -> transmitted -> received + | | + v v + outcome_unknown rejected + | +received -> validating -> accepted -> applied -> acknowledged + | | | + v v v + rejected accepted_ reconciled + pending +``` + +Acceptance means the destination durably owns the received intent. It does not +mean the requested domain effect completed. Receipts distinguish transport, +validation, acceptance, application, and human acknowledgement. + +## Conflict And Autonomy + +- Incoming native objects become local references, mirrors, or newly owned + objects according to the profile. They do not overwrite local authority by + ID coincidence. +- Local mappings are effective-dated and auditable. +- Corrections create a linked revision. They do not erase what the destination + previously observed. +- Revocation is a request and evidence event; the destination applies its own + legal and retention rules. +- Configuration imports use assessment and derivation. A remote package cannot + weaken local policy or install code implicitly. +- A disconnected partner remains a visible pending/failed state; work can be + rerouted through an approved alternative channel. + +## Security And Privacy + +- Use mTLS for paired online transports and signed envelopes for end-to-end + origin evidence. +- Encrypt payload objects for the destination, with key rotation and outcome- + unknown recovery; transport encryption alone is insufficient for queued + bundles. +- Do not put bearer credentials, local permission scopes, or reusable secrets + in an exchange. +- Rate-limit and size-bound discovery and transfer; quarantine unknown schemas + and active content. +- Evaluate current local authorization at every effect even when the envelope + describes historical authority. +- Log metadata separately from protected content so operators can reconcile + without broad content access. +- Subject access, correction, restriction, legal hold, and deletion requests + become federated workflows with local decisions and receipts, not remote + direct database operations. + +## First Reference Proof + +1. Pair two disposable installations with independent tenants, keys, and + policies. +2. Exchange signed descriptors and approve only the Postbox delivery profile. +3. Deliver one Campaign message to a remote function-bound Postbox. +4. Prove replay safety, rejection, timeout/outcome unknown, retry, + acknowledgement, correction, key rotation, and revoked trust. +5. Export the complete evidence bundle and restore both sides from backup. +6. Add configuration-package exchange only after the delivery proof passes. + +The result is a provider-neutral federation contract. A future dedicated +module becomes appropriate only when pairing, trust administration, exchange +queues, and evidence have a lifecycle independent of Connectors and the first +domain owner. diff --git a/docs/INSTITUTIONAL_DIGITAL_TWIN.md b/docs/INSTITUTIONAL_DIGITAL_TWIN.md new file mode 100644 index 0000000..3a20dcd --- /dev/null +++ b/docs/INSTITUTIONAL_DIGITAL_TWIN.md @@ -0,0 +1,149 @@ +# Institutional Digital Twin + +## Definition + +The institutional digital twin is a governed, time-aware projection of how an +institution is constituted and operates. It connects structure, authority, +services, work, information, technology, obligations, controls, evidence, and +outcomes without becoming a second source of truth. + +The twin is not one editable graph database and not an employee-surveillance +system. Domain modules and external systems keep ownership. The twin stores or +materializes exact references, declared relationships, provenance, confidence, +and projection versions. Changes flow through owner actions. + +Implementation is tracked in +[GovOPlaN #43](https://git.add-ideas.de/GovOPlaN/govoplan/issues/43). + +## Questions It Should Answer + +- Which unit and function is responsible for a service, decision, record, + system, dataset, control, or risk at a given valid and recorded time? +- Which mandates and policies permit or constrain an action? +- Which processes, providers, staff capacities, data sources, and records are + required to deliver a service? +- What is affected if a system, provider, organizational unit, role, package, + or legal rule changes? +- Where are responsibilities missing, conflicting, expired, or concentrated? +- Which controls are evidenced, stale, failed, or dependent on an unverified + assertion? +- How do actual process traces differ from defined workflows? +- Which public outcomes can be explained from protected internal evidence? + +## Projection Planes + +| Plane | Meaning | +| --- | --- | +| Current | Valid now, reconstructed from owner projections and current provider state | +| Historical | Valid at and recorded by selected instants, with present-day security enforced | +| Planned | Approved or proposed future structures, services, policies, projects, and package changes | +| Observed | Events, process traces, service measures, incidents, effects, and evidence actually recorded | +| Scenario | Non-authoritative simulation of a proposed change and its estimated consequences | + +The UI must label these planes unambiguously. Scenario output never becomes an +institutional fact until an authorized owner action accepts it. + +## Canonical Graph + +Nodes are stable institutional references, including institution, tenant, +unit, function, assignment, mandate, jurisdiction, service, case, party, task, +workflow, approval, decision, record, file, message, appointment, dataset, +report, provider, system, control, risk, project, asset, and configuration +package. + +Edges have: + +- owner and source authority; +- relationship type and direction; +- valid-from/valid-to and recorded/superseded times; +- exact source revision and evidence digest; +- institution/tenant boundary; +- purpose and visibility classification; +- confidence and derivation method for inferred relationships; +- correction and replacement references. + +Inferred edges are never displayed as owner assertions. They remain +explainable analytical products with source lineage. + +## Ownership And Implementation + +- Core owns neutral institutional references, temporal context, provider + registration, and graph projection contracts. +- Domain modules publish bounded nodes and edges through provider interfaces. +- Search indexes discoverable identities and links. +- Reporting materializes governed analytical projections. +- Dataflow computes derived relationships, quality checks, and scenarios. +- Policy evaluates visibility, purpose, retention, and allowed scenario/action + transitions. +- Audit supplies observed events and evidence references. +- Projects supplies planned change and benefit relationships. +- Views renders role- and task-focused twin perspectives. +- Workflow Engine coordinates accepted changes but does not edit owner tables. + +No new digital-twin module is required for the first slice. A dedicated owner +is justified later if persisted scenario models, graph revisions, and +cross-domain projection lifecycle become independent product objects. + +## Beyond The Current Platform + +### Continuous assurance + +Controls become versioned assertions with evidence requirements, evaluation +frequency, responsible function, exception workflow, and freshness. Dataflow +and provider checks evaluate them continuously; Policy decides whether a stale +or failed control advises, requires review, or blocks an effect. + +### Process mining and conformance + +Governed event histories can derive actual paths, wait times, rework, and +exceptions. Comparison to Workflow definitions should improve procedures, not +rank individuals. Access to personal or small-cohort detail is purpose-limited +and separately governed. + +### Change-impact simulation + +A proposed organizational, provider, policy, or package change can be assessed +against dependencies, mandates, open work, records, controls, capacity, and +recovery plans before activation. Results identify uncertainty rather than +inventing precision. + +### Federated institutional models + +Installations can exchange signed public or partner-specific subsets of their +service, mandate, provider, and evidence graph. Every side maps the references +locally and retains autonomy. Federation does not create one supranational +master graph. + +### Accountable assistance + +Assistance may summarize context, identify missing evidence, draft a decision +or workflow, propose mappings, and explain policy. Every output records model, +inputs, constraints, uncertainty, human review, and accepted edits. Assistance +does not become the acting authority. + +### Public evidence chains + +Transparency packages can publish a minimized chain from rule and aggregate +facts to decision and observed outcome, with digests proving relation to +protected evidence. Public verification does not require disclosure of the +underlying personal data. + +## Guardrails + +- Do not infer competence, misconduct, intent, or personal performance from + graph proximity or incomplete events. +- Do not centralize protected content merely to make graph queries easier. +- Do not use historical authorization to expose data now prohibited. +- Do not let a scenario engine write domain state directly. +- Do not hide source authority, freshness, uncertainty, or missing evidence. +- Do not retain analytical detail longer than the declared purpose requires. + +## Delivery Slices + +1. Publish exact institutional reference/edge providers for the service-to- + decision and monthly-data journeys. +2. Build a current/historical dependency explorer with source and access + explanations. +3. Add planned Project/package changes and bounded impact reports. +4. Add control evidence/freshness and process conformance for one journey. +5. Prove a minimized federated projection and a public evidence package. diff --git a/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md b/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md index 534a642..3e22ed4 100644 --- a/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md +++ b/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md @@ -9,13 +9,17 @@ concepts prepared outside the repositories: - `software_big_picture.md` The source concepts describe GovOPlaN as an operational governance platform for -public institutions. This document merges that direction with the implemented -platform state as of 2026-08-01. It is the canonical repository version of the -direction. Gitea issues remain the source of truth for delivery state. +public institutions. This document is the canonical repository version of that +durable architectural direction. Its implementation table records the accepted +2026-08-01 baseline; it is not a rolling status report. Current reconciliation +lives in [Strategy Status](STRATEGY_STATUS.md), and Gitea issues remain the +source of truth for delivery state. Read this together with: - [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) +- [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) +- [Strategy Status](STRATEGY_STATUS.md) - [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md) - [Module Contracts and Install Boundaries](MODULE_CONTRACTS_AND_INSTALLS.md) - [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md) @@ -70,7 +74,12 @@ compositions described here are now implemented. Subsequent work is **product depth and stronger maturity evidence**, not another runtime rewrite or an unimplemented architecture boundary. -## Implementation status (2026-08-01) +## Accepted implementation baseline (2026-08-01) + +This section is retained as the dated baseline against which the architecture +decision was accepted. Later implementation must be reconciled in +`STRATEGY_STATUS.md` rather than editing individual rows here into a competing +status report. The architecture contract is implemented as a bounded, executable vertical slice. The portfolio declarations and provider governance gates apply to the @@ -79,7 +88,7 @@ were proven now have independent persistent owners: | Area | Implemented state | Remaining rollout | | --- | --- | --- | -| Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for all 62 source manifests. Focused and release checks enforce `--require-architecture`; a new module cannot enter the workspace without truthful declaration and repository-local evidence. | +| Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for the source manifests in the 2026-08-01 snapshot. Focused and release checks enforce `--require-architecture`; current portfolio counts belong in `STRATEGY_STATUS.md`. | | External providers | Core validates provider objects/field groups, operations, integration maturity, source authority, bounded reads, freshness/health, idempotency, conflicts, outcome-unknown handling, evidence, correction, reconciliation, outage, classification, purpose, retention, and secret handling. Addresses/CardDAV, Files remote storage, Mail SMTP/IMAP, Calendar CalDAV/ICS/Graph/EWS, and Connectors tabular/sanctions providers declare the contract and tenant-bounded secret-free runtime state. | Registry validation rejects any declared external provider without a sanitized state provider. Future adapters must cross the same gate before activation. | | Institutional context | Core provides versioned temporal, actor/representation, institution/unit/function/task/mandate/jurisdiction/service/case/party/work-item/workflow/approval/decision/record, legal-basis, evidence, information-governance, external-source, presentation, and geographic references. Events, automation actions, audit records, and the transactional Audit outbox preserve the envelope. | Owning modules must progressively require the relevant subset for consequential operations. | | Semantic provider contracts | Provider-neutral DTOs and protocols cover Mandate resolution, versioned Service definitions, procedure Parties/representation, and formal Decisions. `govoplan-mandates`, `govoplan-services`, `govoplan-parties`, and `govoplan-decisions` now persist immutable revisions behind those contracts with tenant isolation, bounded reads, replay safety, OCC, migrations, uninstall guards, permissions, APIs, capability documentation, and recovery documentation. | The owners are deliberately headless. Procedure-specific UI remains with consuming modules. | @@ -391,7 +400,9 @@ submodule, configuration fragment, package, or profile. - This reconciliation is canonical in the meta repository and mirrored to the Gitea wiki. -- All 62 source manifests carry validated evidence-based architecture metadata. +- All source manifests in the accepted 2026-08-01 baseline carried validated + evidence-based architecture metadata; current counts belong in + `STRATEGY_STATUS.md`. - External-reference, action/effect, operational-health, ownership, policy, audit, and documentation primitives compose into one enforced provider declaration and sanitized runtime-state contract. @@ -446,7 +457,7 @@ submodule, configuration fragment, package, or profile. recovery, accessibility, privacy, security, and operator evidence. This is a maturity gate, not missing architecture implementation. -## What remains after the executable architecture slice +## What remains within the accepted 2026-08-01 architecture slice The remaining work is not another Core or cross-module architecture rewrite. It falls into two explicitly different categories, neither of which can be @@ -487,12 +498,38 @@ persistence, migrations, recovery/disable semantics, documentation and focused tests. Their remaining tickets concern concrete providers, deeper adapters and target evidence, not an unresolved institutional architecture boundary. -Everything else described as architecture in this document now has a +Everything else described in the accepted baseline of this document now has a repository owner, versioned contract, bounded implementation, migration and recovery boundary where state exists, documentation, and executable evidence. Further work in those modules is product breadth, UX depth, provider adoption, and evidence renewal. +## Strategic extensions accepted after the baseline + +The completed baseline does not imply that institutional product architecture +can no longer grow. The 2026-08-05 strategic review accepted four extensions +that consume the existing contracts without reopening the kernel or moving +domain ownership into Core: + +- [Product Experience and Module Boundaries](PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md) + separates technical package topology from stable task/object/product + surfaces; implementation is tracked in Core #283. +- [Federated GovOPlaN Architecture](FEDERATED_GOVOPLAN_ARCHITECTURE.md) + defines governed exchange between autonomous installations; implementation + is tracked in GovOPlaN #41. +- [Assisted and Non-Digital Channels](ASSISTED_AND_NON_DIGITAL_CHANNELS.md) + makes channel inclusion part of the service-to-decision journey; the first + reference proof is tracked in GovOPlaN #42. +- [Institutional Digital Twin](INSTITUTIONAL_DIGITAL_TWIN.md) defines a + time-aware, policy-filtered projection over owner data; implementation is + tracked in GovOPlaN #43. + +The eAkte depth required by those journeys is owned by Records and specified in +`govoplan-records/docs/EAKTE_ARCHITECTURE.md`, tracked in Records #1. These are +new product-depth programs with bounded contracts and acceptance journeys, not +evidence that the original institutional semantics or module architecture +failed. + ## Delivery tracking The completed cross-repository architecture epic is diff --git a/docs/KUBERNETES_TEST_LAB.md b/docs/KUBERNETES_TEST_LAB.md new file mode 100644 index 0000000..5a471e3 --- /dev/null +++ b/docs/KUBERNETES_TEST_LAB.md @@ -0,0 +1,340 @@ +# Kubernetes VM Test Lab + +`tools/lab/govoplan-lab.py` creates and operates an amd64 Ubuntu/K3s test +environment on local or SSH-accessible libvirt hypervisors. It provides the +commands requested for the complete VM lifecycle: + +| Command | Effect | +| --- | --- | +| `doctor` | Validate the strict inventory and, with `--online`, every hypervisor. | +| `create --apply` | Download checksum-pinned cloud images, create VM overlays and boot the declared VMs. | +| `deploy --apply` | Verify the signed GovOPlaN release, deploy shared state, install pinned K3s and apply GovOPlaN. | +| `update --apply` | Pull newly pinned state images, update K3s serially and roll the selected GovOPlaN release. | +| `status` | Show libvirt VM state, Kubernetes nodes and GovOPlaN pods. | +| `pause --apply` | Gracefully shut down workers, control planes and shared state while retaining disks. | +| `resume --apply` | Start the retained environment in dependency order and wait for readiness. | +| `verify` | Collect sanitized live-cluster evidence and optionally perform the API-pod-loss drill. | +| `destroy --apply --confirm ` | Delete only the lab-owned domains and overlays; local evidence is retained by default. | + +Every mutating command is a dry run unless `--apply` is present. Destruction +also requires the exact lab name. Generated credentials, CA keys, manifests and +evidence are written below the configured `state_directory` with owner-only +permissions. Keep that directory outside the repository and include it in the +workstation backup policy. Existing domains are reused or removed only when +their GovOPlaN ownership description and both expected lab disk paths match. + +## What The Lab Proves + +The supplied inventories describe two different assurance levels: + +- `tools/lab/govoplan-lab.example.toml` creates four VMs on one libvirt host. + It is suitable for development, deployment rehearsal, migration testing, + application-pod replacement and recovery-tool exercises. It cannot close + GovOPlaN #27 because one physical host remains one failure domain. +- `tools/lab/govoplan-lab.acceptance.example.toml` places the two workers on + different hypervisors and puts the control and state VMs on a third. It can + produce the bounded stateless application-tier evidence required by #27 when + the declared hypervisors are genuinely independent physical failure domains. + +Both examples use one control-plane VM and one state VM. This keeps the bounded +#27 target economical, but it does not prove control-plane or state-service +high availability. For control-plane failover, declare exactly three control +nodes on independent hosts. PostgreSQL, Redis and object-storage failover must +be tested against independently operated HA services; the lab's single state +VM is intentionally a replaceable integration fixture. + +Approximate minimum capacity for the four-VM profile is 10 vCPUs, 16 GiB RAM +and 192 GiB of thin-provisioned disk. A six-VM profile with three controls needs +additional capacity. Do not overcommit memory on an acceptance target. + +## 1. Prepare The Hypervisors + +On each Ubuntu/Debian libvirt host: + +```bash +sudo apt-get update +sudo apt-get install -y \ + qemu-kvm libvirt-daemon-system libvirt-clients virtinst cloud-image-utils curl +sudo systemctl enable --now libvirtd +``` + +Use a dedicated lab-administration account. Remote hypervisors are managed over +SSH and the lifecycle invokes `sudo -n` there, so that account needs bounded +non-interactive permission for libvirt, image and cloud-init operations. +`NOPASSWD: ALL` is acceptable only on isolated lab hypervisors. + +On a local hypervisor, put the workstation account in the `libvirt` group and +point `vm_image_directory` at a directory writable by that account and +traversable by `libvirt-qemu`. The lifecycle connects explicitly to +`qemu:///system` and does not require passwordless local sudo. Log out and back +in after a new group assignment before running `doctor --online`. Create the +configured image directory before running the doctor; it deliberately rejects +a missing or non-writable storage root instead of silently falling back to a +different filesystem. + +Create a dedicated SSH key on the management workstation: + +```bash +ssh-keygen -t ed25519 -f "$HOME/.ssh/govoplan-lab" \ + -C "GovOPlaN Kubernetes lab" +``` + +Install its public key for every remote hypervisor account. The same public key +is injected into the VMs. The lifecycle keeps its own `ssh_known_hosts` file, +uses `accept-new` for first contact, and rejects changed host keys until a +lab-owned VM is deliberately recreated. + +### Network contract + +The configured `bridge` must exist on every selected hypervisor. All VM +addresses are static. Reserve them outside DHCP allocation and ensure that the +management workstation can route directly to every VM address; the lifecycle +does not tunnel VM traffic through the hypervisor SSH connection. + +Permit only these flows inside the lab network: + +| Port | Source and destination | Purpose | +| --- | --- | --- | +| TCP 22 | management workstation to every VM/hypervisor | Provisioning and evidence collection | +| TCP 6443 | all K3s nodes and management path to controls | Kubernetes API | +| UDP 8472 | K3s node to K3s node | Default Flannel VXLAN; never expose publicly | +| TCP 10250 | K3s node to K3s node | Kubelet metrics and API | +| TCP 2379-2380 | control to control, only with three controls | Embedded etcd | +| TCP 80/443 | test clients to K3s nodes | Traefik/ServiceLB ingress | +| TCP 5432/6379/9443 | K3s nodes to the state VM | PostgreSQL, Redis and TLS-protected Garage S3 | +| TCP 3025/3143 | approved test clients/workers to the state VM | GreenMail SMTP/IMAP test endpoints | + +The official +[K3s networking requirements](https://docs.k3s.io/installation/requirements#networking) +remain authoritative. Restrict state ports to the lab network even though the +generated integration stack binds them on the state VM. + +## 2. Create The Inventory + +Start with the one-host rehearsal: + +```bash +install -d -m 0700 "$HOME/.config/govoplan/labs" +cp tools/lab/govoplan-lab.example.toml \ + "$HOME/.config/govoplan/labs/development.toml" +chmod 0600 "$HOME/.config/govoplan/labs/development.toml" +``` + +Edit at least the bridge, network, static addresses and SSH key paths. For a +multi-host run, copy the acceptance example and replace every example hostname, +failure-domain declaration and network value. Strict parsing rejects unknown +keys, mutable HTTP inputs, malformed checksums, duplicate addresses/MACs and an +acceptance inventory that collapses workers onto one declared hypervisor or +failure domain. + +Cloud image, K3s binary, K3s installer and GovOPlaN release inputs are URL plus +SHA-256 pairs. Updating means changing those reviewed pins and then running the +`update` command; the tool deliberately does not follow `latest` aliases. + +The one-host example uses the dedicated `govoplan-lab` NAT network. Its DHCP +pool ends at `192.168.123.99`; the static lab addresses start at +`192.168.123.201`. Define and start it once on the local hypervisor: + +```bash +virsh --connect qemu:///system net-define \ + tools/lab/libvirt/govoplan-lab-network.xml +virsh --connect qemu:///system net-autostart govoplan-lab +virsh --connect qemu:///system net-start govoplan-lab +``` + +Re-running those commands is unnecessary when `virsh net-info govoplan-lab` +already reports an active, persistent network. The lab destroy command leaves +this reusable network in place. + +## 3. Validate And Create The VMs + +```bash +LAB="$HOME/.config/govoplan/labs/development.toml" +PYTHON="/mnt/DATA/git/govoplan/.venv/bin/python" + +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" doctor --online +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" create +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" create --apply +``` + +The preview is safe to run repeatedly. Creation reuses a domain whose exact +lab-owned name already exists and otherwise creates a thin qcow2 overlay under +`vm_image_directory//`. + +## 4. Deploy GovOPlaN + +If `git.add-ideas.de` requires authentication for release images, export a +read-only package/container-registry identity for this shell. A Gitea package +token can be used as the password: + +```bash +export GOVOPLAN_LAB_REGISTRY_USERNAME='package-reader' +read -r -s GOVOPLAN_LAB_REGISTRY_PASSWORD +export GOVOPLAN_LAB_REGISTRY_PASSWORD +``` + +Then preview and apply: + +```bash +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" deploy +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" deploy --apply +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" status +unset GOVOPLAN_LAB_REGISTRY_PASSWORD +``` + +Deployment verifies the downloaded release manifest and keyring by pinned +digest and by the existing GovOPlaN signature policy. It deploys PostgreSQL, +Redis, single-node Garage and GreenMail on the state VM. The API, WebUI, workers +and scheduler run in K3s from digest-pinned release images. A private lab CA +protects both ingress and S3; backend pods receive only the CA Secret and keep +TLS verification enabled. + +The final output identifies two local files below `state_directory`: + +- `hosts` maps the public GovOPlaN and S3 test names to their VM addresses; +- `pki/ca.crt` is the private lab CA certificate. + +Add the host mappings to the test client's resolver and trust the CA only on +devices used for this lab. On Debian/Ubuntu: + +```bash +STATE="$HOME/.local/share/govoplan/labs/govoplan-k8s-lab" +cat "$STATE/hosts" +sudo install -m 0644 "$STATE/pki/ca.crt" \ + /usr/local/share/ca-certificates/govoplan-k8s-lab.crt +sudo update-ca-certificates +``` + +Review mappings before adding them to `/etc/hosts`; the lifecycle does not edit +the workstation's trust or resolver configuration. + +### Enroll the first administrator + +The production runtime does not create a default password. Issue one expiring, +single-use first-administrator credential inside an API pod and copy its +owner-only artifact out immediately: + +```bash +KUBECTL="$STATE/bin/kubectl" +POD="$($KUBECTL -n govoplan get pods \ + -l app.kubernetes.io/component=api \ + -o jsonpath='{.items[0].metadata.name}')" +ARTIFACT="$STATE/first-admin-enrollment.json" +umask 077 + +$KUBECTL -n govoplan exec "$POD" -- \ + python -m govoplan_core.commands.first_admin issue \ + --reason 'initial Kubernetes lab enrollment' \ + --output /tmp/first-admin-enrollment.json +$KUBECTL -n govoplan exec "$POD" -- \ + cat /tmp/first-admin-enrollment.json > "$ARTIFACT" +$KUBECTL -n govoplan exec "$POD" -- \ + rm -f /tmp/first-admin-enrollment.json +chmod 0600 "$ARTIFACT" +``` + +Submit the token from that artifact once to +`/api/v1/bootstrap/first-admin` with the administrator email, display name, +password, tenant slug and tenant name. The password must contain at least 12 +characters. The lab command performs that exchange without placing either the +token or password in process arguments, rejects redirects, and removes the +artifact only after HTTP 201: + +```bash +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" enroll-admin \ + --email 'owner@example.org' \ + --display-name 'System Owner' \ + --tenant-slug default \ + --tenant-name 'Default Tenant' +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" enroll-admin \ + --email 'owner@example.org' \ + --display-name 'System Owner' \ + --tenant-slug default \ + --tenant-name 'Default Tenant' \ + --apply +``` + +The public lab hostname must already resolve on the management workstation; +the command verifies TLS through the generated private CA directly. + +## 5. Collect #27 Evidence + +Create a short-lived API key authorized to read the Ops status endpoint. In the +current Access administration UI, open **Tenant API keys** and select only +**View tenant settings** (`admin:settings:read`); the Ops endpoint explicitly +accepts that compatibility scope. A dedicated operator credential may instead +use `ops:operations:read`. Then run: + +```bash +export GOVOPLAN_OPS_API_KEY='short-lived-value' +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" verify \ + --exercise-api-pod-loss +unset GOVOPLAN_OPS_API_KEY +``` + +The verifier requires ready API and WebUI pods across at least two Kubernetes +nodes, all Deployments available, consistent runtime composition, queue +coverage and a valid database-connection budget. During the optional drill it +deletes one ready API pod, probes public readiness and waits for replacement. +It writes sanitized output to +`state_directory/evidence/kubernetes-multi-host.json` and never stores the API +key. A rehearsal inventory prints an explicit warning that its result is not +independent-failure-domain evidence. + +Retain these private artifacts together for review: + +1. `inventory.json` and the reviewed inventory TOML; +2. the adopted release manifest/keyring and installation receipt; +3. `kubernetes.json`; +4. the Kubernetes verifier output; +5. private cluster logs for the approved drill window; +6. the operator's out-of-band evidence that the worker hypervisors are + independent physical hosts or availability zones. + +GovOPlaN #37 additionally requires independent assessment and production +approval keys. Running its evidence jobs in containers is supported, but a +container does not create an independent authority. Follow +`TARGET_MATURITY_EVIDENCE_RUNBOOK.md` after the #27 drill passes. + +## 6. Update, Pause, Resume And Remove + +After reviewing and changing pinned image/K3s/release values in the inventory: + +```bash +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" update +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" update --apply +``` + +Workers are cordoned, drained, updated and uncordoned one at a time. K3s +controls are reconciled serially. The release-specific migration Job remains +subject to GovOPlaN's signed backup-evidence gate. The lab update command is not +a substitute for creating recovery evidence before a destructive state-schema +change. + +To stop compute use without deleting disks: + +```bash +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" pause --apply +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" resume --apply +``` + +To remove VM resources while preserving local evidence: + +```bash +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" destroy +"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" destroy \ + --apply --confirm govoplan-k8s-lab +``` + +Add `--purge-local-state` only after evidence and recovery material have been +retained elsewhere. That option deletes the generated local CA, secrets, +manifests and evidence as well as the VMs. + +## Acceptance Boundary + +This tool supplies reproducible infrastructure and executes the bounded +stateless-node drill. It does not certify the truth of operator-entered failure +domains, provide HA PostgreSQL/Redis/Garage, create production backup evidence, +or approve its own results. Those boundaries are deliberate: #27 can close +after a passing run on independently controlled hosts; broader production +maturity remains governed by #35, #37 and the target evidence runbook. diff --git a/docs/PLATFORM_CORE_IDEAS.md b/docs/PLATFORM_CORE_IDEAS.md new file mode 100644 index 0000000..b51ef2c --- /dev/null +++ b/docs/PLATFORM_CORE_IDEAS.md @@ -0,0 +1,173 @@ +# GovOPlaN Platform Core Ideas + +## Purpose + +GovOPlaN is an institutional governance and operations layer. Its central +promise is: + +> Model the institution, orchestrate its work, connect its systems, and +> preserve why and under whose authority it acted. + +The platform should let people complete a real task without understanding its +repository or module graph. It should let institutions retain control over +their data, procedures, providers, and deployment while still sharing +interoperable definitions and evidence. + +This document is the stable summary of the ideas that every product package, +module, interface, and integration must preserve. Current implementation state +lives in [Strategy Status](STRATEGY_STATUS.md). + +## Ten Core Ideas + +### 1. Institutional context before application context + +Work happens for a tenant, institution, organizational unit, function, +mandate, jurisdiction, service, case, and represented party. The real actor +and represented capacity remain distinct. Application permissions alone do not +prove institutional competence. + +### 2. Governance is executable + +Policy is not explanatory prose around an operation. Consequential actions +must expose applicable rules, authority, purpose, expected effects, review +requirements, recovery behavior, and evidence. Inheritance may tighten a rule +but must not silently loosen an upstream constraint. + +### 3. Time has two independent meanings + +Valid time answers when a fact applied. Recorded time answers what the system +knew at a point in history. Historical browsing changes the business-data +projection, never the current authorization context. Corrections and +supersession remain visible rather than rewriting history. + +### 4. One context, many owners + +Cases, tasks, decisions, records, messages, files, appointments, reports, and +external objects remain owned by their domain modules or source systems. Stable +references create one navigable context without a universal copied master +record or cross-module table access. + +### 5. Native and connected operation are peers + +For every integration, GovOPlaN states whether it is authoritative, mirrors an +external source, synchronizes governed fields, adds a governance overlay, or +keeps a link only. An external system can be used today and replaced later +without losing provenance or institutional control. + +### 6. Human work is a first-class system object + +An intake becomes owned, reviewable work. A person can see the current context, +next responsible action, reason, deadline, consequence, and completion +evidence. Workflow Engine coordinates machine and human transitions; focused +views guide people through the relevant platform surfaces. + +### 7. Views reduce complexity without changing authority + +The interface is a task- and role-sensitive projection of installed +capabilities. Views, dashboards, search, documentation, and workflow-guided +surfaces may hide irrelevant functions, but they never grant access. Users can +escape a focused mode when policy permits and can always understand why +something is unavailable. + +### 8. Evidence and recovery are part of the operation + +Intent, exact input versions, approvals, external effects, receipts, +outcome-unknown states, reconciliation, corrections, retention, and recovery +belong to one evidence chain. A retry must be idempotent; rollback claims must +distinguish reversible local state from effects already observed elsewhere. + +### 9. Inclusion is multi-channel, not portal-only + +Public portal, postbox, mail, telephone, paper, in-person assistance, APIs, and +external systems are channels around the same governed work. Assisted entry +records who entered information, for whom, from which source, with which +attestation, and how the affected person receives a usable receipt and +correction path. + +### 10. Successful configurations are portable products + +Modules are ingredients. A usable product is a signed configuration package +with terminology, forms, policies, workflows, views, reports, provider +profiles, documentation, migration rules, and evidence. Institutions derive +local packages without forking code or weakening inherited constraints. + +## Platform Planes + +The planes below are ownership lenses, not navigation groups or mandatory +deployment tiers. + +| Plane | Responsibility | +| --- | --- | +| Experience | Shell, views, dashboard, search, help, accessibility, and task-focused composition | +| Participation and channels | Portal, postbox, mail, campaigns, calendar, scheduling, consultation, and assisted channels | +| Human work and procedure | Services, forms/runtime, cases, tasks, approvals, workflow execution, and domain procedures | +| Content, records, and evidence | Files, templates, DMS, eAkte/records, audit, reporting, transparency, and publication | +| Institutional governance | Identity, access, tenancy, organizations, functions, mandates, policy, trust, and formal decisions | +| Data and integration | Connectors, datasources, dataflow, search, external references, provider health, and reconciliation | +| Runtime and assurance | Module composition, operations, deployment, recovery, security evidence, and signed packages | + +## Canonical Distinctions + +The platform must not collapse these pairs: + +- identity vs account vs represented capacity; +- role/permission vs function/mandate/competence; +- valid time vs recorded time; +- purpose for use vs general technical access; +- document content vs managed file bytes vs institutional record; +- task vs workflow definition vs workflow instance; +- approval vs formal decision; +- message intent vs transport delivery vs recipient acknowledgement; +- source authority vs connector maturity; +- current state vs historical evidence; +- correction/compensation vs erasure of an observed effect; +- a module boundary vs a user-visible product boundary. + +## Product Experience Rule + +The normal user interface speaks in services, work, records, messages, +meetings, decisions, and outcomes. Module names, provider IDs, capability names, +package coordinates, and schema details are technical provenance. They are +visible to administrators and in expandable diagnostics, but they are not the +primary information architecture for ordinary work. + +## Maturity Rule + +A repository, route, model, or unit test does not make a capability complete. +Claims advance only with evidence appropriate to the claim: + +1. `scaffold`: boundary and documentation exist; +2. `vertical_slice`: useful behavior has focused tests; +3. `reference_ready`: an end-to-end reference journey passed target, + accessibility, privacy, security, operations, and recovery evidence; +4. `supported`: upgrades, interoperability, support procedures, and release + guarantees are defined; +5. `lts`: compatibility and maintenance windows are contractual. + +## Deliberate Non-Goals + +GovOPlaN does not aim to: + +- replace every specialist system, ERP, DMS, groupware, or data tool; +- make one database authoritative for every connected fact; +- expose every installed capability to every person; +- infer authority from organizational membership alone; +- make historical browsing weaken current security; +- treat AI output as an unaccountable institutional decision; +- create a repository for every noun in the information model; +- claim production maturity from local development evidence. + +## Decision Test + +A proposed feature fits the platform when it improves at least one real +institutional journey and can answer: + +1. Who owns the object and source of truth? +2. In which institutional and temporal context does it apply? +3. For which declared purpose may it be used? +4. Which policy and authority permit the action? +5. What effect, evidence, retention, and recovery behavior result? +6. How can it operate with an external owner without losing autonomy? +7. How will a person discover and complete it without learning the module + graph? + diff --git a/docs/PRODUCTION_TARGET_HANDOFF.md b/docs/PRODUCTION_TARGET_HANDOFF.md index 2461084..587b3bd 100644 --- a/docs/PRODUCTION_TARGET_HANDOFF.md +++ b/docs/PRODUCTION_TARGET_HANDOFF.md @@ -49,6 +49,13 @@ firewall ports. Those inputs are sufficient to provision a k3s target. They are not sufficient to claim control-plane HA unless three control-plane failure domains are present. +The repository now supplies the strict libvirt/K3s lifecycle and example +inventories for this handoff in +[`KUBERNETES_TEST_LAB.md`](KUBERNETES_TEST_LAB.md). Its `acceptance` mode +rejects a declared topology unless the workers and shared-state fixture occupy +different hypervisor and failure-domain identifiers. Reviewers must still +verify that those identifiers correspond to genuinely independent hosts. + ### Separate deployment and evidence authorities The deployment identity may create and update the namespace, Secret, @@ -74,6 +81,7 @@ python tools/deployment/govoplan-deploy.py render-kubernetes \ --namespace govoplan \ --secret-name govoplan-runtime \ --tls-secret-name govoplan-tls \ + --s3-ca-secret-name govoplan-s3-ca \ --ingress-class-name nginx \ --output /srv/govoplan//kubernetes.json diff --git a/docs/PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md b/docs/PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md new file mode 100644 index 0000000..d170c5e --- /dev/null +++ b/docs/PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md @@ -0,0 +1,147 @@ +# Product Experience and Module Boundaries + +## Problem + +GovOPlaN's runtime modularity is a strength, but the implementation structure +is exposed too directly in the product. Ordinary users encounter module names, +one top-level route per module, one navigation item per repository, package and +provider identifiers, and errors framed as missing modules. This makes the +system look like a toolbox of adjacent applications instead of one operating +environment for institutional work. + +The correction is not a monolithic frontend and not hidden provenance. It is a +separate product information architecture assembled from typed module +contributions. + +Implementation is tracked in +[Core #283](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/283). + +## Current Exposure Inventory + +| Surface | Direct exposure | Appropriate audience | Product-facing alternative | +| --- | --- | --- | --- | +| Side rail | One icon and route for many installed modules | Administrators and power users | Work areas, services, inboxes, records, communication, data and assurance | +| Route paths | Technical owners such as `/dataflow`, `/forms`, or `/postbox` | Deep links and diagnostics | Stable product aliases and journey routes that resolve to owner surfaces | +| Dashboard | Installed module count and module-owned widget library | Operators | Outcome, obligation, work, exception, and service widgets | +| Administration | Package names, database state, capabilities, providers | Module and system administrators | Guided product/package configuration with technical details on demand | +| Errors | "Module/capability not installed" | Diagnostics | Explain the unavailable outcome, responsible administrator, and enabling path | +| Documentation | Topics grouped primarily by module | Administrators | Task, role, service, and object documentation with module provenance secondary | +| Permissions | Module-namespaced scopes | Access administrators | Human-readable responsibility bundles; exact scopes remain inspectable | +| Search | Provider/module as a result facet | Advanced filtering | Object type, institution, time, purpose, case/service, and source authority | +| Workflow | Steps can expose target route/module details | Workflow designers | User-facing action and expected result; technical binding in definition details | +| Connector state | Provider IDs and source types | Integration owners | Named source, authority, freshness, health, last effect, and recovery state | + +## Boundary Decision + +Three layers remain distinct: + +1. **Technical module layer:** package ownership, dependencies, capabilities, + permissions, migrations, routes, and provider identifiers. +2. **Product composition layer:** work areas, object types, journeys, commands, + inboxes, configuration packages, and role-based defaults. +3. **Presentation projection:** active view, tenant policy, current task, + temporal context, language, accessibility preferences, and device layout. + +Modules own implementation and contribute typed product metadata. Core +assembles it. Views filters it. Policy constrains it. Access authorizes the +underlying actions. No consumer imports another optional module's UI directly. + +## Product Surface Contract + +Each WebUI module should be able to announce: + +- `product_areas`: stable areas to which a route, command, widget, or object + belongs; +- `object_types`: user-facing nouns, icons, search context, detail route, and + owner provenance; +- `work_item_sources`: open work, exceptions, deadlines, and responsible + capacity; +- `journey_actions`: launch, resume, review, correct, decide, publish, and + reconcile commands; +- `workspace_surfaces`: embeddable but owner-rendered list, detail, editor, and + status surfaces; +- `configuration_contributions`: guided settings with consequence and + prerequisite metadata; +- `help_contexts`: user/admin documentation for the product identity as well as + the technical owner; +- `technical_provenance`: module, interface version, capability, and provider + identifiers shown only in details and evidence. + +The contract references surfaces. It does not permit Core or a product package +to import their implementation. + +## Navigation Model + +The default shell should prioritize: + +1. global search and create/resume commands; +2. personal and function-bound work; +3. configured product areas; +4. pinned user destinations; +5. administration and technical module inspection when authorized. + +A module route remains a valid deep link. A product area may combine links and +owner-rendered surfaces from several modules. When a required contribution is +absent, the area explains the missing outcome rather than rendering a broken +placeholder. + +Views remain the projection mechanism. They may select product areas, routes, +sections, commands, widgets, and fields. A view must not grant a permission or +change data semantics. Policy can force, allow, or prohibit a surface at system, +tenant, group, or user scope. + +## Error And Provenance Language + +Normal errors answer: + +- what the person was trying to achieve; +- why it is unavailable or failed; +- whether data was saved or an external effect may have occurred; +- who can resolve it and where; +- the correlation/evidence reference. + +An expandable technical section may then identify the module, capability, +provider, request, and version. This keeps the product intelligible without +hiding operational truth. + +## Migration + +### Slice 1: inventory and aliases + +- classify every route, navigation item, widget, setting, search object, and + help context by product area and object type; +- add product aliases without removing existing deep links; +- flag raw module IDs in ordinary-user labels and errors. + +### Slice 2: work-first shell + +- provide a generic work/exception/deadline aggregation capability; +- make work areas and configured packages the default navigation; +- move the complete module catalogue to administration and an optional power- + user surface. + +### Slice 3: composite journeys + +- let product packages define journey launch/resume actions and default views; +- let Workflow Engine activate a view and focus an owner surface without + controlling authorization; +- expose provider provenance and technical bindings on demand. + +### Slice 4: enforceability + +- make product classification mandatory for user-visible manifest surfaces; +- reject duplicate product identities and missing owner routes in CI; +- add browser tests proving that reference users can complete a journey without + knowing module names. + +## Acceptance Criteria + +- An ordinary user can describe every primary navigation item as work or an + institutional object, not as a package. +- A product package can remove irrelevant navigation while retaining deep-link + and help integrity. +- Missing optional modules produce an actionable product explanation. +- Administrators can still inspect exact module, capability, provider, schema, + and evidence provenance. +- Module permutation tests prove that no product surface assumes an optional + owner is installed. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..f43e5de --- /dev/null +++ b/docs/README.md @@ -0,0 +1,75 @@ +# GovOPlaN Documentation Map + +This directory contains cross-repository product, architecture, release, and +operational documentation. The map below defines which document answers which +question. A document not listed as the current status source must not present +volatile repository, issue, release, or maturity counts as current facts. + +## Strategy + +| Question | Canonical source | +| --- | --- | +| What are the stable ideas and boundaries of the platform? | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) | +| What product outcomes should GovOPlaN pursue? | [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) | +| Which institutional concepts and owners form the target architecture? | [Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md) | +| Which end-to-end proofs should guide implementation? | [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md) | +| What is the reconciled state now? | [Strategy Status](STRATEGY_STATUS.md) | + +The dated [Strategic Review](STRATEGIC_REVIEW_2026-08-05.md) explains why the +current reset and sequencing were chosen. It is an assessment record, not a +second live status page. + +## Product Architecture + +| Topic | Canonical source | +| --- | --- | +| Product-facing experience and hiding technical module boundaries | [Product Experience and Module Boundaries](PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md) | +| Federation between autonomous installations | [Federated GovOPlaN Architecture](FEDERATED_GOVOPLAN_ARCHITECTURE.md) | +| Institutional digital twin and continuous assurance | [Institutional Digital Twin](INSTITUTIONAL_DIGITAL_TWIN.md) | +| Assisted and non-digital channels | [Assisted and Non-Digital Channels](ASSISTED_AND_NON_DIGITAL_CHANNELS.md) | +| Cross-module temporal, purpose, retention, and institutional-context adoption | `govoplan-core/docs/INFORMATION_GOVERNANCE_ADOPTION.md` | +| eAkte and digital-record ownership | `govoplan-records/docs/EAKTE_ARCHITECTURE.md` | +| Data source, definition, and transformation graph | [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md) | +| Focused task views | [Views Architecture](VIEWS_ARCHITECTURE.md) | +| Shared interface patterns | [Interface Pattern Language](INTERFACE_PATTERN_LANGUAGE.md) | + +## Runtime And Delivery + +- [Module Contracts and Installs](MODULE_CONTRACTS_AND_INSTALLS.md) +- [Platform Control Plane](PLATFORM_CONTROL_PLANE.md) +- [Installation and Deployment Architecture](INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md) +- [Kubernetes VM Test Lab](KUBERNETES_TEST_LAB.md) +- [Scaling and Multi-Host Deployment](SCALING_AND_MULTI_HOST_DEPLOYMENT.md) +- [Recovery and Rollback Guarantees](RECOVERY_AND_ROLLBACK_GUARANTEES.md) +- [Recovery Ledger Adoption](RECOVERY_LEDGER_ADOPTION.md) +- [Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md) + +## Evidence And Snapshots + +These documents are intentionally dated or pinned. They may remain useful even +after the product changes, but they do not override `STRATEGY_STATUS.md`. + +- [Capability and Infrastructure Fit Assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md), pinned to the 2026-07-22 Campaign composition +- [Strategic Review 2026-08-05](STRATEGIC_REVIEW_2026-08-05.md) +- [Backup and Restore Evidence](BACKUP_AND_RESTORE_EVIDENCE.md) +- [Production Target Handoff](PRODUCTION_TARGET_HANDOFF.md) +- [Target Maturity Evidence Runbook](TARGET_MATURITY_EVIDENCE_RUNBOOK.md) + +Machine-readable schemas and evidence files belong beside the document that +defines them. Generated inventories belong in `audit-reports/` and should not +be edited manually. + +## Maintenance Rules + +1. Gitea issues are the only live work-state source. +2. `STRATEGY_STATUS.md` is the only prose reconciliation of current portfolio + state. Refresh it from manifests, inventories, tests, and Gitea; do not copy + its counts into durable architecture pages. +3. Durable documents state decisions, invariants, ownership, and acceptance + gates. They link to status and issues for implementation depth. +4. Dated assessments retain their original composition and conclusion. Add a + snapshot notice rather than silently updating their claims. +5. Module-specific behavior and user/admin documentation remain in the owning + repository. Meta documentation defines cross-module outcomes and contracts. +6. A new strategy document must replace, narrow, or link an existing source; + it must not introduce a parallel roadmap. diff --git a/docs/REFERENCE_JOURNEY_PROGRAM.md b/docs/REFERENCE_JOURNEY_PROGRAM.md index 249e805..b3e1354 100644 --- a/docs/REFERENCE_JOURNEY_PROGRAM.md +++ b/docs/REFERENCE_JOURNEY_PROGRAM.md @@ -18,6 +18,27 @@ delivered as small, reviewable, green increments and is complete only when its user journey, failure behavior, documentation, and operator evidence work in a pinned composition. +## 2026 outcome reset + +Repository completion is not product completion. From 2026-08-05 onward, work +is accepted primarily through three maintained real-life journeys: + +1. **Governed communication:** select accountable recipients, prepare content + and attachments, approve, deliver through Mail and/or a function-bound + Postbox, reconcile uncertain outcomes, and file the evidence. +2. **Inclusive service-to-decision:** accept a request through a digital or + assisted channel, establish identity and purpose, guide the case through + human and automatic work, decide, notify, and file the resulting eAkte. +3. **Monthly data and sanctions:** acquire immutable source snapshots, validate + and reconcile them interactively, preserve decisions and lineage, produce + reports and files, and deliver the accepted result through Campaign. + +The staged program below remains the architectural build order. These journeys +are the acceptance lens across those stages. Every significant feature should +identify the journey it improves, or provide security, operability, recovery, +accessibility, or usability evidence that those journeys require. Work that +does neither stays in the backlog until a concrete consumer exists. + ## Why this sequence The sequence grows one connected product rather than advancing repositories in @@ -86,6 +107,12 @@ journey needs and supplies contracts shared by all five stages. execution. Database, broker, cache, and worker channels are constrained by deployment network policy and authenticated transport rather than treated as tenant connector profiles. +10. **Information governance.** Temporal browsing, purpose-aware access, + retention/legal-hold behavior, and institutional acting context are applied + to every owned object type. Historical reads use current authorization. + Module manifests state `contract_only`, `partial`, `enforced`, or + `not_applicable` adoption with evidence; supported maturity is blocked until + every applicable dimension is enforced. ## Documentation contract for every reference stage @@ -108,6 +135,9 @@ Every demonstrated journey provides: provenance, evidence, retention, and destructive actions. - **Acceptance view:** runnable examples, expected results, failure injection, and release gates. +- **Channel and records view:** assisted/non-digital intake and output, + representation, provenance, filing, retention, legal hold, and archive + consequences where the journey creates evidence or a record. The Docs module selects and links these views according to installed capabilities and actor context. Feature repositories remain the source of @@ -436,6 +466,9 @@ or the external editor the document-lifecycle owner. link, callback, webhook, file, identity, or data row. - Do not claim a stage complete from local unit tests. Use pinned composition, target integration, failure drills, adaptive docs, and operator evidence. +- Do not claim a module complete while its relevant information-governance + dimensions remain `contract_only` or while the reference journey lacks an + assisted-channel and records outcome where those are applicable. - A later stage may prototype contracts while the preceding gate is being proven, but it may not redefine an owning module's boundary by convenience. diff --git a/docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md b/docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md index 26b7c1a..19f0d1a 100644 --- a/docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md +++ b/docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md @@ -3,6 +3,8 @@ For the exact external handoff, least-privilege collector permissions and live two-node acceptance procedure, see [`PRODUCTION_TARGET_HANDOFF.md`](PRODUCTION_TARGET_HANDOFF.md). +For a reproducible local or multi-hypervisor libvirt/K3s target, use +[`KUBERNETES_TEST_LAB.md`](KUBERNETES_TEST_LAB.md). ## Implemented Contract @@ -74,6 +76,7 @@ python tools/deployment/govoplan-deploy.py render-kubernetes \ --namespace govoplan \ --secret-name govoplan-runtime \ --tls-secret-name govoplan-tls \ + --s3-ca-secret-name govoplan-s3-ca \ --ingress-class-name nginx \ --output /srv/govoplan/default/kubernetes.json ``` @@ -90,11 +93,25 @@ command prints the exact required key contract. Review the generated `FORWARDED_ALLOW_IPS` value and replace it with the exact ingress-proxy network before production use. +When an external S3 endpoint is signed by a private CA, create the optional CA +Secret with a `ca.crt` key and pass `--s3-ca-secret-name`. The renderer mounts +that Secret read-only and sets `AWS_CA_BUNDLE` for API, worker, scheduler, +migration and database-wait containers. It does not disable certificate +verification or replace the WebUI trust store. + The generated containers run as non-root with a read-only root filesystem and -an ephemeral `/tmp`. Runtime Deployments wait for the exact configured database -migration heads before starting. The API exposes `/health/ready`, which fails -while that API node is draining or cannot prove its runtime-coordination -heartbeat. +an ephemeral `/tmp`. Celery Beat keeps its replaceable schedule database there; +durable schedule definitions remain in shared state. The WebUI resolves its +configured API Service when the container starts, so Kubernetes deployments do +not inherit the Compose-only `load-balancer` hostname. Runtime Deployments wait +for the exact dependency-resolved database migration heads before starting. +The API exposes `/health/ready`, which fails while that API node is draining or +cannot prove its runtime-coordination heartbeat. + +Replicated API, WebUI, and worker Deployments use a hard hostname-spread +constraint scoped to the current pod-template hash. A rollout therefore keeps +each replica set distributed across independently schedulable nodes instead of +allowing all replacement pods to settle on one node after the old set exits. ## Runtime Coordination diff --git a/docs/STRATEGIC_REVIEW_2026-08-05.md b/docs/STRATEGIC_REVIEW_2026-08-05.md new file mode 100644 index 0000000..2f8e0f7 --- /dev/null +++ b/docs/STRATEGIC_REVIEW_2026-08-05.md @@ -0,0 +1,126 @@ +# Strategic Review - 2026-08-05 + +## Assessment + +GovOPlaN has not lost its central direction. The architecture now expresses a +coherent institutional governance platform, but architecture and repository +breadth have advanced faster than complete, usable outcomes. The immediate +need is convergence: fewer simultaneous fronts, stronger cross-cutting +adoption, and end-to-end reference journeys that non-developers can complete. + +This is a dated review. Current status belongs in +[Strategy Status](STRATEGY_STATUS.md); stable direction belongs in +[Platform Core Ideas](PLATFORM_CORE_IDEAS.md). + +## What Is Already Strong + +- A modular runtime with manifests, capabilities, interfaces, migrations, + optional integrations, signed releases, and permutation checks. +- Explicit institutional semantics for identity, representation, + organization, function, mandate, service, case, party, approval, decision, + evidence, and record references. +- Governed communication foundations spanning Campaign, Mail, Files, Postbox, + Addresses, Distribution Lists, Templates, Audit, and Policy. +- Governed data foundations spanning Connectors, Datasources, Dataflow, + Reporting, Search, and immutable provenance. +- Bitemporal browsing, views, contextual documentation, action/effect + contracts, event delivery, recovery ledgers, and stateless deployment + contracts. +- A credible deployment and release foundation with signed artifacts and + reproducible composition evidence. + +## Where The Program Veered + +### Repository breadth preceded product proof + +Logical modularity often became a repository before a reference journey proved +that an independent release boundary was required. Scaffolds are useful as +ownership markers, but their number makes the product appear broader and more +complete than its supported outcomes. + +### Foundations outran reference gates + +Later-stage contracts such as federation, encryption, formal governance, +deployment evidence, and broad module metadata were developed while basic +human-work and records journeys remained incomplete. Those foundations are not +wasted; they now need to be consumed by a small number of demonstrable +products. + +### The module graph leaked into the experience + +Navigation, routes, administration, errors, documentation, and configuration +often present module names and package structure directly. This is appropriate +for operators, but ordinary users should see work, services, records, and +outcomes. + +### Status became duplicated + +Roadmaps, target architecture, fit assessments, issue comments, and release +documents each contained partial implementation snapshots. Their stable +decisions remain valuable, but volatile counts and maturity claims diverged. + +### Too much work remained active simultaneously + +The issue portfolio had many high-priority and in-progress items without +milestones. This reduces the signal of both labels and roadmap order and makes +completion harder to demonstrate. + +## Where GovOPlaN Has Not Gone Far Enough + +1. No composition has yet crossed the full `reference_ready` gate. +2. The human-work spine is incomplete: work queues, tasks, handoffs, deadlines, + reminders, escalation, and resumption need a coherent user experience. +3. Records and document management remain too shallow for a public-sector + operating platform. +4. Real target integrations and GovOPlaN-to-GovOPlaN federation are not yet + proven. +5. Temporal browsing, purpose-aware access, retention, and institutional + context exist as contracts but are not adopted uniformly by domain reads + and effects. +6. German completeness, contextual help, accessibility, responsive behavior, + and browser-level journey testing are not yet release gates everywhere. +7. Multi-host, backup/restore, provider interoperability, and independent + signed target evidence still require real environments and operators. + +## Important Omissions + +- a named first institution, bounded users, volumes, and operating constraints; +- measurable usability outcomes, not only functional tests; +- installable sector packages and migration/exit demonstrations; +- support, upgrade, deprecation, and LTS promises; +- complete assisted, paper, telephone, and in-person channel handling; +- a native eAkte/records model that can also overlay an external DMS or archive. + +## Opportunities Beyond The Original Idea + +- an institutional digital twin that exposes responsibilities, dependencies, + obligations, services, work, data, controls, and change impact over time; +- continuous assurance that evaluates controls and evidence as work happens; +- process mining and conformance analysis over governed event histories; +- federated product packages and inter-institution case/evidence exchange; +- accountable assistance that drafts and explains without obscuring authority; +- public evidence chains that disclose decisions and provenance without + exposing protected source data. + +## Recommended Reset + +1. Freeze new repositories unless a real journey proves an independent owner, + release lifecycle, security boundary, or optional installation need. +2. Use one generated maturity/status dashboard and one current status document. +3. Complete governed communication and function-bound Postbox against a real + target. +4. Complete the monthly-data journey, then sanctions screening on the same + data foundations. +5. Complete one browser-driven service-to-decision journey, including assisted + intake and records. +6. Make eAkte/records the next major product-depth program. +7. Tie feature work to a reference journey, a security/recovery gate, or a + measured usability defect. + +## Success Criterion + +The reset succeeds when a public institution can install a signed composition, +configure a named procedure, complete it through digital and assisted channels, +connect an external source, reconstruct the authority and evidence, recover it +after failure, and transfer or retire it without custom code. + diff --git a/docs/STRATEGY_STATUS.md b/docs/STRATEGY_STATUS.md new file mode 100644 index 0000000..0553efa --- /dev/null +++ b/docs/STRATEGY_STATUS.md @@ -0,0 +1,118 @@ +# GovOPlaN Strategy Status + +## Status Record + +| Field | Value | +| --- | --- | +| Reconciled on | 2026-08-05 | +| Source scope | Local workspace manifests, source inventory, focused journey checks, signed release evidence, and live Gitea issue state | +| Stable direction | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) and [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) | +| Delivery source | Gitea issues | + +This is the only prose source for current cross-product status. It is a +reconciliation, not a release certification. Module manifests and target +evidence remain authoritative for specific maturity claims. + +## Portfolio Snapshot + +- 65 source module manifests were loadable and architecture-declared. +- 47 modules declared `vertical_slice`; 18 declared `scaffold`. +- No module declared `reference_ready`, `supported`, or `lts`. +- The live portfolio had 133 open issues, including 39 priority-P1 items. +- 117 open issues had no milestone, so issue labels do not yet express a + reliable completion sequence on their own. +- Three product package manifests existed: governed communication, governed + data and assurance, and service to decision. None had crossed the complete + target-evidence gate. + +These counts are dated. Refresh them rather than copying them into another +document. + +## Interface And Contract Evidence + +The 2026-08-05 source inventory found: + +- 1,247 UI fields and 1,220 UI actions; +- 7,929 stable interface declarations with no duplicate IDs; +- 39 frontend routes and 872 backend endpoints; +- no public WebUI surfaces missing runtime declarations; +- no stale runtime route declarations; +- no unclassified endpoint without a static UI reference; +- all 1,247 fields with a resolvable F1 context; 1,087 remain candidates for + richer field-specific content beyond page/module fallback; +- German (`de`) as the complete reference locale and no used key missing from + the required German or English catalogs; +- 260 module information-governance dimensions classified as `contract_only`. + This is an honest platform-wide baseline, not a claim that temporal, + purpose, retention, and institutional-context adoption is complete. + +## Credible Current Outcomes + +### Platform foundation + +Module discovery, optional dependency validation, migrations, shared WebUI, +tenant and access foundations, signed catalogs/packages, event delivery, +recovery contracts, contextual help, views, temporal titlebar context, and +stateless-runtime patterns are implemented and tested at varying depths. + +### Governed communication + +Campaign authoring, recipient data, attachments, templates, mail profiles, +mock/real delivery paths, audit evidence, reporting, distribution-list +composition, and optional Postbox delivery form the deepest product cluster. +Target provider, accessibility, recovery, and high-volume evidence still +prevent a reference-ready claim. + +### Institutional service and decision + +Services, Forms, Forms Runtime, Cases, Parties, Mandates, Approvals, Committee, +Voting, Decisions, Portal, Postbox, and Audit have an executable service-to- +decision fixture. Browser-complete assisted intake, production identity, +records, delivery, and target evidence remain. + +### Governed data and assurance + +Connectors, Datasources, Dataflow, Reporting, Search, Policy, Risk Compliance, +and Workflow provide source governance, immutable snapshots, transformation, +quality, semantic reporting, and provenance foundations. The monthly-data and +sanctions journeys still need real connectors, complete interactive +reconciliation, publication/export, and guided handoff evidence. + +## Material Gaps + +| Gap | Consequence | Next proof | +| --- | --- | --- | +| No reference-ready product package | The platform cannot yet make a bounded supported-product claim | Complete one named target composition and evidence bundle | +| Human-work spine incomplete | Users still navigate modules and remember unfinished work | Task/work inbox, resumable guided journey, deadlines and handoffs | +| Records/eAkte shallow | Institutional memory and disposition remain fragmented | Native record lifecycle plus external DMS/archive overlay | +| Cross-cutting governance adoption uneven | Historical and purpose-sensitive behavior varies by module | Enforced adoption declarations and route/query/effect migration | +| Explicit help/accessibility depth incomplete | German/reference and F1 association gates now pass, but generic fallback remains too common | High-risk German help content and browser/a11y matrix | +| Real federation absent | Cross-institution exchange remains connector-specific | Paired-instance signed exchange and reconciliation proof | +| External production evidence incomplete | Scale, restore, interoperability and custody claims remain conditional | Real target drills and independent signed evidence | + +## Active Strategic Order + +1. Establish German, help, temporal, purpose, retention, and institutional + context as enforceable platform quality contracts. +2. Complete governed communication and Postbox against a named target. +3. Complete the monthly-data flow and use it as the data foundation for + sanctions screening. +4. Complete one digital and assisted service-to-decision journey with an eAkte. +5. Add native PostgreSQL search coverage for the objects used by those + journeys; keep OpenSearch optional. +6. Prove one external product connector and one GovOPlaN federation exchange. +7. Finish multi-host, restore, provider, accessibility, and independent signed + target evidence before increasing maturity claims. + +## Refresh Procedure + +Refresh this page only from evidence: + +1. run `tools/checks/check-manifest-shapes.py`; +2. run `tools/inventory/platform-interface-inventory.py --strict + --strict-declarations --strict-endpoints`; +3. run the selected reference-journey checks; +4. inspect signed release and target evidence; +5. query live Gitea issue/milestone state; +6. update the dated values and material gaps here; +7. retain prior assessments as dated evidence rather than rewriting them. diff --git a/docs/gitea-labels.json b/docs/gitea-labels.json index 42cc430..0a22605 100644 --- a/docs/gitea-labels.json +++ b/docs/gitea-labels.json @@ -353,6 +353,12 @@ "description": "GovOPlaN Projects module behavior or integration.", "exclusive": false }, + { + "name": "module/records", + "color": "0052cc", + "description": "GovOPlaN Records and eAkte lifecycle behavior or integration.", + "exclusive": false + }, { "name": "module/reporting", "color": "c2e0c6", diff --git a/packages/govoplan-meta/pyproject.toml b/packages/govoplan-meta/pyproject.toml index e264579..047efe8 100644 --- a/packages/govoplan-meta/pyproject.toml +++ b/packages/govoplan-meta/pyproject.toml @@ -4,82 +4,82 @@ build-backend = "setuptools.build_meta" [project] name = "govoplan" -version = "0.1.15" +version = "0.1.16" description = "Developer convenience package for a versioned GovOPlaN composition" readme = "README.md" requires-python = ">=3.12" license = { text = "AGPL-3.0-or-later" } dependencies = [ - "govoplan-core[server]==0.1.15", - "govoplan-tenancy==0.1.15", - "govoplan-organizations==0.1.15", - "govoplan-identity==0.1.15", - "govoplan-idm==0.1.15", - "govoplan-access==0.1.15", - "govoplan-admin==0.1.15", - "govoplan-policy==0.1.15", - "govoplan-audit==0.1.15", - "govoplan-dashboard==0.1.15", - "govoplan-files==0.1.15", - "govoplan-mail==0.1.15", - "govoplan-campaign==0.1.15", - "govoplan-calendar==0.1.15", - "govoplan-docs==0.1.15", - "govoplan-ops==0.1.15", + "govoplan-core[server]==0.1.16", + "govoplan-tenancy==0.1.16", + "govoplan-organizations==0.1.16", + "govoplan-identity==0.1.16", + "govoplan-idm==0.1.16", + "govoplan-access==0.1.16", + "govoplan-admin==0.1.16", + "govoplan-policy==0.1.16", + "govoplan-audit==0.1.16", + "govoplan-dashboard==0.1.16", + "govoplan-files==0.1.16", + "govoplan-mail==0.1.16", + "govoplan-campaign==0.1.16", + "govoplan-calendar==0.1.16", + "govoplan-docs==0.1.16", + "govoplan-ops==0.1.16", ] [project.optional-dependencies] full = [ - "govoplan-addresses==0.1.15", - "govoplan-approvals==0.1.15", - "govoplan-assets==0.1.15", - "govoplan-booking==0.1.15", - "govoplan-cases==0.1.15", - "govoplan-certificates==0.1.15", - "govoplan-committee==0.1.15", - "govoplan-connectors==0.1.15", - "govoplan-consultation==0.1.15", - "govoplan-contracts==0.1.15", - "govoplan-dataflow==0.1.15", - "govoplan-datasources==0.1.15", - "govoplan-decisions==0.1.15", - "govoplan-dist-lists==0.1.15", - "govoplan-encryption==0.1.15", - "govoplan-evaluation==0.1.15", - "govoplan-facilities==0.1.15", - "govoplan-forms==0.1.15", - "govoplan-forms-runtime==0.1.15", - "govoplan-grants==0.1.15", - "govoplan-helpdesk==0.1.15", - "govoplan-identity-trust==0.1.15", - "govoplan-inspections==0.1.15", - "govoplan-learning==0.1.15", - "govoplan-mandates==0.1.15", - "govoplan-notifications==0.1.15", - "govoplan-parties==0.1.15", - "govoplan-permits==0.1.15", - "govoplan-poll==0.1.15", - "govoplan-portal==0.1.15", - "govoplan-postbox==0.1.15", - "govoplan-procurement==0.1.15", - "govoplan-projects==0.1.15", - "govoplan-records==0.1.15", - "govoplan-reporting==0.1.15", - "govoplan-resources==0.1.15", - "govoplan-rest==0.1.15", - "govoplan-risk-compliance==0.1.15", - "govoplan-scheduling==0.1.15", - "govoplan-search==0.1.15", - "govoplan-services==0.1.15", - "govoplan-soap==0.1.15", - "govoplan-templates==0.1.15", - "govoplan-tickets==0.1.15", - "govoplan-transparency==0.1.15", - "govoplan-views==0.1.15", - "govoplan-voting==0.1.15", - "govoplan-wiki==0.1.15", - "govoplan-workflow==0.1.15", - "govoplan-workflow-engine==0.1.15", + "govoplan-addresses==0.1.16", + "govoplan-approvals==0.1.16", + "govoplan-assets==0.1.16", + "govoplan-booking==0.1.16", + "govoplan-cases==0.1.16", + "govoplan-certificates==0.1.16", + "govoplan-committee==0.1.16", + "govoplan-connectors==0.1.16", + "govoplan-consultation==0.1.16", + "govoplan-contracts==0.1.16", + "govoplan-dataflow==0.1.16", + "govoplan-datasources==0.1.16", + "govoplan-decisions==0.1.16", + "govoplan-dist-lists==0.1.16", + "govoplan-encryption==0.1.16", + "govoplan-evaluation==0.1.16", + "govoplan-facilities==0.1.16", + "govoplan-forms==0.1.16", + "govoplan-forms-runtime==0.1.16", + "govoplan-grants==0.1.16", + "govoplan-helpdesk==0.1.16", + "govoplan-identity-trust==0.1.16", + "govoplan-inspections==0.1.16", + "govoplan-learning==0.1.16", + "govoplan-mandates==0.1.16", + "govoplan-notifications==0.1.16", + "govoplan-parties==0.1.16", + "govoplan-permits==0.1.16", + "govoplan-poll==0.1.16", + "govoplan-portal==0.1.16", + "govoplan-postbox==0.1.16", + "govoplan-procurement==0.1.16", + "govoplan-projects==0.1.16", + "govoplan-records==0.1.16", + "govoplan-reporting==0.1.16", + "govoplan-resources==0.1.16", + "govoplan-rest==0.1.16", + "govoplan-risk-compliance==0.1.16", + "govoplan-scheduling==0.1.16", + "govoplan-search==0.1.16", + "govoplan-services==0.1.16", + "govoplan-soap==0.1.16", + "govoplan-templates==0.1.16", + "govoplan-tickets==0.1.16", + "govoplan-transparency==0.1.16", + "govoplan-views==0.1.16", + "govoplan-voting==0.1.16", + "govoplan-wiki==0.1.16", + "govoplan-workflow==0.1.16", + "govoplan-workflow-engine==0.1.16", ] [project.urls] diff --git a/requirements-release.txt b/requirements-release.txt index e524d05..fa530cc 100644 --- a/requirements-release.txt +++ b/requirements-release.txt @@ -1,18 +1,18 @@ # Whole-product release install from immutable, independently versioned module tags. # Only add a module after its referenced tag has been published. ../govoplan-core[server] -govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.15 -govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.15 -govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.15 -govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.15 -govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.15 -govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.15 -govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.15 -govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.15 -govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.15 -govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.15 -govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.15 -govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.15 -govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.15 -govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.15 -govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.15 +govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.16 +govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.16 +govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.16 +govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.16 +govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.16 +govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.16 +govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.16 +govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.16 +govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.16 +govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.16 +govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.16 +govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.16 +govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.16 +govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.16 +govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.16 diff --git a/tests/test_deployment_installer.py b/tests/test_deployment_installer.py index b790828..4091bc7 100644 --- a/tests/test_deployment_installer.py +++ b/tests/test_deployment_installer.py @@ -158,6 +158,81 @@ class DeploymentInstallerTests(unittest.TestCase): evidence["snapshot"]["ready_node_names"], ) + def test_kubernetes_api_loss_uses_a_non_json_mutation_command(self) -> None: + initial_pods = [ + _kubernetes_test_pod("api-a", "api", "node-a"), + _kubernetes_test_pod("api-b", "api", "node-b"), + _kubernetes_test_pod("web-a", "web", "node-a"), + _kubernetes_test_pod("web-b", "web", "node-b"), + ] + replacement_pods = [ + _kubernetes_test_pod("api-b", "api", "node-b"), + _kubernetes_test_pod("api-c", "api", "node-a"), + ] + deleted = False + actions: list[tuple[str, ...]] = [] + + def run(arguments): + if "nodes" in arguments: + return { + "items": [ + { + "metadata": {"name": name}, + "spec": {}, + "status": { + "conditions": [ + {"type": "Ready", "status": "True"} + ] + }, + } + for name in ("node-a", "node-b") + ] + } + if "deployments" in arguments: + return { + "items": [ + _kubernetes_test_deployment("api", 2), + _kubernetes_test_deployment("web", 2), + ] + } + return {"items": replacement_pods if deleted else initial_pods} + + def act(arguments): + nonlocal deleted + actions.append(tuple(arguments)) + deleted = True + + evidence = collect_kubernetes_evidence( + installation_id="govoplan-cluster", + namespace="govoplan", + ops_url="https://govoplan.example.test/api/v1/ops/status", + api_key="not-retained", + exercise_api_pod_loss=True, + command_runner=run, + action_runner=act, + json_fetcher=lambda _url, _key: { + "readiness": {"ready": True}, + "runtime_cluster": { + "composition": {"skewed": False}, + "software_versions": {"skewed": False}, + "queues": {"missing": []}, + }, + "checks": [ + { + "id": "database_capacity", + "state": "ok", + "detail": "Within budget", + } + ], + }, + ) + + self.assertEqual("passed", evidence["api_pod_loss"]["state"]) + self.assertEqual(1, len(actions)) + self.assertIn("delete", actions[0]) + self.assertNotIn("-o", actions[0]) + self.assertNotIn("not-retained", json.dumps(evidence)) + def test_pre_migration_failure_restores_checksum_verified_applied_bundle( self, ) -> None: @@ -347,6 +422,10 @@ class DeploymentInstallerTests(unittest.TestCase): "template" ]["spec"]["containers"][0]["command"] self.assertIn("govoplan_core.commands.fenced_run", scheduler_command) + self.assertEqual( + ["--schedule", "/tmp/celerybeat-schedule"], + scheduler_command[-2:], + ) api_init_command = deployments["govoplan-cluster-api"]["spec"]["template"][ "spec" ]["initContainers"][0]["command"] @@ -385,6 +464,12 @@ class DeploymentInstallerTests(unittest.TestCase): "spec" ]["containers"][0]["command"] self.assertIn("--concurrency", worker_command) + for deployment in deployments.values(): + spread = deployment["spec"]["template"]["spec"][ + "topologySpreadConstraints" + ][0] + self.assertEqual("DoNotSchedule", spread["whenUnsatisfiable"]) + self.assertEqual(["pod-template-hash"], spread["matchLabelKeys"]) self.assertEqual( "62", manifest["metadata"]["annotations"][ @@ -392,6 +477,99 @@ class DeploymentInstallerTests(unittest.TestCase): ], ) + def test_kubernetes_export_mounts_an_optional_s3_ca_on_backend_roles( + self, + ) -> None: + spec = default_spec( + installation_id="govoplan-cluster", + postgres_mode="external", + redis_mode="external", + storage_mode="s3", + api_replicas=2, + web_replicas=2, + worker_replicas=2, + api_image="registry.example.test/govoplan-api@sha256:" + "a" * 64, + web_image="registry.example.test/govoplan-web@sha256:" + "b" * 64, + ) + environment = initial_secrets( + spec, + supplied={ + "DATABASE_URL": "postgresql+psycopg://user:secret@postgres.example.test/govoplan", + "GOVOPLAN_DATABASE_URL_PGTOOLS": "postgresql://user:secret@postgres.example.test/govoplan", + "REDIS_URL": "rediss://:secret@redis.example.test/0", + "FILE_STORAGE_S3_ENDPOINT_URL": "https://s3.example.test", + "FILE_STORAGE_S3_REGION": "eu-test-1", + "FILE_STORAGE_S3_ACCESS_KEY_ID": "object-key", + "FILE_STORAGE_S3_SECRET_ACCESS_KEY": "object-secret", + "FILE_STORAGE_S3_BUCKET": "govoplan", + "GOVOPLAN_DB_CONNECTION_LIMIT": "100", + }, + ) + + manifest = render_kubernetes( + spec, + environment, + s3_ca_secret_name="govoplan-s3-ca", + backup_required=False, + ) + backend_pods = [ + item["spec"]["template"]["spec"] + for item in manifest["items"] + if item["kind"] in {"Deployment", "Job"} + and item["metadata"]["labels"].get("app.kubernetes.io/component") + in {"api", "worker", "scheduler", "migration"} + ] + web = next( + item + for item in manifest["items"] + if item["kind"] == "Deployment" + and item["metadata"]["labels"].get("app.kubernetes.io/component") + == "web" + ) + + self.assertTrue(backend_pods) + for pod in backend_pods: + self.assertIn( + { + "name": "s3-ca", + "secret": { + "secretName": "govoplan-s3-ca", + "items": [{"key": "ca.crt", "path": "s3-ca.crt"}], + }, + }, + pod["volumes"], + ) + for container in [*pod.get("initContainers", []), *pod["containers"]]: + self.assertIn( + { + "name": "AWS_CA_BUNDLE", + "value": "/etc/govoplan/trust/s3-ca.crt", + }, + container["env"], + ) + self.assertIn( + { + "name": "s3-ca", + "mountPath": "/etc/govoplan/trust", + "readOnly": True, + }, + container["volumeMounts"], + ) + self.assertNotIn( + "s3-ca", + { + volume["name"] + for volume in web["spec"]["template"]["spec"]["volumes"] + }, + ) + with self.assertRaisesRegex(ValueError, "S3 CA secret"): + render_kubernetes( + spec, + environment, + s3_ca_secret_name="INVALID_NAME", + backup_required=False, + ) + def test_kubernetes_export_splits_worker_queues_and_rejects_capacity_overrun( self, ) -> None: diff --git a/tests/test_kubernetes_lab.py b/tests/test_kubernetes_lab.py new file mode 100644 index 0000000..8d24e6c --- /dev/null +++ b/tests/test_kubernetes_lab.py @@ -0,0 +1,239 @@ +from __future__ import annotations + +from contextlib import redirect_stdout +import io +import json +from pathlib import Path +import stat +import subprocess +import sys +import tempfile +import unittest +from unittest.mock import MagicMock + + +META_ROOT = Path(__file__).resolve().parents[1] +LAB_TOOLS = META_ROOT / "tools" / "lab" +if str(LAB_TOOLS) not in sys.path: + sys.path.insert(0, str(LAB_TOOLS)) + +from govoplan_lab.cli import main # noqa: E402 +from govoplan_lab.config import LabConfigError, load_config # noqa: E402 +from govoplan_lab.lifecycle import ( # noqa: E402 + CommandRunner, + LabOperationError, + _assert_domain_owned, + _domain_description, + _render_kubectl_wrapper, + destroy, +) +from govoplan_lab.render import ( # noqa: E402 + render_k3s_config, + render_registry_config, + render_state_compose, + write_private, +) + + +REHEARSAL_CONFIG = LAB_TOOLS / "govoplan-lab.example.toml" +ACCEPTANCE_CONFIG = LAB_TOOLS / "govoplan-lab.acceptance.example.toml" + + +class KubernetesLabTests(unittest.TestCase): + def test_example_inventories_describe_their_evidence_boundary(self) -> None: + rehearsal = load_config(REHEARSAL_CONFIG) + acceptance = load_config(ACCEPTANCE_CONFIG) + + self.assertEqual("rehearsal", rehearsal.mode) + self.assertFalse(rehearsal.evidence_capable) + self.assertEqual(2, len(rehearsal.workers)) + self.assertEqual("acceptance", acceptance.mode) + self.assertTrue(acceptance.evidence_capable) + self.assertEqual(3, len({node.hypervisor for node in acceptance.nodes})) + self.assertEqual(3, len({node.failure_domain for node in acceptance.nodes})) + + def test_acceptance_inventory_rejects_collapsed_worker_failure_domains(self) -> None: + source = ACCEPTANCE_CONFIG.read_text(encoding="utf-8") + collapsed = source.replace( + 'hypervisor = "lab-admin@hypervisor-b.example.org"', + 'hypervisor = "lab-admin@hypervisor-a.example.org"', + ).replace('failure_domain = "rack-b"', 'failure_domain = "rack-a"') + with tempfile.TemporaryDirectory(prefix="govoplan-lab-config-") as directory: + path = Path(directory) / "lab.toml" + path.write_text(collapsed, encoding="utf-8") + + with self.assertRaisesRegex(LabConfigError, "acceptance mode"): + load_config(path) + + def test_create_without_apply_is_a_non_mutating_preview(self) -> None: + output = io.StringIO() + with redirect_stdout(output): + exit_code = main(["--config", str(REHEARSAL_CONFIG), "create"]) + + self.assertEqual(0, exit_code) + self.assertIn("Dry run: create", output.getvalue()) + self.assertIn("Re-run with --apply", output.getvalue()) + + def test_local_hypervisor_uses_system_libvirt_without_sudo(self) -> None: + config = load_config(REHEARSAL_CONFIG) + runner = CommandRunner(config) + runner.run = MagicMock( + return_value=subprocess.CompletedProcess([], 0, stdout=b"", stderr=b"") + ) + + runner.hypervisor(config.nodes[0], ["virsh", "dominfo", "test-domain"]) + + runner.run.assert_called_once_with( + [ + "virsh", + "--connect", + "qemu:///system", + "dominfo", + "test-domain", + ], + capture=False, + check=True, + timeout=None, + ) + + def test_local_hypervisor_file_operations_do_not_use_sudo(self) -> None: + config = load_config(REHEARSAL_CONFIG) + runner = CommandRunner(config) + runner.run = MagicMock( + return_value=subprocess.CompletedProcess([], 0, stdout=b"", stderr=b"") + ) + + runner.hypervisor(config.nodes[0], ["install", "-d", "/tmp/lab"]) + + runner.run.assert_called_once_with( + ["install", "-d", "/tmp/lab"], + capture=False, + check=True, + timeout=None, + ) + + def test_kubectl_wrapper_quotes_remote_arguments(self) -> None: + wrapper = _render_kubectl_wrapper( + ["ssh", "-i", "/tmp/lab key", "govoplan@example.test"] + ) + + self.assertIn("shlex.join(_REMOTE)", wrapper) + self.assertIn('["sudo", "--", "k3s", "kubectl", *sys.argv[1:]]', wrapper) + self.assertNotIn('kubectl \"$@\"', wrapper) + + def test_destroy_requires_the_exact_lab_name(self) -> None: + config = load_config(REHEARSAL_CONFIG) + + with self.assertRaisesRegex(LabOperationError, "--confirm"): + destroy( + config, + apply=True, + confirmation="wrong-lab", + purge_local_state=False, + ) + + def test_enroll_admin_without_apply_is_a_non_mutating_preview(self) -> None: + output = io.StringIO() + with redirect_stdout(output): + exit_code = main( + [ + "--config", + str(REHEARSAL_CONFIG), + "enroll-admin", + "--email", + "owner@example.test", + ] + ) + + self.assertEqual(0, exit_code) + self.assertIn("Dry run: enroll-admin", output.getvalue()) + self.assertIn("owner@example.test", output.getvalue()) + + def test_domain_ownership_requires_marker_and_expected_disks(self) -> None: + config = load_config(REHEARSAL_CONFIG) + node = config.nodes[0] + node_directory = f"{config.vm_image_directory}/{config.name}/{node.name}" + runner = MagicMock() + runner.hypervisor.side_effect = [ + subprocess.CompletedProcess( + [], + 0, + stdout=(_domain_description(config, node) + "\n").encode(), + stderr=b"", + ), + subprocess.CompletedProcess( + [], + 0, + stdout=( + f"file disk vda {node_directory}/root.qcow2\n" + f"file cdrom sda {node_directory}/seed.img\n" + ).encode(), + stderr=b"", + ), + ] + + _assert_domain_owned(config, runner, node) + + runner.hypervisor.side_effect = [ + subprocess.CompletedProcess( + [], + 0, + stdout=b"unrelated domain\n", + stderr=b"", + ) + ] + with self.assertRaisesRegex(LabOperationError, "ownership marker"): + _assert_domain_owned(config, runner, node) + + def test_state_compose_uses_only_supplied_pinned_images(self) -> None: + names = ("postgres", "redis", "garage", "managed_ingress", "test_mail") + images = { + name: f"registry.example.test/{name}@sha256:{index:064x}" + for index, name in enumerate(names, start=1) + } + + compose = json.loads(render_state_compose(images)) + + self.assertEqual(images["postgres"], compose["services"]["postgres"]["image"]) + self.assertEqual(images["redis"], compose["services"]["redis"]["image"]) + self.assertEqual(images["garage"], compose["services"]["garage"]["image"]) + self.assertEqual( + images["managed_ingress"], compose["services"]["s3-tls"]["image"] + ) + self.assertEqual( + images["test_mail"], compose["services"]["test-mail"]["image"] + ) + + def test_k3s_workers_join_the_primary_control_and_receive_failure_labels( + self, + ) -> None: + config = load_config(ACCEPTANCE_CONFIG) + worker = config.workers[0] + + rendered = render_k3s_config(config, worker, cluster_token="test-token") + + self.assertIn(f'server: "https://{config.primary_control.address}:6443"', rendered) + self.assertIn( + f'topology.govoplan.add-ideas.de/failure-domain={worker.failure_domain}', + rendered, + ) + self.assertNotIn("cluster-init", rendered) + + def test_registry_credentials_are_all_or_nothing(self) -> None: + self.assertEqual("", render_registry_config("", "")) + with self.assertRaisesRegex(ValueError, "supplied together"): + render_registry_config("publisher", "") + rendered = render_registry_config("publisher", "secret") + self.assertIn('"git.add-ideas.de"', rendered) + self.assertIn("publisher", rendered) + + def test_private_writer_enforces_owner_only_permissions(self) -> None: + with tempfile.TemporaryDirectory(prefix="govoplan-lab-private-") as directory: + path = Path(directory) / "nested" / "secret.txt" + write_private(path, "secret\n") + + self.assertEqual(0o600, stat.S_IMODE(path.stat().st_mode)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_release_catalog_entry_synthesis.py b/tests/test_release_catalog_entry_synthesis.py index 49f5f15..e8eb165 100644 --- a/tests/test_release_catalog_entry_synthesis.py +++ b/tests/test_release_catalog_entry_synthesis.py @@ -1,6 +1,7 @@ from __future__ import annotations import sys +import tomllib import unittest from pathlib import Path @@ -26,6 +27,13 @@ from govoplan_release.catalog_entry_synthesis import ( # noqa: E402 from govoplan_release.selective_catalog import apply_repo_updates # noqa: E402 +def repository_version(name: str) -> str: + payload = tomllib.loads( + (META_ROOT.parent / name / "pyproject.toml").read_text(encoding="utf-8") + ) + return str(payload["project"]["version"]) + + class ReleaseCatalogEntrySynthesisTests(unittest.TestCase): def test_catalog_entry_preserves_architecture_and_provider_declarations( self, @@ -92,6 +100,12 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase): ) self.assertEqual("vertical_slice", entry["architecture"]["maturity"]) + self.assertEqual( + "contract_only", + entry["information_governance"]["dimensions"]["retention"][ + "adoption" + ], + ) self.assertEqual( "external_mirror", entry["external_providers"][0]["objects"][0][ @@ -116,9 +130,12 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase): changes = apply_repo_updates( payload, repo_versions={ - "govoplan-addresses": "0.1.9", - "govoplan-poll": "0.1.11", - "govoplan-scheduling": "0.1.11", + name: repository_version(name) + for name in ( + "govoplan-addresses", + "govoplan-poll", + "govoplan-scheduling", + ) }, repo_contracts={}, repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN", diff --git a/tests/test_runtime_distribution_build.py b/tests/test_runtime_distribution_build.py index 8207d0f..9848352 100644 --- a/tests/test_runtime_distribution_build.py +++ b/tests/test_runtime_distribution_build.py @@ -121,6 +121,22 @@ class RuntimeDistributionBuildTests(unittest.TestCase): ): self.assertIn(temporary_path, nginx) + def test_web_runtime_resolves_the_configured_api_service_at_startup(self) -> None: + nginx = (ROOT / "tools/release/runtime/nginx.conf").read_text( + encoding="utf-8" + ) + dockerfile = (ROOT / "tools/release/runtime/Dockerfile.web").read_text( + encoding="utf-8" + ) + entrypoint = (ROOT / "tools/release/runtime/web-entrypoint.sh").read_text( + encoding="utf-8" + ) + + self.assertIn("proxy_pass ${GOVOPLAN_API_UPSTREAM};", nginx) + self.assertIn("nginx.conf.template", dockerfile) + self.assertIn("govoplan-web-entrypoint", dockerfile) + self.assertIn("envsubst '${GOVOPLAN_API_UPSTREAM}'", entrypoint) + def test_workflow_verifies_portable_bootstrap_artifacts_before_execution( self, ) -> None: diff --git a/tools/checks/check-manifest-shapes.py b/tools/checks/check-manifest-shapes.py index 49fd1c3..70ec3e9 100644 --- a/tools/checks/check-manifest-shapes.py +++ b/tools/checks/check-manifest-shapes.py @@ -128,6 +128,13 @@ def main() -> int: manifest=manifest, ) ) + errors.extend( + _information_governance_evidence_errors( + repository_name=repository_name, + repository_root=repository_root, + manifest=manifest, + ) + ) manifests.append(manifest) @@ -153,6 +160,20 @@ def main() -> int: f"Architecture declaration coverage: {declared}/{len(manifests)} modules " f"({(declared / len(manifests) * 100):.1f}%)." ) + governance_counts: dict[str, int] = {} + for manifest in manifests: + for dimension in manifest.information_governance.dimensions.values(): + governance_counts[dimension.adoption] = ( + governance_counts.get(dimension.adoption, 0) + 1 + ) + print( + "Information-governance adoption: " + + ", ".join( + f"{status}={count}" + for status, count in sorted(governance_counts.items()) + ) + + "." + ) return 0 @@ -201,6 +222,39 @@ def _architecture_evidence_errors( return errors +def _information_governance_evidence_errors( + *, + repository_name: str, + repository_root: Path, + manifest: object, +) -> list[str]: + declaration = getattr(manifest, "information_governance", None) + if declaration is None: + return [ + f"{repository_name}: module has no information-governance declaration" + ] + errors: list[str] = [] + for dimension_name, dimension in declaration.dimensions.items(): + for reference in dimension.evidence: + if not _looks_like_repository_reference(reference): + continue + candidate = (repository_root / reference).resolve() + try: + candidate.relative_to(repository_root.resolve()) + except ValueError: + errors.append( + f"{repository_name}: {dimension_name} evidence escapes the " + f"repository: {reference!r}" + ) + continue + if not candidate.exists(): + errors.append( + f"{repository_name}: {dimension_name} evidence does not exist: " + f"{reference!r}" + ) + return errors + + def _looks_like_repository_reference(reference: str) -> bool: normalized = reference.strip() if not normalized or "://" in normalized: diff --git a/tools/deployment/govoplan_deploy/cli.py b/tools/deployment/govoplan_deploy/cli.py index 82d60b5..6d13728 100644 --- a/tools/deployment/govoplan_deploy/cli.py +++ b/tools/deployment/govoplan_deploy/cli.py @@ -219,6 +219,13 @@ def build_parser() -> argparse.ArgumentParser: kubernetes.add_argument("--namespace", default="govoplan") kubernetes.add_argument("--secret-name", default="govoplan-runtime") kubernetes.add_argument("--tls-secret-name", default="govoplan-tls") + kubernetes.add_argument( + "--s3-ca-secret-name", + help=( + "Optional Secret containing ca.crt for the external S3 endpoint; " + "mounted read-only into backend runtime roles." + ), + ) kubernetes.add_argument("--ingress-class-name") kubernetes.add_argument( "--output", @@ -239,7 +246,10 @@ def build_parser() -> argparse.ArgumentParser: verify_kubernetes.add_argument( "--api-key-env", default="GOVOPLAN_OPS_API_KEY", - help="Environment variable containing an API key with Ops read scope.", + help=( + "Environment variable containing an API key authorized to read " + "Ops status." + ), ) verify_kubernetes.add_argument( "--exercise-api-pod-loss", @@ -1146,6 +1156,7 @@ def _render_kubernetes(args: argparse.Namespace) -> int: namespace=args.namespace, secret_name=args.secret_name, tls_secret_name=args.tls_secret_name, + s3_ca_secret_name=args.s3_ca_secret_name, ingress_class_name=args.ingress_class_name, backup_required=backup_required, backup_evidence=backup_summary, @@ -1170,7 +1181,8 @@ def _verify_kubernetes(args: argparse.Namespace) -> int: api_key = str(os.environ.get(args.api_key_env) or "").strip() if not api_key: raise ValueError( - f"{args.api_key_env} must contain an API key with Ops read scope" + f"{args.api_key_env} must contain an API key authorized to read " + "Ops status" ) ops_url = args.ops_url or (spec.public_url.rstrip("/") + "/api/v1/ops/status") evidence = collect_kubernetes_evidence( diff --git a/tools/deployment/govoplan_deploy/cluster_evidence.py b/tools/deployment/govoplan_deploy/cluster_evidence.py index 13a361f..e5e477e 100644 --- a/tools/deployment/govoplan_deploy/cluster_evidence.py +++ b/tools/deployment/govoplan_deploy/cluster_evidence.py @@ -14,6 +14,7 @@ from urllib.request import Request, urlopen JsonObject = dict[str, Any] CommandRunner = Callable[[Sequence[str]], JsonObject] +ActionRunner = Callable[[Sequence[str]], None] JsonFetcher = Callable[[str, str], JsonObject] @@ -26,11 +27,13 @@ def collect_kubernetes_evidence( exercise_api_pod_loss: bool = False, timeout_seconds: float = 180.0, command_runner: CommandRunner | None = None, + action_runner: ActionRunner | None = None, json_fetcher: JsonFetcher | None = None, ) -> JsonObject: """Inspect a live cluster and optionally exercise one API pod replacement.""" run_json = command_runner or _kubectl_json + run_action = action_runner or _kubectl_action fetch_json = json_fetcher or _fetch_json nodes = run_json(("get", "nodes", "-o", "json")) pods = run_json( @@ -76,6 +79,7 @@ def collect_kubernetes_evidence( initial_pods=pods, timeout_seconds=timeout_seconds, run_json=run_json, + run_action=run_action, fetch_json=fetch_json, ) evidence = { @@ -211,6 +215,7 @@ def _exercise_api_pod_loss( initial_pods: Mapping[str, Any], timeout_seconds: float, run_json: CommandRunner, + run_action: ActionRunner, fetch_json: JsonFetcher, ) -> JsonObject: candidates = [ @@ -228,7 +233,7 @@ def _exercise_api_pod_loss( victim = sorted(candidates, key=lambda item: item["name"])[0] initial_uids = {item["uid"] for item in candidates} desired_ready = len(candidates) - run_json( + run_action( ( "-n", namespace, @@ -236,8 +241,6 @@ def _exercise_api_pod_loss( "pod", victim["name"], "--wait=false", - "-o", - "json", ) ) deadline = time.monotonic() + timeout_seconds @@ -309,6 +312,22 @@ def _kubectl_json(arguments: Sequence[str]) -> JsonObject: return payload +def _kubectl_action(arguments: Sequence[str]) -> None: + kubectl = shutil.which("kubectl") + if kubectl is None: + raise ValueError("kubectl is required for Kubernetes evidence collection") + result = subprocess.run( + (kubectl, *arguments), + check=False, + capture_output=True, + text=True, + timeout=60, + ) + if result.returncode: + detail = result.stderr.strip() or result.stdout.strip() + raise ValueError(f"kubectl failed: {detail}") + + def _fetch_json(url: str, api_key: str) -> JsonObject: request = Request( url, diff --git a/tools/deployment/govoplan_deploy/kubernetes.py b/tools/deployment/govoplan_deploy/kubernetes.py index 182340b..f95c473 100644 --- a/tools/deployment/govoplan_deploy/kubernetes.py +++ b/tools/deployment/govoplan_deploy/kubernetes.py @@ -76,13 +76,20 @@ def render_kubernetes( namespace: str = "govoplan", secret_name: str = "govoplan-runtime", tls_secret_name: str = "govoplan-tls", + s3_ca_secret_name: str | None = None, ingress_class_name: str | None = None, backup_required: bool = True, backup_evidence: Mapping[str, object] | None = None, ) -> dict[str, Any]: """Render runtime roles only; shared state services stay externally managed.""" - _validate_cluster_profile(spec, environment, namespace, secret_name) + _validate_cluster_profile( + spec, + environment, + namespace, + secret_name, + s3_ca_secret_name=s3_ca_secret_name, + ) worker_pools = _worker_pools(spec, environment) database_capacity = _database_capacity(spec, environment, worker_pools) name = _resource_name(spec.installation_id) @@ -155,6 +162,7 @@ def render_kubernetes( ), config_name=config_name, secret_name=secret_name, + s3_ca_secret_name=s3_ca_secret_name, service_account=service_account, container_port=8000, readiness_path="/health/ready", @@ -179,6 +187,7 @@ def render_kubernetes( command=(), config_name=None, secret_name=None, + s3_ca_secret_name=None, service_account=service_account, container_port=8080, extra_environment={"GOVOPLAN_API_UPSTREAM": f"http://{name}-api:8000"}, @@ -198,6 +207,7 @@ def render_kubernetes( image=spec.release.api_image, config_name=config_name, secret_name=secret_name, + s3_ca_secret_name=s3_ca_secret_name, service_account=service_account, database_environment=_role_database_environment( environment, @@ -237,6 +247,7 @@ def render_kubernetes( ), config_name=config_name, secret_name=secret_name, + s3_ca_secret_name=s3_ca_secret_name, service_account=service_account, extra_environment={ **_role_database_environment(environment, "WORKER"), @@ -287,9 +298,12 @@ def render_kubernetes( "beat", "--loglevel", "INFO", + "--schedule", + "/tmp/celerybeat-schedule", ), config_name=config_name, secret_name=secret_name, + s3_ca_secret_name=s3_ca_secret_name, service_account=service_account, extra_environment=_role_database_environment( environment, @@ -318,6 +332,7 @@ def render_kubernetes( "annotations": { "govoplan.add-ideas.de/profile": "stateless-shared-state", "govoplan.add-ideas.de/secret-contract": ",".join(_SECRET_KEYS), + "govoplan.add-ideas.de/s3-ca-secret": s3_ca_secret_name or "", "govoplan.add-ideas.de/database-connection-peak": str( database_capacity["peak"] ), @@ -342,9 +357,13 @@ def _validate_cluster_profile( environment: Mapping[str, str], namespace: str, secret_name: str, + *, + s3_ca_secret_name: str | None, ) -> None: if not _DNS_LABEL.fullmatch(namespace) or not _DNS_LABEL.fullmatch(secret_name): raise ValueError("Kubernetes namespace and secret names must be DNS labels") + if s3_ca_secret_name is not None and not _DNS_LABEL.fullmatch(s3_ca_secret_name): + raise ValueError("Kubernetes S3 CA secret name must be a DNS label") if spec.installation_id == "govoplan-local": raise ValueError( "Kubernetes export requires a non-default stable installation id" @@ -633,6 +652,7 @@ def _deployment( command: tuple[str, ...], config_name: str | None, secret_name: str | None, + s3_ca_secret_name: str | None, service_account: str, container_port: int | None = None, readiness_path: str | None = None, @@ -660,6 +680,10 @@ def _deployment( ) if secret_name: environment.extend(_secret_environment(secret_name)) + if s3_ca_secret_name: + environment.append( + {"name": "AWS_CA_BUNDLE", "value": "/etc/govoplan/trust/s3-ca.crt"} + ) container: dict[str, Any] = { "name": role, "image": image, @@ -704,12 +728,16 @@ def _deployment( { "maxSkew": 1, "topologyKey": "kubernetes.io/hostname", - "whenUnsatisfiable": "ScheduleAnyway", + "whenUnsatisfiable": "DoNotSchedule", + "matchLabelKeys": ["pod-template-hash"], "labelSelector": {"matchLabels": role_labels}, } ], } container["volumeMounts"] = [{"name": "tmp", "mountPath": "/tmp"}] + if s3_ca_secret_name: + pod_spec["volumes"].append(_s3_ca_volume(s3_ca_secret_name)) + container["volumeMounts"].append(_s3_ca_volume_mount()) if config_name: pod_spec["containers"][0]["envFrom"] = [{"configMapRef": {"name": config_name}}] if config_name and secret_name: @@ -731,6 +759,16 @@ def _deployment( {"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration-wait"}, {"name": "GOVOPLAN_DB_POOL_SIZE", "value": "1"}, {"name": "GOVOPLAN_DB_MAX_OVERFLOW", "value": "0"}, + *( + [ + { + "name": "AWS_CA_BUNDLE", + "value": "/etc/govoplan/trust/s3-ca.crt", + } + ] + if s3_ca_secret_name + else [] + ), *_secret_environment(secret_name), ], "securityContext": { @@ -738,7 +776,10 @@ def _deployment( "capabilities": {"drop": ["ALL"]}, "readOnlyRootFilesystem": True, }, - "volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}], + "volumeMounts": [ + {"name": "tmp", "mountPath": "/tmp"}, + *([_s3_ca_volume_mount()] if s3_ca_secret_name else []), + ], } ] return { @@ -769,6 +810,7 @@ def _migration_job( image: str, config_name: str, secret_name: str, + s3_ca_secret_name: str | None, service_account: str, database_environment: Mapping[str, str], backup_required: bool, @@ -832,6 +874,16 @@ def _migration_job( "env": [ {"name": "TMPDIR", "value": "/tmp"}, {"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration"}, + *( + [ + { + "name": "AWS_CA_BUNDLE", + "value": "/etc/govoplan/trust/s3-ca.crt", + } + ] + if s3_ca_secret_name + else [] + ), *( {"name": key, "value": value} for key, value in sorted( @@ -851,10 +903,24 @@ def _migration_job( "capabilities": {"drop": ["ALL"]}, "readOnlyRootFilesystem": True, }, - "volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}], + "volumeMounts": [ + {"name": "tmp", "mountPath": "/tmp"}, + *( + [_s3_ca_volume_mount()] + if s3_ca_secret_name + else [] + ), + ], } ], - "volumes": [{"name": "tmp", "emptyDir": {}}], + "volumes": [ + {"name": "tmp", "emptyDir": {}}, + *( + [_s3_ca_volume(s3_ca_secret_name)] + if s3_ca_secret_name + else [] + ), + ], }, }, }, @@ -873,6 +939,24 @@ def _secret_environment(secret_name: str) -> list[dict[str, Any]]: ] +def _s3_ca_volume(secret_name: str) -> dict[str, Any]: + return { + "name": "s3-ca", + "secret": { + "secretName": secret_name, + "items": [{"key": "ca.crt", "path": "s3-ca.crt"}], + }, + } + + +def _s3_ca_volume_mount() -> dict[str, Any]: + return { + "name": "s3-ca", + "mountPath": "/etc/govoplan/trust", + "readOnly": True, + } + + def _service( *, name: str, diff --git a/tools/inventory/extract-webui-structure.mjs b/tools/inventory/extract-webui-structure.mjs index 933828c..1dc2068 100644 --- a/tools/inventory/extract-webui-structure.mjs +++ b/tools/inventory/extract-webui-structure.mjs @@ -521,11 +521,32 @@ function inspectSource(repository, sourceRoot, sourcePath) { ) { return propertyNameText(current.name); } + if ( + ts.isVariableDeclaration(current) && + ts.isIdentifier(current.name) && + (current.name.text === "en" || current.name.text === "de") && + current.initializer && + isCatalogObject(current.initializer) + ) { + return current.name.text; + } current = current.parent; } return null; } + function isCatalogObject(node) { + let current = node; + while ( + ts.isAsExpression(current) || + ts.isSatisfiesExpression(current) || + ts.isParenthesizedExpression(current) + ) { + current = current.expression; + } + return ts.isObjectLiteralExpression(current); + } + function ancestorPropertyName(node, expected) { let current = node.parent; while (current) { diff --git a/tools/inventory/platform-interface-inventory.py b/tools/inventory/platform-interface-inventory.py index 47a28d9..7c655ee 100644 --- a/tools/inventory/platform-interface-inventory.py +++ b/tools/inventory/platform-interface-inventory.py @@ -31,6 +31,8 @@ ENDPOINT_SURFACE_CATEGORIES = { DEFAULT_ENDPOINT_DECLARATIONS = ( META_ROOT / "tools" / "inventory" / "endpoint-surface-declarations.json" ) +REQUIRED_LOCALES = ("de", "en") +REFERENCE_LOCALE = "de" def main() -> int: @@ -353,6 +355,14 @@ def _extract_manifests( } for permission in manifest.permissions ], + "architecture": ( + manifest.architecture.to_dict() + if manifest.architecture is not None + else None + ), + "information_governance": ( + manifest.information_governance.to_dict() + ), "interface_catalog": manifest_interface_catalog(manifest), "frontend": ( { @@ -447,20 +457,27 @@ def _assemble_inventory( usages = {item["key"] for item in webui["translationUsages"]} catalogs = webui["translationCatalog"] catalog_keys = {locale: set(entries) for locale, entries in catalogs.items()} - expected_locales = sorted(catalog_keys) + expected_locales = sorted(set(catalog_keys) | set(REQUIRED_LOCALES)) missing_catalog_entries = [ { "key": key, "missing_locales": [ - locale for locale in expected_locales if key not in catalog_keys[locale] + locale + for locale in expected_locales + if key not in catalog_keys.get(locale, set()) ], } for key in sorted(usages) - if any(key not in catalog_keys[locale] for locale in expected_locales) + if any(key not in catalog_keys.get(locale, set()) for locale in expected_locales) ] fields = webui["fields"] help_candidates = [field for field in fields if field["helpCandidate"]] dynamic_help = [field for field in fields if field.get("helpDynamic")] + governance_adoption = Counter( + dimension["adoption"] + for manifest in manifests + for dimension in manifest["information_governance"]["dimensions"].values() + ) source_declarations = _source_interface_declarations(webui, manifests) declaration_health = _declaration_health(source_declarations, manifests) runtime_comparison = ( @@ -502,9 +519,26 @@ def _assemble_inventory( }, "translation_health": { "locales": expected_locales, + "reference_locale": REFERENCE_LOCALE, + "reference_locale_entries": len(catalog_keys.get(REFERENCE_LOCALE, set())), + "reference_locale_complete": not any( + REFERENCE_LOCALE in item["missing_locales"] + for item in missing_catalog_entries + ), "used_keys": len(usages), "missing_catalog_entries": missing_catalog_entries, }, + "information_governance_health": { + "dimensions": len(manifests) * 4, + "adoption_counts": dict(sorted(governance_adoption.items())), + "modules": [ + { + "module_id": manifest["id"], + "dimensions": manifest["information_governance"]["dimensions"], + } + for manifest in manifests + ], + }, "api": { "backend_endpoints": classified_endpoints, "frontend_references": frontend_refs, @@ -517,6 +551,7 @@ def _assemble_inventory( "modules": len(manifests), "ui_fields": len(fields), "ui_fields_with_static_help": len(fields) - len(help_candidates), + "ui_fields_with_resolvable_f1_context": len(fields), "help_review_candidates": len(help_candidates), "dynamic_help_references": len(dynamic_help), "ui_actions": len(webui.get("actions", [])), @@ -536,6 +571,12 @@ def _assemble_inventory( "backend_endpoints_without_static_webui_reference": len(unreferenced), "unclassified_backend_endpoints": len(unclassified), "stale_endpoint_declarations": len(stale_declarations), + "information_governance_dimensions": len(manifests) * 4, + "information_governance_enforced": governance_adoption["enforced"], + "information_governance_partial": governance_adoption["partial"], + "information_governance_contract_only": governance_adoption[ + "contract_only" + ], }, } @@ -856,6 +897,7 @@ def _render_markdown(inventory: dict[str, Any]) -> str: f"- UI fields: {summary['ui_fields']}", f"- UI actions: {summary['ui_actions']}", f"- Fields with statically associated help: {summary['ui_fields_with_static_help']}", + f"- Fields with a resolvable F1 context: {summary['ui_fields_with_resolvable_f1_context']}", f"- Fields with dynamic help references: {summary['dynamic_help_references']}", f"- Help review candidates: {summary['help_review_candidates']}", f"- Stable interface declarations: {summary['interface_declarations']}", @@ -872,7 +914,12 @@ def _render_markdown(inventory: dict[str, Any]) -> str: ), f"- Unclassified backend endpoints: {summary['unclassified_backend_endpoints']}", f"- Stale endpoint declarations: {summary['stale_endpoint_declarations']}", - f"- Used translation keys missing from a locale catalog: {len(missing)}", + f"- Reference locale: `{inventory['translation_health']['reference_locale']}`", + f"- Reference locale complete: `{str(inventory['translation_health']['reference_locale_complete']).lower()}`", + f"- Used translation keys missing from a required locale catalog: {len(missing)}", + f"- Information-governance dimensions enforced: {summary['information_governance_enforced']}", + f"- Information-governance dimensions partial: {summary['information_governance_partial']}", + f"- Information-governance dimensions contract-only: {summary['information_governance_contract_only']}", "", "## Help Review Candidates", "", diff --git a/tools/lab/govoplan-lab.acceptance.example.toml b/tools/lab/govoplan-lab.acceptance.example.toml new file mode 100644 index 0000000..243fa09 --- /dev/null +++ b/tools/lab/govoplan-lab.acceptance.example.toml @@ -0,0 +1,85 @@ +schema_version = 1 +name = "govoplan-k8s-acceptance" +mode = "acceptance" +state_directory = "~/.local/share/govoplan/labs/govoplan-k8s-acceptance" +vm_image_directory = "/var/lib/libvirt/images/govoplan-labs" +ssh_user = "govoplan" +ssh_private_key = "~/.ssh/govoplan-lab" +ssh_public_key = "~/.ssh/govoplan-lab.pub" +namespace = "govoplan" +public_host = "govoplan.acceptance.example.org" +s3_host = "s3.govoplan.acceptance.example.org" +ingress_class = "traefik" +module_set = "base" +api_replicas = 2 +web_replicas = 2 +worker_replicas = 2 +db_connection_limit = 100 + +[network] +prefix_length = 24 +gateway = "10.77.10.1" +dns_servers = ["10.77.10.1", "1.1.1.1"] +bridge = "br0" + +[image] +url = "https://cloud-images.ubuntu.com/releases/noble/release-20260801/ubuntu-24.04-server-cloudimg-amd64.img" +sha256 = "0533b0655c32e68b31d792ecd6ccfca95abdbc536c4446874fe0513bd4140ffe" + +[k3s] +version = "v1.36.1+k3s1" +binary_url = "https://github.com/k3s-io/k3s/releases/download/v1.36.1%2Bk3s1/k3s" +binary_sha256 = "a443db3fe9820cd93617ae67e4386d87c1514c1e96ceb30f4c2791c39065653c" +install_script_url = "https://raw.githubusercontent.com/k3s-io/k3s/v1.36.1%2Bk3s1/install.sh" +install_script_sha256 = "46177d4c99440b4c0311b67233823a8e8a2fc09693f6c89af1a7161e152fbfad" + +[release] +manifest_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-manifest.json" +manifest_sha256 = "09ac1ade6ede4958bab0dfb7fd8f99246f4d991846308db1f410b25b46267840" +keyring_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-keyring.json" +keyring_sha256 = "92a9f8e3bac0ef525ad9a063c76faa36233a9070cd4a5db9ee9b3f92323b602f" +channel = "stable" + +[[nodes]] +name = "control-1" +role = "control" +address = "10.77.10.21" +hypervisor = "lab-admin@hypervisor-state.example.org" +failure_domain = "rack-c" +mac_address = "52:54:00:68:01:01" +cpus = 2 +memory_mib = 4096 +disk_gib = 32 + +[[nodes]] +name = "worker-1" +role = "worker" +address = "10.77.10.22" +hypervisor = "lab-admin@hypervisor-a.example.org" +failure_domain = "rack-a" +mac_address = "52:54:00:68:01:02" +cpus = 2 +memory_mib = 4096 +disk_gib = 40 + +[[nodes]] +name = "worker-2" +role = "worker" +address = "10.77.10.23" +hypervisor = "lab-admin@hypervisor-b.example.org" +failure_domain = "rack-b" +mac_address = "52:54:00:68:01:03" +cpus = 2 +memory_mib = 4096 +disk_gib = 40 + +[[nodes]] +name = "state-1" +role = "state" +address = "10.77.10.24" +hypervisor = "lab-admin@hypervisor-state.example.org" +failure_domain = "rack-c" +mac_address = "52:54:00:68:01:04" +cpus = 4 +memory_mib = 8192 +disk_gib = 120 diff --git a/tools/lab/govoplan-lab.example.toml b/tools/lab/govoplan-lab.example.toml new file mode 100644 index 0000000..4247ca6 --- /dev/null +++ b/tools/lab/govoplan-lab.example.toml @@ -0,0 +1,85 @@ +schema_version = 1 +name = "govoplan-k8s-lab" +mode = "rehearsal" +state_directory = "~/.local/share/govoplan/labs/govoplan-k8s-lab" +vm_image_directory = "/var/lib/libvirt/images/govoplan-labs" +ssh_user = "govoplan" +ssh_private_key = "~/.ssh/govoplan-lab" +ssh_public_key = "~/.ssh/govoplan-lab.pub" +namespace = "govoplan" +public_host = "govoplan.lab.test" +s3_host = "s3.govoplan.lab.test" +ingress_class = "traefik" +module_set = "base" +api_replicas = 2 +web_replicas = 2 +worker_replicas = 2 +db_connection_limit = 100 + +[network] +prefix_length = 24 +gateway = "192.168.123.1" +dns_servers = ["192.168.123.1", "1.1.1.1"] +bridge = "virbr-gplab" + +[image] +url = "https://cloud-images.ubuntu.com/releases/noble/release-20260801/ubuntu-24.04-server-cloudimg-amd64.img" +sha256 = "0533b0655c32e68b31d792ecd6ccfca95abdbc536c4446874fe0513bd4140ffe" + +[k3s] +version = "v1.36.1+k3s1" +binary_url = "https://github.com/k3s-io/k3s/releases/download/v1.36.1%2Bk3s1/k3s" +binary_sha256 = "a443db3fe9820cd93617ae67e4386d87c1514c1e96ceb30f4c2791c39065653c" +install_script_url = "https://raw.githubusercontent.com/k3s-io/k3s/v1.36.1%2Bk3s1/install.sh" +install_script_sha256 = "46177d4c99440b4c0311b67233823a8e8a2fc09693f6c89af1a7161e152fbfad" + +[release] +manifest_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-manifest.json" +manifest_sha256 = "09ac1ade6ede4958bab0dfb7fd8f99246f4d991846308db1f410b25b46267840" +keyring_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-keyring.json" +keyring_sha256 = "92a9f8e3bac0ef525ad9a063c76faa36233a9070cd4a5db9ee9b3f92323b602f" +channel = "stable" + +[[nodes]] +name = "control-1" +role = "control" +address = "192.168.123.201" +hypervisor = "local" +failure_domain = "local-host" +mac_address = "52:54:00:67:01:01" +cpus = 2 +memory_mib = 4096 +disk_gib = 32 + +[[nodes]] +name = "worker-1" +role = "worker" +address = "192.168.123.202" +hypervisor = "local" +failure_domain = "local-host" +mac_address = "52:54:00:67:01:02" +cpus = 2 +memory_mib = 4096 +disk_gib = 40 + +[[nodes]] +name = "worker-2" +role = "worker" +address = "192.168.123.203" +hypervisor = "local" +failure_domain = "local-host" +mac_address = "52:54:00:67:01:03" +cpus = 2 +memory_mib = 4096 +disk_gib = 40 + +[[nodes]] +name = "state-1" +role = "state" +address = "192.168.123.204" +hypervisor = "local" +failure_domain = "local-host" +mac_address = "52:54:00:67:01:04" +cpus = 4 +memory_mib = 4096 +disk_gib = 80 diff --git a/tools/lab/govoplan-lab.py b/tools/lab/govoplan-lab.py new file mode 100644 index 0000000..38b9af0 --- /dev/null +++ b/tools/lab/govoplan-lab.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 +"""Executable entry point for the GovOPlaN Kubernetes VM lab.""" + +from __future__ import annotations + +from pathlib import Path +import sys + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from govoplan_lab.cli import main + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/lab/govoplan_lab/__init__.py b/tools/lab/govoplan_lab/__init__.py new file mode 100644 index 0000000..3669102 --- /dev/null +++ b/tools/lab/govoplan_lab/__init__.py @@ -0,0 +1,5 @@ +"""Reproducible GovOPlaN Kubernetes acceptance lab.""" + +from .config import LabConfig, LabConfigError, LabNode, load_config + +__all__ = ["LabConfig", "LabConfigError", "LabNode", "load_config"] diff --git a/tools/lab/govoplan_lab/cli.py b/tools/lab/govoplan_lab/cli.py new file mode 100644 index 0000000..00df1a5 --- /dev/null +++ b/tools/lab/govoplan_lab/cli.py @@ -0,0 +1,153 @@ +"""Command-line interface for the GovOPlaN Kubernetes VM lab.""" + +from __future__ import annotations + +import argparse +from pathlib import Path +from typing import Sequence + +from .config import LabConfigError, load_config +from .lifecycle import ( + LabOperationError, + create, + deploy, + destroy, + doctor, + enroll_admin, + pause, + resume, + status, + update, + verify, +) + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="govoplan-lab", + description="Create and operate a libvirt-backed GovOPlaN Kubernetes test lab.", + ) + parser.add_argument( + "--config", + type=Path, + default=Path("govoplan-lab.toml"), + help="Strict TOML lab inventory (default: ./govoplan-lab.toml).", + ) + parser.add_argument("--verbose", action="store_true") + subparsers = parser.add_subparsers(dest="command", required=True) + + doctor_parser = subparsers.add_parser("doctor", help="Validate inventory and prerequisites.") + doctor_parser.add_argument( + "--online", + action="store_true", + help="Also connect to every hypervisor and verify its toolchain.", + ) + + subparsers.add_parser("status", help="Show VM, Kubernetes node, and pod state.") + _mutation_parser(subparsers, "create", "Create or reuse all declared VMs.") + _mutation_parser(subparsers, "deploy", "Deploy shared state, K3s, and GovOPlaN.") + _mutation_parser(subparsers, "update", "Reconcile pinned K3s and GovOPlaN inputs serially.") + _mutation_parser(subparsers, "pause", "Gracefully stop the lab while preserving disks.") + _mutation_parser(subparsers, "resume", "Start a paused lab in dependency order.") + + destroy_parser = _mutation_parser( + subparsers, + "destroy", + "Destroy lab-owned VMs and disks with an explicit name confirmation.", + ) + destroy_parser.add_argument("--confirm", default="") + destroy_parser.add_argument( + "--purge-local-state", + action="store_true", + help="Also delete local secrets, manifests, and evidence after VM teardown.", + ) + + verify_parser = subparsers.add_parser( + "verify", + help=( + "Collect sanitized live-cluster evidence using an " + "Ops-read-authorized GOVOPLAN_OPS_API_KEY." + ), + ) + verify_parser.add_argument( + "--exercise-api-pod-loss", + action="store_true", + help="Delete one ready API pod during the bounded availability drill.", + ) + enroll_parser = _mutation_parser( + subparsers, + "enroll-admin", + "Securely consume the first-administrator enrollment artifact.", + ) + enroll_parser.add_argument("--email", required=True) + enroll_parser.add_argument("--display-name", default=None) + enroll_parser.add_argument("--tenant-slug", default="default") + enroll_parser.add_argument("--tenant-name", default="Default Tenant") + return parser + + +def _mutation_parser( + subparsers: argparse._SubParsersAction[argparse.ArgumentParser], + name: str, + help_text: str, +) -> argparse.ArgumentParser: + parser = subparsers.add_parser(name, help=help_text) + parser.add_argument( + "--apply", + action="store_true", + help="Perform mutations; without this flag the command is a dry run.", + ) + return parser + + +def main(argv: Sequence[str] | None = None) -> int: + args = build_parser().parse_args(argv) + try: + config = load_config(args.config) + if args.command == "doctor": + return doctor(config, online=args.online, verbose=args.verbose) + if args.command == "status": + return status(config, verbose=args.verbose) + if args.command == "create": + create(config, apply=args.apply, verbose=args.verbose) + elif args.command == "deploy": + deploy(config, apply=args.apply, verbose=args.verbose) + elif args.command == "update": + update(config, apply=args.apply, verbose=args.verbose) + elif args.command == "pause": + pause(config, apply=args.apply, verbose=args.verbose) + elif args.command == "resume": + resume(config, apply=args.apply, verbose=args.verbose) + elif args.command == "destroy": + destroy( + config, + apply=args.apply, + confirmation=args.confirm, + purge_local_state=args.purge_local_state, + verbose=args.verbose, + ) + elif args.command == "verify": + verify( + config, + exercise_api_pod_loss=args.exercise_api_pod_loss, + verbose=args.verbose, + ) + elif args.command == "enroll-admin": + enroll_admin( + config, + email=args.email, + display_name=args.display_name, + tenant_slug=args.tenant_slug, + tenant_name=args.tenant_name, + apply=args.apply, + ) + else: + raise RuntimeError(f"unsupported command: {args.command}") + return 0 + except (LabConfigError, LabOperationError, OSError, ValueError) as exc: + print(f"error: {exc}", file=__import__("sys").stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/lab/govoplan_lab/config.py b/tools/lab/govoplan_lab/config.py new file mode 100644 index 0000000..e3b6a86 --- /dev/null +++ b/tools/lab/govoplan_lab/config.py @@ -0,0 +1,466 @@ +"""Strict TOML model for the GovOPlaN Kubernetes VM lab.""" + +from __future__ import annotations + +from dataclasses import dataclass +import ipaddress +from pathlib import Path +import re +import tomllib +from typing import Any, Mapping +from urllib.parse import urlsplit + + +SCHEMA_VERSION = 1 +_NAME = re.compile(r"^[a-z][a-z0-9-]{1,47}$") +_HOSTNAME = re.compile( + r"^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)*" + r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$" +) +_SHA256 = re.compile(r"^[0-9a-f]{64}$") +_K3S_VERSION = re.compile(r"^v[0-9]+\.[0-9]+\.[0-9]+\+k3s[0-9]+$") +_SSH_TARGET = re.compile(r"^(?:[A-Za-z0-9_.-]+@)?[A-Za-z0-9_.:-]+$") +_BRIDGE = re.compile(r"^[A-Za-z0-9_.:-]{1,32}$") +_ROLE = {"control", "worker", "state"} +_MODE = {"rehearsal", "acceptance"} + + +class LabConfigError(ValueError): + """Raised when the lab inventory cannot be used safely.""" + + +@dataclass(frozen=True, slots=True) +class LabNode: + name: str + role: str + address: ipaddress.IPv4Address + hypervisor: str + failure_domain: str + mac_address: str + cpus: int + memory_mib: int + disk_gib: int + + @property + def is_local(self) -> bool: + return self.hypervisor == "local" + + +@dataclass(frozen=True, slots=True) +class NetworkConfig: + prefix_length: int + gateway: ipaddress.IPv4Address + dns_servers: tuple[ipaddress.IPv4Address, ...] + bridge: str + + +@dataclass(frozen=True, slots=True) +class ImageConfig: + url: str + sha256: str + + +@dataclass(frozen=True, slots=True) +class K3sConfig: + version: str + binary_url: str + binary_sha256: str + install_script_url: str + install_script_sha256: str + + +@dataclass(frozen=True, slots=True) +class ReleaseConfig: + manifest_url: str + manifest_sha256: str + keyring_url: str + keyring_sha256: str + channel: str + + +@dataclass(frozen=True, slots=True) +class LabConfig: + source: Path + schema_version: int + name: str + mode: str + state_directory: Path + vm_image_directory: str + ssh_user: str + ssh_private_key: Path + ssh_public_key: Path + namespace: str + public_host: str + s3_host: str + ingress_class: str + module_set: str + api_replicas: int + web_replicas: int + worker_replicas: int + db_connection_limit: int + network: NetworkConfig + image: ImageConfig + k3s: K3sConfig + release: ReleaseConfig + nodes: tuple[LabNode, ...] + + @property + def controls(self) -> tuple[LabNode, ...]: + return tuple(node for node in self.nodes if node.role == "control") + + @property + def workers(self) -> tuple[LabNode, ...]: + return tuple(node for node in self.nodes if node.role == "worker") + + @property + def state_node(self) -> LabNode: + return next(node for node in self.nodes if node.role == "state") + + @property + def primary_control(self) -> LabNode: + return self.controls[0] + + @property + def public_url(self) -> str: + return f"https://{self.public_host}" + + @property + def s3_url(self) -> str: + return f"https://{self.s3_host}:9443" + + @property + def evidence_capable(self) -> bool: + worker_domains = {node.failure_domain for node in self.workers} + worker_hypervisors = {node.hypervisor for node in self.workers} + return ( + self.mode == "acceptance" + and len(worker_domains) == len(self.workers) + and len(worker_hypervisors) == len(self.workers) + and self.state_node.failure_domain not in worker_domains + and self.state_node.hypervisor not in worker_hypervisors + ) + + +def load_config(path: Path) -> LabConfig: + source = path.expanduser().resolve() + try: + raw = tomllib.loads(source.read_text(encoding="utf-8")) + except FileNotFoundError as exc: + raise LabConfigError(f"lab configuration does not exist: {source}") from exc + except tomllib.TOMLDecodeError as exc: + raise LabConfigError(f"lab configuration is not valid TOML: {exc}") from exc + root = _mapping(raw, "lab configuration") + _only_keys( + root, + { + "schema_version", + "name", + "mode", + "state_directory", + "vm_image_directory", + "ssh_user", + "ssh_private_key", + "ssh_public_key", + "namespace", + "public_host", + "s3_host", + "ingress_class", + "module_set", + "api_replicas", + "web_replicas", + "worker_replicas", + "db_connection_limit", + "network", + "image", + "k3s", + "release", + "nodes", + }, + "lab configuration", + ) + schema_version = _integer(root, "schema_version") + if schema_version != SCHEMA_VERSION: + raise LabConfigError( + f"schema_version must be {SCHEMA_VERSION}; found {schema_version}" + ) + name = _pattern(root, "name", _NAME) + mode = _choice(root, "mode", _MODE) + base = source.parent + state_directory = _path(root, "state_directory", base) + vm_image_directory = _absolute_posix_path(root, "vm_image_directory") + ssh_user = _pattern(root, "ssh_user", re.compile(r"^[a-z_][a-z0-9_-]{0,31}$")) + ssh_private_key = _path(root, "ssh_private_key", base) + ssh_public_key = _path(root, "ssh_public_key", base) + namespace = _pattern(root, "namespace", _NAME) + public_host = _pattern(root, "public_host", _HOSTNAME) + s3_host = _pattern(root, "s3_host", _HOSTNAME) + if public_host == s3_host: + raise LabConfigError("public_host and s3_host must be different") + ingress_class = _pattern(root, "ingress_class", _NAME) + module_set = _choice(root, "module_set", {"core", "base", "full"}) + api_replicas = _bounded_integer(root, "api_replicas", 2, 32) + web_replicas = _bounded_integer(root, "web_replicas", 2, 32) + worker_replicas = _bounded_integer(root, "worker_replicas", 2, 64) + db_connection_limit = _bounded_integer(root, "db_connection_limit", 50, 10000) + network = _parse_network(_mapping(root.get("network"), "network")) + image = _parse_image(_mapping(root.get("image"), "image")) + k3s = _parse_k3s(_mapping(root.get("k3s"), "k3s")) + release = _parse_release(_mapping(root.get("release"), "release")) + raw_nodes = root.get("nodes") + if not isinstance(raw_nodes, list) or not raw_nodes: + raise LabConfigError("nodes must be a non-empty array of tables") + nodes = tuple(_parse_node(item, index=index) for index, item in enumerate(raw_nodes)) + config = LabConfig( + source=source, + schema_version=schema_version, + name=name, + mode=mode, + state_directory=state_directory, + vm_image_directory=vm_image_directory.rstrip("/"), + ssh_user=ssh_user, + ssh_private_key=ssh_private_key, + ssh_public_key=ssh_public_key, + namespace=namespace, + public_host=public_host, + s3_host=s3_host, + ingress_class=ingress_class, + module_set=module_set, + api_replicas=api_replicas, + web_replicas=web_replicas, + worker_replicas=worker_replicas, + db_connection_limit=db_connection_limit, + network=network, + image=image, + k3s=k3s, + release=release, + nodes=nodes, + ) + _validate_topology(config) + return config + + +def _parse_network(raw: Mapping[str, Any]) -> NetworkConfig: + _only_keys(raw, {"prefix_length", "gateway", "dns_servers", "bridge"}, "network") + prefix_length = _bounded_integer(raw, "prefix_length", 8, 30) + gateway = _ipv4(raw, "gateway") + dns_raw = raw.get("dns_servers") + if not isinstance(dns_raw, list) or not dns_raw or len(dns_raw) > 4: + raise LabConfigError("network.dns_servers must contain 1-4 IPv4 addresses") + dns_servers = tuple(_ipv4_value(value, "network.dns_servers") for value in dns_raw) + bridge = _pattern(raw, "bridge", _BRIDGE) + return NetworkConfig(prefix_length, gateway, dns_servers, bridge) + + +def _parse_image(raw: Mapping[str, Any]) -> ImageConfig: + _only_keys(raw, {"url", "sha256"}, "image") + return ImageConfig( + url=_https_url(raw, "url"), + sha256=_pattern(raw, "sha256", _SHA256), + ) + + +def _parse_k3s(raw: Mapping[str, Any]) -> K3sConfig: + _only_keys( + raw, + { + "version", + "binary_url", + "binary_sha256", + "install_script_url", + "install_script_sha256", + }, + "k3s", + ) + return K3sConfig( + version=_pattern(raw, "version", _K3S_VERSION), + binary_url=_https_url(raw, "binary_url"), + binary_sha256=_pattern(raw, "binary_sha256", _SHA256), + install_script_url=_https_url(raw, "install_script_url"), + install_script_sha256=_pattern(raw, "install_script_sha256", _SHA256), + ) + + +def _parse_release(raw: Mapping[str, Any]) -> ReleaseConfig: + _only_keys( + raw, + {"manifest_url", "manifest_sha256", "keyring_url", "keyring_sha256", "channel"}, + "release", + ) + return ReleaseConfig( + manifest_url=_https_url(raw, "manifest_url"), + manifest_sha256=_pattern(raw, "manifest_sha256", _SHA256), + keyring_url=_https_url(raw, "keyring_url"), + keyring_sha256=_pattern(raw, "keyring_sha256", _SHA256), + channel=_pattern(raw, "channel", _NAME), + ) + + +def _parse_node(value: object, *, index: int) -> LabNode: + raw = _mapping(value, f"nodes[{index}]") + _only_keys( + raw, + { + "name", + "role", + "address", + "hypervisor", + "failure_domain", + "mac_address", + "cpus", + "memory_mib", + "disk_gib", + }, + f"nodes[{index}]", + ) + hypervisor = _string(raw, "hypervisor") + if hypervisor != "local" and not _SSH_TARGET.fullmatch(hypervisor): + raise LabConfigError( + f"nodes[{index}].hypervisor must be 'local' or a simple SSH target" + ) + mac_address = _string(raw, "mac_address").lower() + try: + octets = mac_address.split(":") + valid_mac = len(octets) == 6 and all( + len(octet) == 2 and 0 <= int(octet, 16) <= 255 for octet in octets + ) + except ValueError: + valid_mac = False + if not valid_mac: + raise LabConfigError(f"nodes[{index}].mac_address is not a canonical MAC address") + return LabNode( + name=_pattern(raw, "name", _NAME), + role=_choice(raw, "role", _ROLE), + address=_ipv4(raw, "address"), + hypervisor=hypervisor, + failure_domain=_pattern(raw, "failure_domain", _NAME), + mac_address=mac_address, + cpus=_bounded_integer(raw, "cpus", 1, 64), + memory_mib=_bounded_integer(raw, "memory_mib", 2048, 262144), + disk_gib=_bounded_integer(raw, "disk_gib", 16, 4096), + ) + + +def _validate_topology(config: LabConfig) -> None: + names = [node.name for node in config.nodes] + addresses = [node.address for node in config.nodes] + mac_addresses = [node.mac_address for node in config.nodes] + for label, values in ( + ("node names", names), + ("node addresses", addresses), + ("node MAC addresses", mac_addresses), + ): + if len(values) != len(set(values)): + raise LabConfigError(f"{label} must be unique") + too_long = [ + node.name + for node in config.nodes + if len(f"{config.name}-{node.name}") > 63 + ] + if too_long: + raise LabConfigError( + "lab name plus node name must fit the 63-character libvirt domain limit: " + + ", ".join(too_long) + ) + if len(config.controls) not in {1, 3}: + raise LabConfigError("the lab requires exactly one or three control-plane nodes") + if len(config.workers) < 2: + raise LabConfigError("the lab requires at least two worker nodes") + if sum(node.role == "state" for node in config.nodes) != 1: + raise LabConfigError("the lab requires exactly one external shared-state node") + network = ipaddress.ip_network( + f"{config.network.gateway}/{config.network.prefix_length}", strict=False + ) + if any(node.address not in network for node in config.nodes): + raise LabConfigError("every node address must be in the configured IPv4 network") + if config.network.gateway in addresses: + raise LabConfigError("the network gateway cannot also be a node address") + if config.mode == "acceptance" and not config.evidence_capable: + raise LabConfigError( + "acceptance mode requires each worker and the shared-state node to use " + "distinct hypervisors and failure_domain values" + ) + + +def _mapping(value: object, label: str) -> Mapping[str, Any]: + if not isinstance(value, dict): + raise LabConfigError(f"{label} must be a table") + return value + + +def _only_keys(raw: Mapping[str, Any], allowed: set[str], label: str) -> None: + unexpected = sorted(set(raw) - allowed) + if unexpected: + raise LabConfigError(f"{label} contains unsupported keys: {', '.join(unexpected)}") + + +def _string(raw: Mapping[str, Any], key: str) -> str: + value = raw.get(key) + if not isinstance(value, str) or not value.strip(): + raise LabConfigError(f"{key} must be a non-empty string") + return value.strip() + + +def _integer(raw: Mapping[str, Any], key: str) -> int: + value = raw.get(key) + if not isinstance(value, int) or isinstance(value, bool): + raise LabConfigError(f"{key} must be an integer") + return value + + +def _bounded_integer(raw: Mapping[str, Any], key: str, minimum: int, maximum: int) -> int: + value = _integer(raw, key) + if not minimum <= value <= maximum: + raise LabConfigError(f"{key} must be between {minimum} and {maximum}") + return value + + +def _pattern(raw: Mapping[str, Any], key: str, pattern: re.Pattern[str]) -> str: + value = _string(raw, key) + if pattern.fullmatch(value) is None: + raise LabConfigError(f"{key} has an unsupported format") + return value + + +def _choice(raw: Mapping[str, Any], key: str, choices: set[str]) -> str: + value = _string(raw, key) + if value not in choices: + raise LabConfigError(f"{key} must be one of: {', '.join(sorted(choices))}") + return value + + +def _ipv4(raw: Mapping[str, Any], key: str) -> ipaddress.IPv4Address: + return _ipv4_value(_string(raw, key), key) + + +def _ipv4_value(value: object, label: str) -> ipaddress.IPv4Address: + if not isinstance(value, str): + raise LabConfigError(f"{label} must contain strings") + try: + parsed = ipaddress.ip_address(value) + except ValueError as exc: + raise LabConfigError(f"{label} contains an invalid IP address") from exc + if not isinstance(parsed, ipaddress.IPv4Address): + raise LabConfigError(f"{label} supports IPv4 only in schema version 1") + return parsed + + +def _path(raw: Mapping[str, Any], key: str, base: Path) -> Path: + value = Path(_string(raw, key)).expanduser() + return (value if value.is_absolute() else base / value).resolve() + + +def _absolute_posix_path(raw: Mapping[str, Any], key: str) -> str: + value = _string(raw, key) + if not value.startswith("/") or ".." in Path(value).parts: + raise LabConfigError(f"{key} must be an absolute path without '..'") + return value + + +def _https_url(raw: Mapping[str, Any], key: str) -> str: + value = _string(raw, key) + parsed = urlsplit(value) + if parsed.scheme != "https" or not parsed.hostname or parsed.username or parsed.password: + raise LabConfigError(f"{key} must be a credential-free HTTPS URL") + if parsed.fragment: + raise LabConfigError(f"{key} must not contain a fragment") + return value diff --git a/tools/lab/govoplan_lab/lifecycle.py b/tools/lab/govoplan_lab/lifecycle.py new file mode 100644 index 0000000..4419c5f --- /dev/null +++ b/tools/lab/govoplan_lab/lifecycle.py @@ -0,0 +1,1922 @@ +"""Lifecycle operations for the libvirt-backed GovOPlaN Kubernetes lab.""" + +from __future__ import annotations + +from dataclasses import asdict +import getpass +import hashlib +import http.client +import ipaddress +import json +import os +from pathlib import Path +import secrets +import shlex +import shutil +import socket +import ssl +import stat +import subprocess +import sys +import time +from typing import Any, Iterable, Mapping, Sequence +from urllib.parse import quote, urlsplit +from urllib.request import Request, urlopen + +from .config import LabConfig, LabNode +from .render import ( + render_caddyfile, + render_cloud_init, + render_garage_config, + render_hosts, + render_k3s_config, + render_meta_data, + render_network_config, + render_registry_config, + render_secret_manifest, + render_state_compose, + render_state_environment, + render_tls_secret_manifest, + write_private, +) + + +REPOSITORY_ROOT = Path(__file__).resolve().parents[3] +DEPLOYER = REPOSITORY_ROOT / "tools" / "deployment" / "govoplan-deploy.py" +_SSH_TIMEOUT_SECONDS = 600 +_STATE_REMOTE_DIRECTORY = "govoplan-lab-state" +_SYSTEM_LIBVIRT_URI = "qemu:///system" +_SECRET_KEYS = ( + "MASTER_KEY_B64", + "DATABASE_URL", + "GOVOPLAN_DATABASE_URL_PGTOOLS", + "REDIS_URL", + "FILE_STORAGE_S3_ACCESS_KEY_ID", + "FILE_STORAGE_S3_SECRET_ACCESS_KEY", +) + + +class LabOperationError(RuntimeError): + """Raised when a lifecycle operation cannot complete safely.""" + + +class CommandRunner: + def __init__(self, config: LabConfig, *, verbose: bool = False) -> None: + self.config = config + self.verbose = verbose + self.known_hosts = config.state_directory / "ssh_known_hosts" + + def run( + self, + argv: Sequence[str], + *, + input_bytes: bytes | None = None, + capture: bool = False, + check: bool = True, + timeout: float | None = None, + sensitive: bool = False, + env: Mapping[str, str] | None = None, + ) -> subprocess.CompletedProcess[bytes]: + if self.verbose: + rendered = "" if sensitive else shlex.join(argv) + print(f"+ {rendered}", file=sys.stderr) + try: + return subprocess.run( + list(argv), + input=input_bytes, + stdout=subprocess.PIPE if capture else None, + stderr=subprocess.PIPE if capture else None, + check=check, + timeout=timeout, + env=dict(env) if env is not None else None, + ) + except FileNotFoundError as exc: + raise LabOperationError(f"required command is unavailable: {argv[0]}") from exc + except subprocess.CalledProcessError as exc: + stderr = (exc.stderr or b"").decode(errors="replace").strip() + detail = f": {stderr}" if stderr else "" + raise LabOperationError(f"command failed ({argv[0]}){detail}") from exc + except subprocess.TimeoutExpired as exc: + raise LabOperationError(f"command timed out ({argv[0]})") from exc + + def ssh_arguments(self, target: str) -> list[str]: + return [ + "ssh", + "-i", + str(self.config.ssh_private_key), + "-o", + "IdentitiesOnly=yes", + "-o", + "BatchMode=yes", + "-o", + "ConnectTimeout=10", + "-o", + f"UserKnownHostsFile={self.known_hosts}", + "-o", + "StrictHostKeyChecking=accept-new", + target, + ] + + def vm( + self, + node: LabNode, + argv: Sequence[str], + *, + sudo: bool = False, + input_bytes: bytes | None = None, + capture: bool = False, + check: bool = True, + timeout: float | None = None, + sensitive: bool = False, + ) -> subprocess.CompletedProcess[bytes]: + target = f"{self.config.ssh_user}@{node.address}" + command = [*(("sudo", "-n", "--") if sudo else ()), *argv] + return self.run( + [*self.ssh_arguments(target), shlex.join(command)], + input_bytes=input_bytes, + capture=capture, + check=check, + timeout=timeout, + sensitive=sensitive, + ) + + def hypervisor( + self, + node: LabNode, + argv: Sequence[str], + *, + capture: bool = False, + check: bool = True, + timeout: float | None = None, + ) -> subprocess.CompletedProcess[bytes]: + if node.is_local: + return self.run( + _local_hypervisor_command(argv), + capture=capture, + check=check, + timeout=timeout, + ) + command = ["sudo", "-n", "--", *argv] + return self.run( + [*self.ssh_arguments(node.hypervisor), shlex.join(command)], + capture=capture, + check=check, + timeout=timeout, + ) + + def copy_to_vm(self, node: LabNode, source: Path, destination: str) -> None: + target = f"{self.config.ssh_user}@{node.address}:{destination}" + self.run(["scp", *self.ssh_arguments("")[1:-1], str(source), target]) + + def copy_to_hypervisor(self, node: LabNode, source: Path, destination: str) -> None: + if node.is_local: + self.run(["install", "-m", "0600", str(source), destination]) + return + upload = f"/tmp/{self.config.name}-{node.name}-{source.name}.upload" + target = f"{node.hypervisor}:{upload}" + self.run(["scp", *self.ssh_arguments("")[1:-1], str(source), target]) + self.hypervisor( + node, + ["install", "-m", "0600", upload, destination], + ) + self.hypervisor(node, ["rm", "-f", upload]) + + +def _local_hypervisor_command(argv: Sequence[str]) -> list[str]: + command = list(argv) + if command and command[0] in {"virsh", "virt-install"}: + return [command[0], "--connect", _SYSTEM_LIBVIRT_URI, *command[1:]] + return command + + +def doctor(config: LabConfig, *, online: bool = False, verbose: bool = False) -> int: + checks: list[tuple[str, bool, str]] = [] + for command in ("ssh", "scp", "ssh-keygen", "openssl", "curl"): + checks.append((f"local.{command}", shutil.which(command) is not None, command)) + checks.extend( + [ + ( + "ssh.private_key", + config.ssh_private_key.is_file(), + str(config.ssh_private_key), + ), + ( + "ssh.public_key", + config.ssh_public_key.is_file(), + str(config.ssh_public_key), + ), + ( + "topology.workers", + len(config.workers) >= 2, + f"{len(config.workers)} worker(s)", + ), + ( + "topology.state", + len([node for node in config.nodes if node.role == "state"]) == 1, + str(config.state_node.address), + ), + ( + "topology.evidence", + config.mode == "rehearsal" or config.evidence_capable, + "evidence-capable" if config.evidence_capable else "same-host rehearsal", + ), + ] + ) + if config.ssh_private_key.exists(): + mode = stat.S_IMODE(config.ssh_private_key.stat().st_mode) + checks.append(("ssh.private_key_mode", mode & 0o077 == 0, oct(mode))) + if online: + runner = CommandRunner(config, verbose=verbose) + for hypervisor, node in _representative_hypervisors(config): + result = runner.hypervisor( + node, + [ + "sh", + "-c", + "command -v virsh && command -v virt-install && " + "command -v qemu-img && command -v cloud-localds && command -v curl", + ], + capture=True, + check=False, + timeout=30, + ) + checks.append( + ( + f"hypervisor.{hypervisor}", + result.returncode == 0, + (result.stderr or result.stdout or b"").decode(errors="replace").strip(), + ) + ) + storage_exists = runner.hypervisor( + node, + ["test", "-d", config.vm_image_directory], + capture=True, + check=False, + timeout=30, + ) + storage_writable = runner.hypervisor( + node, + ["test", "-w", config.vm_image_directory], + capture=True, + check=False, + timeout=30, + ) + checks.append( + ( + f"hypervisor.{hypervisor}.vm_image_directory", + storage_exists.returncode == 0 and storage_writable.returncode == 0, + config.vm_image_directory, + ) + ) + bridge = runner.hypervisor( + node, + ["test", "-d", f"/sys/class/net/{config.network.bridge}"], + capture=True, + check=False, + timeout=30, + ) + checks.append( + ( + f"hypervisor.{hypervisor}.bridge", + bridge.returncode == 0, + config.network.bridge, + ) + ) + for check_id, passed, detail in checks: + print(f"{'OK' if passed else 'FAIL'} {check_id}: {detail}") + failed = [check for check in checks if not check[1]] + if not failed: + print( + "Lab configuration is valid for " + + ("target evidence." if config.evidence_capable else "a same-host rehearsal.") + ) + return 1 if failed else 0 + + +def create(config: LabConfig, *, apply: bool, verbose: bool = False) -> None: + if not apply: + _print_mutation_preview( + "create", + config, + [f"create or reuse VM {node.name} on {node.hypervisor}" for node in config.nodes], + ) + return + _prepare_local_state(config) + runner = CommandRunner(config, verbose=verbose) + public_key = config.ssh_public_key.read_text(encoding="utf-8").strip() + if len(public_key.splitlines()) != 1 or not public_key.startswith( + ("ssh-ed25519 ", "ssh-rsa ", "ecdsa-sha2-", "sk-ssh-ed25519@openssh.com ") + ): + raise LabOperationError("ssh_public_key is not an OpenSSH public key") + for node in config.nodes: + _create_node(config, runner, node, public_key) + for node in config.nodes: + _wait_for_ssh(config, runner, node) + runner.vm( + node, + ["cloud-init", "status", "--wait"], + sudo=True, + timeout=_SSH_TIMEOUT_SECONDS, + ) + _write_inventory(config) + print(f"Created {len(config.nodes)} VM(s) for lab {config.name}.") + + +def _create_node( + config: LabConfig, + runner: CommandRunner, + node: LabNode, + public_key: str, +) -> None: + domain = _domain_name(config, node) + existing = runner.hypervisor( + node, + ["virsh", "dominfo", domain], + capture=True, + check=False, + timeout=20, + ) + if existing.returncode == 0: + _assert_domain_owned(config, runner, node) + print(f"Reusing existing domain {domain} on {node.hypervisor}.") + runner.hypervisor(node, ["virsh", "start", domain], check=False) + return + _forget_vm_host_key(config, runner, node) + rendered = config.state_directory / "rendered" / node.name + rendered.mkdir(parents=True, exist_ok=True, mode=0o700) + write_private(rendered / "user-data", render_cloud_init(config, node, public_key)) + write_private(rendered / "meta-data", render_meta_data(config, node)) + write_private(rendered / "network-config", render_network_config(config, node)) + base_directory = f"{config.vm_image_directory}/base" + node_directory = f"{config.vm_image_directory}/{config.name}/{node.name}" + base_image = f"{base_directory}/{config.image.sha256}.qcow2" + disk_image = f"{node_directory}/root.qcow2" + seed_image = f"{node_directory}/seed.img" + runner.hypervisor(node, ["install", "-d", "-m", "0750", base_directory, node_directory]) + current = runner.hypervisor( + node, + ["sha256sum", base_image], + capture=True, + check=False, + timeout=120, + ) + current_digest = ( + current.stdout.decode(errors="replace").split(maxsplit=1)[0] + if current.returncode == 0 and current.stdout + else "" + ) + if current_digest != config.image.sha256: + temporary = f"{base_image}.download" + runner.hypervisor( + node, + [ + "curl", + "--proto", + "=https", + "--tlsv1.2", + "--fail", + "--location", + "--output", + temporary, + config.image.url, + ], + timeout=1800, + ) + downloaded = runner.hypervisor( + node, + ["sha256sum", temporary], + capture=True, + timeout=1800, + ) + digest = downloaded.stdout.decode().split(maxsplit=1)[0] + if digest != config.image.sha256: + runner.hypervisor(node, ["rm", "-f", temporary], check=False) + raise LabOperationError(f"cloud image digest mismatch on {node.hypervisor}") + runner.hypervisor(node, ["mv", temporary, base_image]) + runner.hypervisor( + node, + [ + "qemu-img", + "create", + "-f", + "qcow2", + "-F", + "qcow2", + "-b", + base_image, + disk_image, + f"{node.disk_gib}G", + ], + ) + for name in ("user-data", "meta-data", "network-config"): + runner.copy_to_hypervisor(node, rendered / name, f"{node_directory}/{name}") + runner.hypervisor( + node, + [ + "cloud-localds", + "--network-config", + f"{node_directory}/network-config", + seed_image, + f"{node_directory}/user-data", + f"{node_directory}/meta-data", + ], + ) + runner.hypervisor( + node, + [ + "virt-install", + "--name", + domain, + "--description", + _domain_description(config, node), + "--memory", + str(node.memory_mib), + "--vcpus", + str(node.cpus), + "--cpu", + "host-passthrough", + "--os-variant", + "ubuntu24.04", + "--disk", + f"path={disk_image},format=qcow2,bus=virtio", + "--disk", + f"path={seed_image},device=cdrom", + "--network", + f"bridge={config.network.bridge},model=virtio,mac={node.mac_address}", + "--graphics", + "none", + "--console", + "pty,target_type=serial", + "--import", + "--noautoconsole", + ], + timeout=120, + ) + runner.hypervisor(node, ["virsh", "autostart", "--disable", domain], check=False) + + +def _wait_for_ssh(config: LabConfig, runner: CommandRunner, node: LabNode) -> None: + deadline = time.monotonic() + _SSH_TIMEOUT_SECONDS + last_error = "" + while time.monotonic() < deadline: + try: + with socket.create_connection((str(node.address), 22), timeout=3): + pass + result = runner.vm(node, ["true"], capture=True, check=False, timeout=15) + if result.returncode == 0: + return + last_error = (result.stderr or b"").decode(errors="replace").strip() + except OSError as exc: + last_error = str(exc) + time.sleep(5) + raise LabOperationError(f"SSH did not become ready on {node.name}: {last_error}") + + +def _prepare_local_state(config: LabConfig) -> None: + config.state_directory.mkdir(parents=True, exist_ok=True, mode=0o700) + config.state_directory.chmod(0o700) + (config.state_directory / "ssh_known_hosts").touch(mode=0o600, exist_ok=True) + (config.state_directory / "ssh_known_hosts").chmod(0o600) + + +def _write_inventory(config: LabConfig) -> None: + payload = { + "schema_version": 1, + "name": config.name, + "mode": config.mode, + "evidence_capable": config.evidence_capable, + "nodes": [ + { + **asdict(node), + "address": str(node.address), + } + for node in config.nodes + ], + } + write_private( + config.state_directory / "inventory.json", + json.dumps(payload, indent=2, sort_keys=True) + "\n", + ) + + +def _representative_hypervisors( + config: LabConfig, +) -> Iterable[tuple[str, LabNode]]: + seen: set[str] = set() + for node in config.nodes: + if node.hypervisor in seen: + continue + seen.add(node.hypervisor) + yield node.hypervisor, node + + +def _print_mutation_preview(operation: str, config: LabConfig, actions: Sequence[str]) -> None: + print(f"Dry run: {operation} lab {config.name} ({config.mode})") + for action in actions: + print(f"- {action}") + print("Re-run with --apply to perform these mutations.") + + +def _domain_name(config: LabConfig, node: LabNode) -> str: + return f"{config.name}-{node.name}" + + +def _domain_description(config: LabConfig, node: LabNode) -> str: + return ( + f"GovOPlaN managed lab; lab={config.name}; node={node.name}; " + f"schema={config.schema_version}" + ) + + +def _assert_domain_owned( + config: LabConfig, + runner: CommandRunner, + node: LabNode, +) -> None: + domain = _domain_name(config, node) + description = runner.hypervisor( + node, + ["virsh", "desc", domain], + capture=True, + check=False, + timeout=20, + ) + if ( + description.returncode != 0 + or description.stdout.decode(errors="replace").strip() + != _domain_description(config, node) + ): + raise LabOperationError( + f"domain {domain} does not carry the expected GovOPlaN lab ownership marker" + ) + block_devices = runner.hypervisor( + node, + ["virsh", "domblklist", "--details", domain], + capture=True, + timeout=20, + ).stdout.decode(errors="replace") + node_directory = f"{config.vm_image_directory}/{config.name}/{node.name}" + expected_paths = { + f"{node_directory}/root.qcow2", + f"{node_directory}/seed.img", + } + missing = sorted(path for path in expected_paths if path not in block_devices) + if missing: + raise LabOperationError( + f"domain {domain} does not use the expected lab-owned disks: " + + ", ".join(missing) + ) + + +def deploy(config: LabConfig, *, apply: bool, verbose: bool = False) -> None: + actions = [ + "verify and adopt the signed GovOPlaN release", + f"start PostgreSQL, Redis, Garage and test mail on {config.state_node.name}", + f"install pinned K3s {config.k3s.version} on control and worker nodes", + "apply runtime, TLS and CA secrets without printing them", + "render and apply the supported stateless Kubernetes profile", + ] + if not apply: + _print_mutation_preview("deploy", config, actions) + return + _prepare_local_state(config) + runner = CommandRunner(config, verbose=verbose) + secrets_value = _load_or_create_secrets(config) + manifest, installation = _prepare_release(config, runner, secrets_value) + _ensure_certificates(config, runner) + _deploy_state_services(config, runner, manifest, secrets_value) + _deploy_k3s(config, runner, secrets_value, update=False) + _deploy_application(config, runner, installation) + print(f"GovOPlaN lab deployment is ready at {config.public_url}.") + print(f"Trust the private lab CA at {config.state_directory / 'pki' / 'ca.crt'}.") + print(f"Host mappings are recorded in {config.state_directory / 'hosts'}.") + + +def update(config: LabConfig, *, apply: bool, verbose: bool = False) -> None: + actions = [ + f"serially reconcile K3s to {config.k3s.version}", + "pull the newly pinned shared-state images", + "verify/adopt the configured GovOPlaN release and apply its migration/runtime manifest", + ] + if not apply: + _print_mutation_preview("update", config, actions) + return + _prepare_local_state(config) + runner = CommandRunner(config, verbose=verbose) + secrets_value = _load_or_create_secrets(config) + manifest, installation = _prepare_release(config, runner, secrets_value) + _ensure_certificates(config, runner) + _deploy_state_services(config, runner, manifest, secrets_value) + _deploy_k3s(config, runner, secrets_value, update=True) + _deploy_application(config, runner, installation) + print(f"Updated lab {config.name} to the configured immutable inputs.") + + +def _prepare_release( + config: LabConfig, + runner: CommandRunner, + lab_secrets: Mapping[str, str], +) -> tuple[dict[str, Any], Path]: + release_directory = config.state_directory / "release" + release_directory.mkdir(parents=True, exist_ok=True, mode=0o700) + manifest_path = release_directory / "distribution-manifest.json" + keyring_path = release_directory / "distribution-keyring.json" + _download_verified( + config.release.manifest_url, + config.release.manifest_sha256, + manifest_path, + maximum_bytes=4 * 1024 * 1024, + ) + _download_verified( + config.release.keyring_url, + config.release.keyring_sha256, + keyring_path, + maximum_bytes=256 * 1024, + ) + try: + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError) as exc: + raise LabOperationError("downloaded release manifest is not valid JSON") from exc + if not isinstance(manifest, dict): + raise LabOperationError("downloaded release manifest must be a JSON object") + installation = config.state_directory / "installation" + if not (installation / "installation.json").is_file(): + runner.run( + [ + sys.executable, + str(DEPLOYER), + "init", + "--directory", + str(installation), + "--non-interactive", + "--installation-id", + config.name, + "--profile", + "self-hosted", + "--public-url", + config.public_url, + "--postgres", + "external", + "--database-url", + "postgresql://placeholder:placeholder@127.0.0.1/govoplan", + "--redis", + "external", + "--redis-url", + "redis://:placeholder@127.0.0.1:6379/0", + "--mail", + "external-relay", + "--storage", + "s3", + "--s3-endpoint-url", + config.s3_url, + "--s3-region", + "garage", + "--s3-access-key-id", + "GK00000000000000000000000000000000", + "--s3-secret-access-key", + "lab-placeholder-not-a-secret", + "--s3-bucket", + "govoplan-files", + "--ingress", + "existing-proxy", + "--trusted-proxy-cidr", + str( + ipaddress.ip_network( + f"{config.network.gateway}/{config.network.prefix_length}", + strict=False, + ) + ), + "--api-replicas", + str(config.api_replicas), + "--web-replicas", + str(config.web_replicas), + "--worker-replicas", + str(config.worker_replicas), + "--module-set", + config.module_set, + "--release-channel", + config.release.channel, + ], + timeout=120, + ) + _reconcile_installation_environment(config, installation, lab_secrets) + runner.run( + [ + sys.executable, + str(DEPLOYER), + "verify-release", + "--directory", + str(installation), + "--manifest", + str(manifest_path), + "--manifest-sha256", + config.release.manifest_sha256, + "--trusted-keyring", + str(keyring_path), + "--adopt", + ], + timeout=120, + ) + _reconcile_installation_environment(config, installation, lab_secrets) + return manifest, installation + + +def _reconcile_installation_environment( + config: LabConfig, installation: Path, lab_secrets: Mapping[str, str] +) -> None: + deployment_path = REPOSITORY_ROOT / "tools" / "deployment" + if str(deployment_path) not in sys.path: + sys.path.insert(0, str(deployment_path)) + from govoplan_deploy.bundle import read_env, write_env # type: ignore[import-not-found] + + env_path = installation / "secrets.env" + values = read_env(env_path) + database_url, pgtools_url = _database_urls(config, lab_secrets) + values.update( + { + "DATABASE_URL": database_url, + "GOVOPLAN_DATABASE_URL_PGTOOLS": pgtools_url, + "REDIS_URL": _redis_url(config, lab_secrets), + "FILE_STORAGE_S3_ENDPOINT_URL": config.s3_url, + "FILE_STORAGE_S3_REGION": "garage", + "FILE_STORAGE_S3_ACCESS_KEY_ID": lab_secrets["garage_access_key"], + "FILE_STORAGE_S3_SECRET_ACCESS_KEY": lab_secrets["garage_secret_key"], + "FILE_STORAGE_S3_BUCKET": "govoplan-files", + "FILE_STORAGE_S3_DEPLOYMENT_MANAGED": "false", + "FILE_STORAGE_S3_ENDPOINT_TRUSTED": "true", + "GOVOPLAN_DB_CONNECTION_LIMIT": str(config.db_connection_limit), + } + ) + write_env(env_path, values) + + +def _download_verified( + url: str, + expected_sha256: str, + output: Path, + *, + maximum_bytes: int, +) -> None: + if output.is_file() and _sha256_file(output) == expected_sha256: + return + request = Request(url, headers={"User-Agent": "GovOPlaN-lab/1"}) + try: + with urlopen(request, timeout=60) as response: + if urlsplit(response.geturl()).scheme != "https": + raise LabOperationError(f"pinned input redirected outside HTTPS: {url}") + payload = response.read(maximum_bytes + 1) + except OSError as exc: + raise LabOperationError(f"could not download pinned input: {url}") from exc + if len(payload) > maximum_bytes: + raise LabOperationError(f"downloaded input exceeds the size limit: {url}") + digest = hashlib.sha256(payload).hexdigest() + if digest != expected_sha256: + raise LabOperationError(f"downloaded input digest mismatch: {url}") + write_private(output, payload) + + +def _load_or_create_secrets(config: LabConfig) -> dict[str, str]: + path = config.state_directory / "lab-secrets.json" + if path.is_file(): + try: + value = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as exc: + raise LabOperationError("lab-secrets.json is not valid JSON") from exc + if not isinstance(value, dict) or not all( + isinstance(key, str) and isinstance(item, str) and item + for key, item in value.items() + ): + raise LabOperationError("lab-secrets.json has an invalid shape") + missing = sorted(_generated_secret_names() - set(value)) + if missing: + raise LabOperationError("lab-secrets.json is missing: " + ", ".join(missing)) + return dict(value) + value = { + "cluster_token": secrets.token_urlsafe(48), + "postgres_password": secrets.token_urlsafe(36), + "redis_password": secrets.token_urlsafe(36), + "garage_access_key": f"GK{secrets.token_hex(16)}", + "garage_secret_key": secrets.token_hex(32), + "garage_rpc_secret": secrets.token_hex(32), + "garage_admin_token": secrets.token_urlsafe(48), + "garage_metrics_token": secrets.token_urlsafe(48), + } + write_private(path, json.dumps(value, indent=2, sort_keys=True) + "\n") + return value + + +def _generated_secret_names() -> set[str]: + return { + "cluster_token", + "postgres_password", + "redis_password", + "garage_access_key", + "garage_secret_key", + "garage_rpc_secret", + "garage_admin_token", + "garage_metrics_token", + } + + +def _database_urls( + config: LabConfig, lab_secrets: Mapping[str, str] +) -> tuple[str, str]: + password = quote(lab_secrets["postgres_password"], safe="") + host = config.state_node.address + return ( + f"postgresql+psycopg://govoplan:{password}@{host}:5432/govoplan?sslmode=disable", + f"postgresql://govoplan:{password}@{host}:5432/govoplan?sslmode=disable", + ) + + +def _redis_url(config: LabConfig, lab_secrets: Mapping[str, str]) -> str: + password = quote(lab_secrets["redis_password"], safe="") + return f"redis://:{password}@{config.state_node.address}:6379/0" + + +def _ensure_certificates(config: LabConfig, runner: CommandRunner) -> None: + pki = config.state_directory / "pki" + pki.mkdir(parents=True, exist_ok=True, mode=0o700) + ca_key = pki / "ca.key" + ca_cert = pki / "ca.crt" + server_key = pki / "server.key" + server_csr = pki / "server.csr" + server_cert = pki / "server.crt" + extension = pki / "server.ext" + server_inputs = pki / "server-inputs.json" + expected_server_inputs = json.dumps( + { + "public_host": config.public_host, + "public_address": str(config.primary_control.address), + "s3_host": config.s3_host, + "s3_address": str(config.state_node.address), + }, + indent=2, + sort_keys=True, + ) + "\n" + ca_valid = ( + ca_key.is_file() + and ca_cert.is_file() + and runner.run( + [ + "openssl", + "x509", + "-checkend", + str(30 * 24 * 60 * 60), + "-noout", + "-in", + str(ca_cert), + ], + capture=True, + check=False, + ).returncode + == 0 + ) + ca_rotated = not ca_valid + if ca_rotated: + runner.run( + [ + "openssl", + "genpkey", + "-algorithm", + "RSA", + "-pkeyopt", + "rsa_keygen_bits:3072", + "-out", + str(ca_key), + ], + sensitive=True, + ) + runner.run( + [ + "openssl", + "req", + "-x509", + "-new", + "-sha256", + "-days", + "3650", + "-key", + str(ca_key), + "-subj", + f"/CN={config.name} private lab CA", + "-out", + str(ca_cert), + ], + ) + server_valid = ( + not ca_rotated + and server_key.is_file() + and server_cert.is_file() + and server_inputs.is_file() + and server_inputs.read_text(encoding="utf-8") == expected_server_inputs + and runner.run( + [ + "openssl", + "x509", + "-checkend", + str(30 * 24 * 60 * 60), + "-noout", + "-in", + str(server_cert), + ], + capture=True, + check=False, + ).returncode + == 0 + ) + if not server_valid: + runner.run( + [ + "openssl", + "genpkey", + "-algorithm", + "RSA", + "-pkeyopt", + "rsa_keygen_bits:3072", + "-out", + str(server_key), + ], + sensitive=True, + ) + runner.run( + [ + "openssl", + "req", + "-new", + "-sha256", + "-key", + str(server_key), + "-subj", + f"/CN={config.public_host}", + "-out", + str(server_csr), + ], + ) + write_private( + extension, + "basicConstraints=critical,CA:FALSE\n" + "keyUsage=critical,digitalSignature,keyEncipherment\n" + "extendedKeyUsage=serverAuth\n" + f"subjectAltName=DNS:{config.public_host},DNS:{config.s3_host}," + f"IP:{config.primary_control.address},IP:{config.state_node.address}\n", + ) + runner.run( + [ + "openssl", + "x509", + "-req", + "-sha256", + "-days", + "397", + "-in", + str(server_csr), + "-CA", + str(ca_cert), + "-CAkey", + str(ca_key), + "-CAcreateserial", + "-extfile", + str(extension), + "-out", + str(server_cert), + ], + ) + write_private(server_inputs, expected_server_inputs) + for path in (ca_key, server_key): + path.chmod(0o600) + write_private(config.state_directory / "hosts", render_hosts(config)) + + +def _deploy_state_services( + config: LabConfig, + runner: CommandRunner, + manifest: Mapping[str, Any], + lab_secrets: Mapping[str, str], +) -> None: + dependencies = manifest.get("dependencies") + if not isinstance(dependencies, dict) or not all( + isinstance(key, str) and isinstance(value, str) + for key, value in dependencies.items() + ): + raise LabOperationError("release manifest dependencies have an invalid shape") + local = config.state_directory / "rendered" / "state" + tls = local / "tls" + tls.mkdir(parents=True, exist_ok=True, mode=0o700) + write_private(local / "compose.json", render_state_compose(dependencies)) + write_private(local / "garage.toml", render_garage_config()) + write_private(local / "Caddyfile", render_caddyfile()) + write_private( + local / ".env", + render_state_environment( + { + "POSTGRES_DB": "govoplan", + "POSTGRES_USER": "govoplan", + "POSTGRES_PASSWORD": lab_secrets["postgres_password"], + "REDIS_PASSWORD": lab_secrets["redis_password"], + "GARAGE_DEFAULT_ACCESS_KEY": lab_secrets["garage_access_key"], + "GARAGE_DEFAULT_SECRET_KEY": lab_secrets["garage_secret_key"], + "GARAGE_DEFAULT_BUCKET": "govoplan-files", + "GARAGE_RPC_SECRET": lab_secrets["garage_rpc_secret"], + "GARAGE_ADMIN_TOKEN": lab_secrets["garage_admin_token"], + "GARAGE_METRICS_TOKEN": lab_secrets["garage_metrics_token"], + } + ), + ) + write_private(tls / "server.crt", (config.state_directory / "pki/server.crt").read_bytes()) + write_private(tls / "server.key", (config.state_directory / "pki/server.key").read_bytes()) + node = config.state_node + remote = f"/home/{config.ssh_user}/{_STATE_REMOTE_DIRECTORY}" + runner.vm(node, ["mkdir", "-p", f"{remote}/tls"]) + for source, destination in ( + (local / "compose.json", f"{remote}/compose.json"), + (local / "garage.toml", f"{remote}/garage.toml"), + (local / "Caddyfile", f"{remote}/Caddyfile"), + (local / ".env", f"{remote}/.env"), + (tls / "server.crt", f"{remote}/tls/server.crt"), + (tls / "server.key", f"{remote}/tls/server.key"), + ): + runner.copy_to_vm(node, source, destination) + runner.vm( + node, + [ + "docker", + "compose", + "--env-file", + f"{remote}/.env", + "--file", + f"{remote}/compose.json", + "pull", + ], + sudo=True, + timeout=1800, + ) + runner.vm( + node, + [ + "docker", + "compose", + "--env-file", + f"{remote}/.env", + "--file", + f"{remote}/compose.json", + "up", + "--detach", + "--remove-orphans", + ], + sudo=True, + timeout=300, + ) + runner.vm( + node, + [ + "docker", + "compose", + "--env-file", + f"{remote}/.env", + "--file", + f"{remote}/compose.json", + "restart", + "s3-tls", + ], + sudo=True, + timeout=120, + ) + _wait_for_tcp(str(node.address), 5432, label="PostgreSQL") + _wait_for_tcp(str(node.address), 6379, label="Redis") + _wait_for_tcp(str(node.address), 9443, label="Garage TLS endpoint") + _wait_for_https( + config.s3_url, + resolve_address=str(node.address), + ca_certificate=config.state_directory / "pki/ca.crt", + label="Garage TLS endpoint", + require_success_status=False, + ) + + +def _deploy_k3s( + config: LabConfig, + runner: CommandRunner, + lab_secrets: Mapping[str, str], + *, + update: bool, +) -> None: + username = os.environ.get("GOVOPLAN_LAB_REGISTRY_USERNAME", "").strip() + password = os.environ.get("GOVOPLAN_LAB_REGISTRY_PASSWORD", "").strip() + registry = render_registry_config(username, password) + ordered = [config.primary_control, *config.controls[1:], *config.workers] + for node in ordered: + if update and node.role == "worker": + _kubectl( + config, + runner, + ["cordon", node.name], + timeout=60, + ) + _kubectl( + config, + runner, + [ + "drain", + node.name, + "--ignore-daemonsets", + "--delete-emptydir-data", + "--timeout=10m", + ], + timeout=630, + ) + _install_k3s_node( + config, + runner, + node, + cluster_token=lab_secrets["cluster_token"], + registry=registry, + update=update, + ) + if node.role == "control": + _wait_for_k3s_api(config, runner) + _wait_for_k3s_node(config, runner, node) + if update and node.role == "worker": + _kubectl(config, runner, ["uncordon", node.name], timeout=60) + + +def _install_k3s_node( + config: LabConfig, + runner: CommandRunner, + node: LabNode, + *, + cluster_token: str, + registry: str, + update: bool, +) -> None: + rendered = config.state_directory / "rendered" / "k3s" / node.name + rendered.mkdir(parents=True, exist_ok=True, mode=0o700) + config_path = rendered / "config.yaml" + registry_path = rendered / "registries.yaml" + write_private( + config_path, + render_k3s_config(config, node, cluster_token=cluster_token), + ) + if registry: + write_private(registry_path, registry) + temporary_prefix = f"/tmp/{config.name}-{node.name}" + runner.copy_to_vm(node, config_path, f"{temporary_prefix}-config.yaml") + if registry: + runner.copy_to_vm(node, registry_path, f"{temporary_prefix}-registries.yaml") + runner.vm(node, ["install", "-d", "-m", "0700", "/etc/rancher/k3s"], sudo=True) + runner.vm( + node, + [ + "install", + "-m", + "0600", + f"{temporary_prefix}-config.yaml", + "/etc/rancher/k3s/config.yaml", + ], + sudo=True, + ) + if registry: + runner.vm( + node, + [ + "install", + "-m", + "0600", + f"{temporary_prefix}-registries.yaml", + "/etc/rancher/k3s/registries.yaml", + ], + sudo=True, + sensitive=True, + ) + else: + runner.vm( + node, + ["rm", "-f", "/etc/rancher/k3s/registries.yaml"], + sudo=True, + ) + runner.vm( + node, + [ + "rm", + "-f", + f"{temporary_prefix}-config.yaml", + f"{temporary_prefix}-registries.yaml", + ], + ) + _install_verified_remote( + runner, + node, + url=config.k3s.binary_url, + sha256=config.k3s.binary_sha256, + destination="/usr/local/bin/k3s", + mode="0755", + ) + _install_verified_remote( + runner, + node, + url=config.k3s.install_script_url, + sha256=config.k3s.install_script_sha256, + destination=f"{temporary_prefix}-install.sh", + mode="0700", + ) + service = "k3s" if node.role == "control" else "k3s-agent" + role = "server" if node.role == "control" else "agent" + runner.vm( + node, + [ + "env", + "INSTALL_K3S_SKIP_DOWNLOAD=true", + "INSTALL_K3S_SKIP_START=true", + f"INSTALL_K3S_VERSION={config.k3s.version}", + f"{temporary_prefix}-install.sh", + role, + ], + sudo=True, + timeout=180, + ) + runner.vm(node, ["systemctl", "enable", service], sudo=True) + runner.vm( + node, + ["systemctl", "restart" if update else "start", service], + sudo=True, + timeout=120, + ) + runner.vm( + node, + ["systemctl", "is-active", "--quiet", service], + sudo=True, + timeout=60, + ) + runner.vm( + node, + ["rm", "-f", f"{temporary_prefix}-install.sh"], + sudo=True, + check=False, + ) + + +def _install_verified_remote( + runner: CommandRunner, + node: LabNode, + *, + url: str, + sha256: str, + destination: str, + mode: str, +) -> None: + current = runner.vm( + node, + ["sha256sum", destination], + sudo=True, + capture=True, + check=False, + timeout=60, + ) + current_digest = ( + current.stdout.decode(errors="replace").split(maxsplit=1)[0] + if current.returncode == 0 and current.stdout + else "" + ) + if current_digest == sha256: + return + temporary = f"{destination}.download" + runner.vm( + node, + [ + "curl", + "--proto", + "=https", + "--tlsv1.2", + "--fail", + "--location", + "--output", + temporary, + url, + ], + sudo=True, + timeout=900, + ) + downloaded = runner.vm( + node, + ["sha256sum", temporary], + sudo=True, + capture=True, + timeout=900, + ) + digest = downloaded.stdout.decode().split(maxsplit=1)[0] + if digest != sha256: + runner.vm(node, ["rm", "-f", temporary], sudo=True, check=False) + raise LabOperationError(f"pinned download digest mismatch on {node.name}: {url}") + runner.vm( + node, + ["install", "-m", mode, temporary, destination], + sudo=True, + ) + runner.vm(node, ["rm", "-f", temporary], sudo=True, check=False) + + +def _wait_for_k3s_api(config: LabConfig, runner: CommandRunner) -> None: + _wait_for_tcp(str(config.primary_control.address), 6443, label="K3s API") + deadline = time.monotonic() + 300 + while time.monotonic() < deadline: + result = runner.vm( + config.primary_control, + ["k3s", "kubectl", "get", "--raw=/readyz"], + sudo=True, + capture=True, + check=False, + timeout=20, + ) + if result.returncode == 0 and result.stdout.strip() == b"ok": + return + time.sleep(5) + raise LabOperationError("K3s API did not become ready") + + +def _wait_for_k3s_node( + config: LabConfig, runner: CommandRunner, node: LabNode +) -> None: + deadline = time.monotonic() + 300 + while time.monotonic() < deadline: + result = _kubectl( + config, + runner, + [ + "get", + "node", + node.name, + "-o", + "jsonpath={.status.conditions[?(@.type=='Ready')].status}", + ], + capture=True, + check=False, + timeout=30, + ) + if result.returncode == 0 and result.stdout.strip() == b"True": + return + time.sleep(5) + raise LabOperationError(f"K3s node did not become Ready: {node.name}") + + +def _deploy_application( + config: LabConfig, runner: CommandRunner, installation: Path +) -> None: + deployment_path = REPOSITORY_ROOT / "tools" / "deployment" + if str(deployment_path) not in sys.path: + sys.path.insert(0, str(deployment_path)) + from govoplan_deploy.bundle import read_env # type: ignore[import-not-found] + + values = read_env(installation / "secrets.env") + missing = [key for key in _SECRET_KEYS if not values.get(key)] + if missing: + raise LabOperationError("installation secrets are missing: " + ", ".join(missing)) + namespace_manifest = ( + json.dumps( + { + "apiVersion": "v1", + "kind": "Namespace", + "metadata": {"name": config.namespace}, + } + ) + + "\n" + ).encode() + _kubectl(config, runner, ["apply", "-f", "-"], input_bytes=namespace_manifest) + runtime_secret = render_secret_manifest( + namespace=config.namespace, + name="govoplan-runtime", + values={key: values[key] for key in _SECRET_KEYS}, + ) + ca_certificate = (config.state_directory / "pki/ca.crt").read_bytes() + ca_secret_name = f"{config.name}-s3-ca" + ca_secret = render_secret_manifest( + namespace=config.namespace, + name=ca_secret_name, + values={"ca.crt": ca_certificate}, + ) + tls_secret = render_tls_secret_manifest( + namespace=config.namespace, + name="govoplan-tls", + certificate=(config.state_directory / "pki/server.crt").read_bytes(), + private_key=(config.state_directory / "pki/server.key").read_bytes(), + ) + for secret_manifest in (runtime_secret, ca_secret, tls_secret): + _kubectl( + config, + runner, + ["apply", "-f", "-"], + input_bytes=secret_manifest, + sensitive=True, + ) + kubernetes_manifest = installation / "kubernetes.json" + runner.run( + [ + sys.executable, + str(DEPLOYER), + "render-kubernetes", + "--directory", + str(installation), + "--namespace", + config.namespace, + "--secret-name", + "govoplan-runtime", + "--tls-secret-name", + "govoplan-tls", + "--ingress-class-name", + config.ingress_class, + "--s3-ca-secret-name", + ca_secret_name, + "--output", + str(kubernetes_manifest), + ], + timeout=120, + ) + _kubectl( + config, + runner, + ["apply", "-f", "-"], + input_bytes=kubernetes_manifest.read_bytes(), + timeout=180, + ) + rendered = json.loads(kubernetes_manifest.read_text(encoding="utf-8")) + jobs = [ + str(item["metadata"]["name"]) + for item in rendered.get("items", []) + if item.get("kind") == "Job" + ] + for job in jobs: + _kubectl( + config, + runner, + [ + "wait", + "--namespace", + config.namespace, + "--for=condition=complete", + f"job/{job}", + "--timeout=15m", + ], + timeout=930, + ) + _kubectl( + config, + runner, + [ + "wait", + "--namespace", + config.namespace, + "--for=condition=available", + "deployment", + "--all", + "--timeout=15m", + ], + timeout=930, + ) + _write_kubectl_wrapper(config, runner) + _wait_for_https( + config.public_url, + resolve_address=str(config.primary_control.address), + ca_certificate=config.state_directory / "pki/ca.crt", + label="GovOPlaN ingress", + require_success_status=True, + ) + + +def _kubectl( + config: LabConfig, + runner: CommandRunner, + arguments: Sequence[str], + *, + input_bytes: bytes | None = None, + capture: bool = False, + check: bool = True, + timeout: float | None = None, + sensitive: bool = False, +) -> subprocess.CompletedProcess[bytes]: + return runner.vm( + config.primary_control, + ["k3s", "kubectl", *arguments], + sudo=True, + input_bytes=input_bytes, + capture=capture, + check=check, + timeout=timeout, + sensitive=sensitive, + ) + + +def status(config: LabConfig, *, verbose: bool = False) -> int: + _prepare_local_state(config) + runner = CommandRunner(config, verbose=verbose) + failed = False + for node in config.nodes: + result = runner.hypervisor( + node, + ["virsh", "domstate", _domain_name(config, node)], + capture=True, + check=False, + timeout=20, + ) + state = ( + result.stdout.decode(errors="replace").strip() + if result.returncode == 0 + else "missing" + ) + print(f"VM {node.name}: {state} ({node.hypervisor}, {node.failure_domain})") + failed = failed or result.returncode != 0 + cluster = _kubectl( + config, + runner, + ["get", "nodes", "-o", "wide"], + capture=True, + check=False, + timeout=30, + ) + if cluster.returncode == 0: + print(cluster.stdout.decode(errors="replace").rstrip()) + pods = _kubectl( + config, + runner, + ["get", "pods", "--namespace", config.namespace, "-o", "wide"], + capture=True, + check=False, + timeout=30, + ) + print(pods.stdout.decode(errors="replace").rstrip()) + failed = failed or pods.returncode != 0 + else: + print("Kubernetes API is not reachable.") + failed = True + return 1 if failed else 0 + + +def pause(config: LabConfig, *, apply: bool, verbose: bool = False) -> None: + if not apply: + _print_mutation_preview( + "pause", + config, + ["gracefully shut down workers, control planes, then shared state"], + ) + return + runner = CommandRunner(config, verbose=verbose) + ordered = [*config.workers, *reversed(config.controls), config.state_node] + for node in ordered: + domain = _domain_name(config, node) + current = runner.hypervisor( + node, + ["virsh", "domstate", domain], + capture=True, + check=False, + ) + if current.returncode != 0 or b"shut off" in current.stdout.lower(): + continue + _assert_domain_owned(config, runner, node) + runner.hypervisor(node, ["virsh", "shutdown", domain]) + _wait_for_domain_state(config, runner, node, "shut off", timeout=180) + print(f"Paused lab {config.name}; disks and evidence remain in place.") + + +def resume(config: LabConfig, *, apply: bool, verbose: bool = False) -> None: + if not apply: + _print_mutation_preview( + "resume", + config, + ["start shared state, control planes, then workers and wait for readiness"], + ) + return + _prepare_local_state(config) + runner = CommandRunner(config, verbose=verbose) + ordered = [config.state_node, *config.controls, *config.workers] + for node in ordered: + domain = _domain_name(config, node) + _assert_domain_owned(config, runner, node) + runner.hypervisor(node, ["virsh", "start", domain], check=False) + _wait_for_ssh(config, runner, node) + if node.role == "state": + _wait_for_tcp(str(node.address), 5432, label="PostgreSQL") + elif node.role == "control": + _wait_for_k3s_api(config, runner) + else: + _wait_for_k3s_node(config, runner, node) + print(f"Resumed lab {config.name}.") + + +def destroy( + config: LabConfig, + *, + apply: bool, + confirmation: str, + purge_local_state: bool, + verbose: bool = False, +) -> None: + actions = [ + f"destroy and undefine {len(config.nodes)} lab-owned libvirt domains", + "delete their lab-owned qcow2 overlays and cloud-init seeds", + "preserve local evidence" if not purge_local_state else "delete local lab state and evidence", + ] + if not apply: + _print_mutation_preview("destroy", config, actions) + return + if confirmation != config.name: + raise LabOperationError( + f"destruction requires --confirm {config.name}; received {confirmation!r}" + ) + runner = CommandRunner(config, verbose=verbose) + for node in [*config.workers, *reversed(config.controls), config.state_node]: + domain = _domain_name(config, node) + existing = runner.hypervisor( + node, + ["virsh", "dominfo", domain], + capture=True, + check=False, + timeout=20, + ) + if existing.returncode != 0: + print(f"Skipping missing domain {domain}; no disk directory was removed.") + _forget_vm_host_key(config, runner, node) + continue + _assert_domain_owned(config, runner, node) + runner.hypervisor(node, ["virsh", "destroy", domain], check=False) + runner.hypervisor( + node, + ["virsh", "undefine", domain, "--nvram"], + check=False, + ) + node_directory = f"{config.vm_image_directory}/{config.name}/{node.name}" + runner.hypervisor( + node, + ["rm", "--recursive", "--force", "--one-file-system", node_directory], + ) + _forget_vm_host_key(config, runner, node) + if purge_local_state: + resolved = config.state_directory.resolve() + home = Path.home().resolve() + if resolved in {Path("/"), home} or len(resolved.parts) < 4: + raise LabOperationError("refusing to purge an unsafe state_directory") + shutil.rmtree(resolved) + print(f"Destroyed VM resources for lab {config.name}.") + + +def verify( + config: LabConfig, + *, + exercise_api_pod_loss: bool, + verbose: bool = False, +) -> None: + api_key = os.environ.get("GOVOPLAN_OPS_API_KEY", "").strip() + if not api_key: + raise LabOperationError( + "GOVOPLAN_OPS_API_KEY must contain a short-lived key authorized " + "to read Ops status" + ) + runner = CommandRunner(config, verbose=verbose) + wrapper = _write_kubectl_wrapper(config, runner) + environment = dict(os.environ) + environment["PATH"] = f"{wrapper.parent}:{environment.get('PATH', '')}" + environment["SSL_CERT_FILE"] = str(config.state_directory / "pki/ca.crt") + environment["GOVOPLAN_OPS_API_KEY"] = api_key + output = config.state_directory / "evidence" / "kubernetes-multi-host.json" + output.parent.mkdir(parents=True, exist_ok=True, mode=0o700) + command = [ + sys.executable, + str(DEPLOYER), + "verify-kubernetes", + "--directory", + str(config.state_directory / "installation"), + "--namespace", + config.namespace, + "--output", + str(output), + ] + if exercise_api_pod_loss: + command.append("--exercise-api-pod-loss") + runner.run(command, timeout=900, env=environment) + print(f"Wrote sanitized Kubernetes evidence to {output}.") + if not config.evidence_capable: + print("This is rehearsal evidence only: the worker VMs do not prove independent failure domains.") + + +def enroll_admin( + config: LabConfig, + *, + email: str, + display_name: str | None, + tenant_slug: str, + tenant_name: str, + apply: bool, +) -> None: + artifact = config.state_directory / "first-admin-enrollment.json" + if not apply: + _print_mutation_preview( + "enroll-admin", + config, + [ + f"consume {artifact} over the private lab CA", + f"create the first administrator {email!r} in tenant {tenant_slug!r}", + "delete the local enrollment artifact after a successful response", + ], + ) + return + if not artifact.is_file(): + raise LabOperationError(f"first-administrator artifact is missing: {artifact}") + mode = stat.S_IMODE(artifact.stat().st_mode) + if mode & 0o077: + raise LabOperationError( + f"first-administrator artifact must be owner-only; found mode {oct(mode)}" + ) + try: + credential = json.loads(artifact.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + raise LabOperationError( + "first-administrator artifact is not valid JSON" + ) from exc + if not isinstance(credential, dict): + raise LabOperationError("first-administrator artifact must be a JSON object") + endpoint = str(credential.get("endpoint") or "") + header = str(credential.get("header") or "") + token = str(credential.get("enrollment_token") or "") + if endpoint != "/api/v1/bootstrap/first-admin": + raise LabOperationError("first-administrator artifact has an unexpected endpoint") + if header != "X-GovOPlaN-Enrollment-Token" or len(token) < 32: + raise LabOperationError("first-administrator artifact has an invalid token contract") + password = getpass.getpass("First administrator password: ") + confirmation = getpass.getpass("Confirm password: ") + if password != confirmation: + raise LabOperationError("password confirmation does not match") + if len(password) < 12: + raise LabOperationError("first administrator password must contain at least 12 characters") + public = urlsplit(config.public_url) + if public.scheme != "https" or not public.hostname: + raise LabOperationError("first-administrator enrollment requires an HTTPS public URL") + body = json.dumps( + { + "email": email, + "display_name": display_name or None, + "password": password, + "tenant_slug": tenant_slug, + "tenant_name": tenant_name, + }, + separators=(",", ":"), + ).encode("utf-8") + context = ssl.create_default_context( + cafile=str(config.state_directory / "pki" / "ca.crt") + ) + connection = http.client.HTTPSConnection( + public.hostname, + public.port or 443, + timeout=30, + context=context, + ) + try: + connection.request( + "POST", + endpoint, + body=body, + headers={ + "Accept": "application/json", + "Content-Type": "application/json", + header: token, + }, + ) + response = connection.getresponse() + response_body = response.read(64 * 1024 + 1) + except (OSError, http.client.HTTPException) as exc: + raise LabOperationError(f"first-administrator enrollment failed: {exc}") from exc + finally: + connection.close() + if len(response_body) > 64 * 1024: + raise LabOperationError("first-administrator enrollment response was too large") + if response.status != 201: + detail = response_body.decode("utf-8", errors="replace").strip() + raise LabOperationError( + f"first-administrator enrollment returned HTTP {response.status}: {detail}" + ) + artifact.unlink() + print(f"Enrolled first administrator {email!r} and removed {artifact}.") + + +def _write_kubectl_wrapper(config: LabConfig, runner: CommandRunner) -> Path: + binary_directory = config.state_directory / "bin" + binary_directory.mkdir(parents=True, exist_ok=True, mode=0o700) + path = binary_directory / "kubectl" + target = f"{config.ssh_user}@{config.primary_control.address}" + ssh = runner.ssh_arguments(target) + write_private(path, _render_kubectl_wrapper(ssh)) + path.chmod(0o700) + return path + + +def _render_kubectl_wrapper(ssh: Sequence[str]) -> str: + return ( + "#!/usr/bin/env python3\n" + "import os\n" + "import shlex\n" + "import sys\n\n" + f"_SSH = {list(ssh)!r}\n" + '_REMOTE = ["sudo", "--", "k3s", "kubectl", *sys.argv[1:]]\n' + "os.execvp(_SSH[0], [*_SSH, shlex.join(_REMOTE)])\n" + ) + + +def _wait_for_domain_state( + config: LabConfig, + runner: CommandRunner, + node: LabNode, + desired: str, + *, + timeout: float, +) -> None: + deadline = time.monotonic() + timeout + domain = _domain_name(config, node) + while time.monotonic() < deadline: + result = runner.hypervisor( + node, + ["virsh", "domstate", domain], + capture=True, + check=False, + timeout=20, + ) + if result.returncode == 0 and result.stdout.decode().strip().lower() == desired: + return + time.sleep(3) + raise LabOperationError(f"domain {domain} did not reach state {desired!r}") + + +def _wait_for_tcp(host: str, port: int, *, label: str, timeout: float = 300) -> None: + deadline = time.monotonic() + timeout + last_error = "" + while time.monotonic() < deadline: + try: + with socket.create_connection((host, port), timeout=3): + return + except OSError as exc: + last_error = str(exc) + time.sleep(3) + raise LabOperationError(f"{label} did not become reachable at {host}:{port}: {last_error}") + + +def _wait_for_https( + url: str, + *, + resolve_address: str, + ca_certificate: Path, + label: str, + require_success_status: bool, + timeout: float = 300, +) -> None: + parsed = urlsplit(url) + if parsed.scheme != "https" or parsed.hostname is None: + raise LabOperationError(f"{label} URL must be HTTPS") + port = parsed.port or 443 + command = [ + "curl", + "--silent", + "--show-error", + "--output", + "/dev/null", + "--connect-timeout", + "5", + "--max-time", + "15", + "--cacert", + str(ca_certificate), + "--resolve", + f"{parsed.hostname}:{port}:{resolve_address}", + ] + if require_success_status: + command.append("--fail") + command.append(url) + deadline = time.monotonic() + timeout + last_error = "" + while time.monotonic() < deadline: + result = subprocess.run(command, capture_output=True, check=False) + if result.returncode == 0: + return + last_error = result.stderr.decode(errors="replace").strip() + time.sleep(3) + raise LabOperationError(f"{label} did not become ready: {last_error}") + + +def _sha256_file(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def _forget_vm_host_key( + config: LabConfig, + runner: CommandRunner, + node: LabNode, +) -> None: + runner.run( + [ + "ssh-keygen", + "-f", + str(config.state_directory / "ssh_known_hosts"), + "-R", + str(node.address), + ], + check=False, + capture=True, + ) diff --git a/tools/lab/govoplan_lab/render.py b/tools/lab/govoplan_lab/render.py new file mode 100644 index 0000000..913e73e --- /dev/null +++ b/tools/lab/govoplan_lab/render.py @@ -0,0 +1,311 @@ +"""Deterministic configuration rendering for the Kubernetes VM lab.""" + +from __future__ import annotations + +import base64 +import json +from pathlib import Path +from typing import Mapping + +from .config import LabConfig, LabNode + + +def render_cloud_init(config: LabConfig, node: LabNode, public_key: str) -> str: + packages = ["ca-certificates", "curl", "qemu-guest-agent"] + if node.role == "state": + packages.extend(["docker.io", "docker-compose-v2", "openssl"]) + package_lines = "\n".join(f" - {item}" for item in packages) + return f"""#cloud-config +hostname: {node.name} +manage_etc_hosts: true +package_update: true +package_upgrade: false +packages: +{package_lines} +users: + - default + - name: {config.ssh_user} + groups: [adm, sudo] + shell: /bin/bash + sudo: ALL=(ALL) NOPASSWD:ALL + lock_passwd: true + ssh_authorized_keys: + - {json.dumps(public_key.strip())} +ssh_pwauth: false +disable_root: true +runcmd: + - [systemctl, enable, --now, qemu-guest-agent] + - [sh, -c, "test ! -e /usr/bin/docker || systemctl enable --now docker"] +final_message: "GovOPlaN lab node is ready" +""" + + +def render_network_config(config: LabConfig, node: LabNode) -> str: + dns = ", ".join(str(item) for item in config.network.dns_servers) + return f"""version: 2 +ethernets: + primary: + match: + macaddress: {node.mac_address} + set-name: eth0 + addresses: + - {node.address}/{config.network.prefix_length} + routes: + - to: default + via: {config.network.gateway} + nameservers: + addresses: [{dns}] +""" + + +def render_meta_data(config: LabConfig, node: LabNode) -> str: + return f"instance-id: {config.name}-{node.name}\nlocal-hostname: {node.name}\n" + + +def render_k3s_config( + config: LabConfig, + node: LabNode, + *, + cluster_token: str, +) -> str: + lines = [ + f'node-name: "{node.name}"', + f'node-ip: "{node.address}"', + f'token: "{cluster_token}"', + ] + if node.role == "control": + if node == config.primary_control: + lines.append("cluster-init: true") + else: + lines.append(f'server: "https://{config.primary_control.address}:6443"') + lines.extend( + [ + 'write-kubeconfig-mode: "0600"', + "secrets-encryption: true", + "tls-san:", + f' - "{config.primary_control.address}"', + "node-taint:", + ' - "node-role.kubernetes.io/control-plane=true:NoSchedule"', + ] + ) + elif node.role == "worker": + lines.extend( + [ + f'server: "https://{config.primary_control.address}:6443"', + "node-label:", + ' - "govoplan.add-ideas.de/runtime=true"', + f' - "topology.govoplan.add-ideas.de/failure-domain={node.failure_domain}"', + ] + ) + else: + raise ValueError("state nodes do not receive K3s configuration") + return "\n".join(lines) + "\n" + + +def render_registry_config(username: str, password: str) -> str: + if not username and not password: + return "" + if not username or not password: + raise ValueError("registry username and password must be supplied together") + return ( + 'mirrors:\n "git.add-ideas.de":\n' + ' endpoint:\n - "https://git.add-ideas.de"\n' + 'configs:\n "git.add-ideas.de":\n auth:\n' + f" username: {json.dumps(username)}\n" + f" password: {json.dumps(password)}\n" + ) + + +def render_garage_config() -> str: + return """metadata_dir = "/var/lib/garage/meta" +data_dir = "/var/lib/garage/data" +db_engine = "sqlite" + +replication_factor = 1 + +rpc_bind_addr = "[::]:3901" +rpc_public_addr = "127.0.0.1:3901" + +[s3_api] +s3_region = "garage" +api_bind_addr = "[::]:3900" +root_domain = ".s3.garage.localhost" + +[admin] +api_bind_addr = "[::]:3903" +""" + + +def render_caddyfile() -> str: + return """:9443 { + tls /etc/caddy/tls/server.crt /etc/caddy/tls/server.key + reverse_proxy garage:3900 +} +""" + + +def render_state_compose(images: Mapping[str, str]) -> str: + required = {"postgres", "redis", "garage", "managed_ingress", "test_mail"} + missing = sorted(required - set(images)) + if missing: + raise ValueError("release manifest is missing state images: " + ", ".join(missing)) + value = { + "name": "govoplan-lab-state", + "services": { + "postgres": { + "image": images["postgres"], + "restart": "unless-stopped", + "environment": { + "POSTGRES_DB": "${POSTGRES_DB}", + "POSTGRES_USER": "${POSTGRES_USER}", + "POSTGRES_PASSWORD": "${POSTGRES_PASSWORD}", + }, + "ports": ["5432:5432"], + "healthcheck": { + "test": [ + "CMD-SHELL", + 'pg_isready -U "$${POSTGRES_USER}" -d "$${POSTGRES_DB}"', + ], + "interval": "5s", + "timeout": "3s", + "retries": 30, + }, + "volumes": ["postgres-data:/var/lib/postgresql/data"], + "networks": ["internal"], + }, + "redis": { + "image": images["redis"], + "restart": "unless-stopped", + "command": [ + "sh", + "-ec", + 'exec redis-server --appendonly yes --requirepass "$$REDIS_PASSWORD"', + ], + "environment": {"REDIS_PASSWORD": "${REDIS_PASSWORD}"}, + "ports": ["6379:6379"], + "healthcheck": { + "test": [ + "CMD-SHELL", + 'redis-cli -a "$${REDIS_PASSWORD}" --no-auth-warning ping', + ], + "interval": "5s", + "timeout": "3s", + "retries": 30, + }, + "volumes": ["redis-data:/data"], + "networks": ["internal"], + }, + "garage": { + "image": images["garage"], + "restart": "unless-stopped", + "command": ["/garage", "server", "--single-node", "--default-bucket"], + "environment": { + "GARAGE_DEFAULT_ACCESS_KEY": "${GARAGE_DEFAULT_ACCESS_KEY}", + "GARAGE_DEFAULT_SECRET_KEY": "${GARAGE_DEFAULT_SECRET_KEY}", + "GARAGE_DEFAULT_BUCKET": "${GARAGE_DEFAULT_BUCKET}", + "GARAGE_RPC_SECRET": "${GARAGE_RPC_SECRET}", + "GARAGE_ADMIN_TOKEN": "${GARAGE_ADMIN_TOKEN}", + "GARAGE_METRICS_TOKEN": "${GARAGE_METRICS_TOKEN}", + }, + "healthcheck": { + "test": ["CMD", "/garage", "status"], + "interval": "10s", + "timeout": "5s", + "retries": 30, + "start_period": "15s", + }, + "security_opt": ["no-new-privileges:true"], + "volumes": [ + "./garage.toml:/etc/garage.toml:ro", + "garage-meta:/var/lib/garage/meta", + "garage-data:/var/lib/garage/data", + ], + "networks": ["internal"], + }, + "s3-tls": { + "image": images["managed_ingress"], + "restart": "unless-stopped", + "depends_on": {"garage": {"condition": "service_healthy"}}, + "ports": ["9443:9443"], + "volumes": [ + "./Caddyfile:/etc/caddy/Caddyfile:ro", + "./tls:/etc/caddy/tls:ro", + ], + "security_opt": ["no-new-privileges:true"], + "networks": ["internal"], + }, + "test-mail": { + "image": images["test_mail"], + "restart": "unless-stopped", + "environment": { + "GREENMAIL_OPTS": ( + "-Dgreenmail.setup.test.smtp -Dgreenmail.setup.test.imap " + "-Dgreenmail.hostname=0.0.0.0" + ) + }, + "ports": ["3025:3025", "3143:3143"], + "networks": ["internal"], + }, + }, + "volumes": { + "postgres-data": {}, + "redis-data": {}, + "garage-meta": {}, + "garage-data": {}, + }, + "networks": {"internal": {"driver": "bridge"}}, + } + return json.dumps(value, indent=2, sort_keys=True) + "\n" + + +def render_state_environment(values: Mapping[str, str]) -> str: + return "".join(f"{key}={_env_quote(value)}\n" for key, value in sorted(values.items())) + + +def render_secret_manifest( + *, namespace: str, name: str, values: Mapping[str, bytes | str] +) -> bytes: + encoded = { + key: base64.b64encode(value.encode() if isinstance(value, str) else value).decode() + for key, value in sorted(values.items()) + } + payload = { + "apiVersion": "v1", + "kind": "Secret", + "metadata": {"name": name, "namespace": namespace}, + "type": "Opaque", + "data": encoded, + } + return (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode() + + +def render_tls_secret_manifest( + *, namespace: str, name: str, certificate: bytes, private_key: bytes +) -> bytes: + payload = json.loads( + render_secret_manifest( + namespace=namespace, + name=name, + values={"tls.crt": certificate, "tls.key": private_key}, + ) + ) + payload["type"] = "kubernetes.io/tls" + return (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode() + + +def render_hosts(config: LabConfig) -> str: + return ( + f"{config.primary_control.address} {config.public_host}\n" + f"{config.state_node.address} {config.s3_host}\n" + ) + + +def _env_quote(value: str) -> str: + return json.dumps(value, ensure_ascii=True) + + +def write_private(path: Path, value: str | bytes) -> None: + path.parent.mkdir(parents=True, exist_ok=True, mode=0o700) + data = value.encode() if isinstance(value, str) else value + path.write_bytes(data) + path.chmod(0o600) diff --git a/tools/lab/libvirt/govoplan-lab-network.xml b/tools/lab/libvirt/govoplan-lab-network.xml new file mode 100644 index 0000000..17c3292 --- /dev/null +++ b/tools/lab/libvirt/govoplan-lab-network.xml @@ -0,0 +1,11 @@ + + govoplan-lab + + + + + + + + + diff --git a/tools/release/generate-release-catalog.py b/tools/release/generate-release-catalog.py index 6e7116b..54bf676 100644 --- a/tools/release/generate-release-catalog.py +++ b/tools/release/generate-release-catalog.py @@ -332,6 +332,7 @@ def _manifest_catalog_metadata(manifest: ModuleManifest | None) -> dict[str, obj payload["optional_dependencies"] = list(manifest.optional_dependencies) if manifest.architecture is not None: payload["architecture"] = manifest.architecture.to_dict() + payload["information_governance"] = manifest.information_governance.to_dict() if manifest.external_providers: payload["external_providers"] = [ declaration.to_dict() diff --git a/tools/release/govoplan_release/catalog_entry_synthesis.py b/tools/release/govoplan_release/catalog_entry_synthesis.py index d517081..cbc21b7 100644 --- a/tools/release/govoplan_release/catalog_entry_synthesis.py +++ b/tools/release/govoplan_release/catalog_entry_synthesis.py @@ -167,6 +167,7 @@ def manifest_catalog_entry( entry["optional_dependencies"] = list(manifest.optional_dependencies) if manifest.architecture is not None: entry["architecture"] = manifest.architecture.to_dict() + entry["information_governance"] = manifest.information_governance.to_dict() if manifest.external_providers: entry["external_providers"] = [ declaration.to_dict() diff --git a/tools/release/runtime/Dockerfile.web b/tools/release/runtime/Dockerfile.web index f41cf51..741c7d2 100644 --- a/tools/release/runtime/Dockerfile.web +++ b/tools/release/runtime/Dockerfile.web @@ -11,11 +11,13 @@ LABEL org.opencontainers.image.title="GovOPlaN WebUI runtime" \ USER 0 RUN rm -rf /usr/share/nginx/html/* /etc/nginx/conf.d/* COPY web-dist/ /usr/share/nginx/html/ -COPY nginx.conf /etc/nginx/nginx.conf +COPY nginx.conf /etc/nginx/nginx.conf.template +COPY web-entrypoint.sh /usr/local/bin/govoplan-web-entrypoint RUN chown -R 101:101 /usr/share/nginx/html \ - && chmod -R a-w /usr/share/nginx/html /etc/nginx/nginx.conf + && chmod -R a-w /usr/share/nginx/html /etc/nginx/nginx.conf.template \ + && chmod 0555 /usr/local/bin/govoplan-web-entrypoint USER 101:101 EXPOSE 8080 -ENTRYPOINT [] -CMD ["nginx", "-g", "daemon off;"] +ENTRYPOINT ["/usr/local/bin/govoplan-web-entrypoint"] +CMD [] diff --git a/tools/release/runtime/nginx.conf b/tools/release/runtime/nginx.conf index 36d9668..bbe80a1 100644 --- a/tools/release/runtime/nginx.conf +++ b/tools/release/runtime/nginx.conf @@ -35,7 +35,7 @@ http { } location /api/ { - proxy_pass http://load-balancer:8000; + proxy_pass ${GOVOPLAN_API_UPSTREAM}; proxy_http_version 1.1; proxy_set_header Host $host; proxy_set_header X-Forwarded-Host $host;