feat: add tenant semantic documentation lifecycle

This commit is contained in:
2026-08-21 15:37:01 +02:00
parent 77eb7339e6
commit d6db344d81
22 changed files with 3717 additions and 8 deletions
+2
View File
@@ -6,6 +6,8 @@
GovOPlaN Docs provides the documentation layer for configured GovOPlaN systems. GovOPlaN Docs provides the documentation layer for configured GovOPlaN systems.
It also stores tenant-owned semantic documentation for stable configured subjects. See [Tenant semantic documentation](docs/SEMANTIC_DOCUMENTATION.md) for lifecycle, authorization, publication policy, export, recovery, and provider obligations.
It answers "what does this installation provide for me?" before it answers "what could the full product do?". The goal is to reduce documentation complexity by starting from installed modules, enabled configuration, tenant policy, role, route availability, and visible capabilities. It answers "what does this installation provide for me?" before it answers "what could the full product do?". The goal is to reduce documentation complexity by starting from installed modules, enabled configuration, tenant policy, role, route availability, and visible capabilities.
## Ownership ## Ownership
+42
View File
@@ -0,0 +1,42 @@
# Tenant semantic documentation
Docs stores tenant-specific explanations of stable configured subjects supplied by owning modules. Forms and Workflow are the first providers; additional modules can implement the Core semantic-subject contract without Docs importing their internals.
## Lifecycle and authorization
An entry is unique by tenant, stable subject identity, anchor, and locale. Every create, edit, publish, supersede, and retire operation appends an immutable revision and requires optimistic concurrency. A published revision remains reader-visible while a newer draft is being reviewed.
The tenant setting `docs.semantic_publication_policy` accepts:
- `reviewer_required` (default): the account that authored the current draft cannot publish it.
- `direct`: an author with the publish permission may publish their own draft.
Creation, editing, publication, supersession, retirement, tenant export, and policy changes use separate permission scopes. All mutations write audit events containing stable references and revision numbers, never the authored prose.
Reads always intersect:
- active tenant isolation;
- current authorization returned by the owning subject provider;
- scopes required by that subject;
- the semantic entry's classification and typed audience selectors;
- publication state and requested locale.
The same read-time check protects direct entry URLs, search authorization rechecks, contextual consumers, and tenant export. Provider denial is indistinguishable from absence. Changed, superseded, missing, and temporarily unavailable subjects are represented explicitly; locale fallback is exposed in the response.
## Content and safety
Semantic content is bounded plain text. Links must be local absolute paths or HTTPS URLs without embedded credentials. Restricted content requires at least one typed audience selector: `account:`, `group:`, `role:`, `function:`, `scope:`, or `authenticated`.
Search indexes only published revisions and always requires provider reauthorization before returning a result. Generic public documentation generation reads static manifest topics only, so it cannot include tenant semantic entries. The separately authorized tenant export includes current entries and immutable history and sends `private, no-store`.
## Backup, recovery, and module removal
Back up `docs_semantic_entries` and `docs_semantic_revisions` together with Core tenant and audit state. Restoring only one table breaks revision pointers and is unsupported. The installer blocks normal uninstall while rows remain. Destructive retirement is explicit, requires a database snapshot, and drops revision history before entries.
Published authorship and review references are retained as configuration-governance evidence during data-subject erasure. Draft attribution is returned for governed manual review rather than silently anonymized because ownership and stewardship may need reassignment first.
## Provider obligations
An owning module registers exactly `documentation.semantic_subjects.<module_id>` with Core contract version `1`. It must return only tenant-local, currently authorized descriptors; stable subject and anchor IDs; current revision and fingerprint; localized labels; and relevant route and scope metadata. Resolution must return `None` when a principal may not learn whether a subject exists.
Provider unavailability does not expose stored content. Subject deletion or replacement must produce an explicit missing or superseded resolution so configurators can govern the documentation lifecycle.
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "govoplan-docs" name = "govoplan-docs"
version = "0.1.18" version = "0.1.19"
description = "GovOPlaN documentation module for configured-system, available, and evidence documentation." description = "GovOPlaN documentation module for configured-system, available, and evidence documentation."
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
+147
View File
@@ -32,12 +32,19 @@ from govoplan_core.core.versioning import (
from govoplan_core.db.session import get_database from govoplan_core.db.session import get_database
from govoplan_docs.backend.manifest import DOCS_ADMIN_READ_SCOPES, DOCS_READ_SCOPES from govoplan_docs.backend.manifest import DOCS_ADMIN_READ_SCOPES, DOCS_READ_SCOPES
from govoplan_docs.backend.api.v1.semantic_routes import router as semantic_router
from govoplan_docs.backend.sources import ( from govoplan_docs.backend.sources import (
RegisteredDocumentationSource, RegisteredDocumentationSource,
build_documentation_source_registry, build_documentation_source_registry,
) )
from govoplan_docs.backend.semantic_service import (
list_semantic_entries,
select_locale_entries,
semantic_entry_payload,
)
router = APIRouter(prefix="/docs", tags=["docs"]) router = APIRouter(prefix="/docs", tags=["docs"])
router.include_router(semantic_router)
TOPIC_KINDS = ("workflow", "reference", "pattern", "system") TOPIC_KINDS = ("workflow", "reference", "pattern", "system")
_CONFIGURATION_ACTIVE_STATES = frozenset({"enabled", "inherited"}) _CONFIGURATION_ACTIVE_STATES = frozenset({"enabled", "inherited"})
@@ -958,9 +965,149 @@ def _collect_documentation_topics(
for topic in provided_topics: for topic in provided_topics:
if not user_workflow_scope_condition_issues(topic): if not user_workflow_scope_condition_issues(topic):
topics.append((manifest.id, topic)) topics.append((manifest.id, topic))
if session is not None:
try:
topics.extend(
("docs", topic)
for topic in _semantic_documentation_topics(
registry,
principal,
session=session,
locale=locale,
)
)
except Exception as exc:
if documentation_type == "admin":
topics.append(
(
"docs",
DocumentationTopic(
id="docs.semantic-runtime-unavailable",
title="Tenant semantic documentation unavailable",
summary="Stored tenant semantics could not be projected for this request.",
body="Static module documentation remains available. Check the Docs database migration, subject providers, and application logs.",
layer="evidence",
documentation_types=("admin",),
source_module_id="docs",
metadata={
"kind": "system",
"error_type": type(exc).__name__,
},
),
)
)
return topics return topics
def _semantic_documentation_topics(
registry: PlatformRegistry,
principal: ApiPrincipal,
*,
session: object,
locale: str,
) -> tuple[DocumentationTopic, ...]:
entries = select_locale_entries(
list_semantic_entries(session, principal),
locale=locale,
)
topics: list[DocumentationTopic] = []
for entry in entries:
payload = semantic_entry_payload(
session,
registry,
principal,
entry=entry,
editor=False,
requested_locale=locale,
)
if payload is None:
continue
content = payload["content"]
resolution = payload["subject_resolution"]
subject = resolution.get("subject") if isinstance(resolution, Mapping) else None
route = subject.get("route") if isinstance(subject, Mapping) else None
route_anchor = (
subject.get("route_anchor") if isinstance(subject, Mapping) else None
)
links = [
DocumentationLink(
label=str(item["label"]),
href=str(item["href"]),
kind="runtime" if str(item["href"]).startswith("/") else "external",
)
for item in content.get("links", ())
if isinstance(item, Mapping) and item.get("label") and item.get("href")
]
if isinstance(route, str) and route.startswith("/"):
links.insert(
0,
DocumentationLink(
label="Open configured subject",
href=(f"{route}#{route_anchor}" if route_anchor else route),
kind="runtime",
),
)
body_parts = [
str(content.get(key) or "").strip()
for key in (
"meaning",
"body",
"intended_use",
"non_intended_use",
)
]
topics.append(
DocumentationTopic(
id=f"docs.semantic.{entry.id}",
title=str(content.get("title") or "Semantic documentation"),
summary=str(content.get("summary") or "Tenant semantic guidance"),
body="\n\n".join(part for part in body_parts if part),
layer="configured",
documentation_types=("admin", "user"),
source_module_id="docs",
order=200,
links=tuple(links),
related_modules=(entry.subject_module_id,),
metadata={
"kind": "reference",
"source_badge": "tenant_semantic",
"semantic_entry_id": entry.id,
"semantic_subject": payload["subject"],
"semantic_subject_stable_key": entry.subject_stable_key,
"subject_availability": resolution.get("availability"),
"locale": payload["locale"],
"requested_locale": payload["requested_locale"],
"locale_fallback": payload["locale_fallback"],
"lifecycle_state": payload["lifecycle_state"],
"pending_draft": payload["pending_draft"],
"route": route,
"route_anchor": route_anchor,
"help_contexts": [
_semantic_help_context(entry),
],
},
)
)
return tuple(topics)
def _semantic_help_context(entry: object) -> str:
values = [
"semantic",
str(getattr(entry, "subject_module_id")),
str(getattr(entry, "subject_kind")),
str(getattr(entry, "subject_id")),
]
if getattr(entry, "anchor_kind", None) and getattr(entry, "anchor_id", None):
values.extend(
(
str(getattr(entry, "anchor_kind")),
str(getattr(entry, "anchor_id")),
)
)
return ".".join(values)
def _topic_matches_documentation_type(topic: DocumentationTopic, documentation_type: DocumentationType) -> bool: def _topic_matches_documentation_type(topic: DocumentationTopic, documentation_type: DocumentationType) -> bool:
return documentation_type in (topic.documentation_types or ("admin",)) return documentation_type in (topic.documentation_types or ("admin",))
@@ -0,0 +1,496 @@
from __future__ import annotations
from collections.abc import Callable
from typing import Annotated, Any
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response, status
from sqlalchemy.orm import Session
from govoplan_core.audit.logging import audit_from_principal
from govoplan_core.auth import ApiPrincipal, require_any_scope
from govoplan_core.core.semantic_documentation import (
SemanticDocumentationContractError,
SemanticDocumentationSubjectQuery,
SemanticDocumentationSubjectReference,
list_semantic_documentation_subjects,
)
from govoplan_core.db.session import get_session
from govoplan_docs.backend.manifest import (
DOCS_ADMIN_READ_SCOPE,
DOCS_READ_SCOPE,
DOCS_SEMANTIC_CREATE_SCOPE,
DOCS_SEMANTIC_EDIT_SCOPE,
DOCS_SEMANTIC_EXPORT_SCOPE,
DOCS_SEMANTIC_POLICY_SCOPE,
DOCS_SEMANTIC_PUBLISH_SCOPE,
DOCS_SEMANTIC_RETIRE_SCOPE,
DOCS_SEMANTIC_SUPERSEDE_SCOPE,
)
from govoplan_docs.backend.semantic_schemas import (
SemanticDocumentationCreateRequest,
SemanticDocumentationPolicyUpdateRequest,
SemanticDocumentationSupersedeRequest,
SemanticDocumentationTransitionRequest,
SemanticDocumentationUpdateRequest,
)
from govoplan_docs.backend.semantic_service import (
SemanticDocumentationAuthorizationError,
SemanticDocumentationConflictError,
SemanticDocumentationError,
SemanticDocumentationNotFoundError,
content_visible_to_principal,
create_semantic_entry,
get_semantic_entry,
list_semantic_entries,
publication_policy,
publish_semantic_entry,
retire_semantic_entry,
revision_payload,
select_locale_entries,
semantic_entry_history,
semantic_entry_payload,
set_publication_policy,
supersede_semantic_entry,
update_semantic_entry,
)
router = APIRouter(prefix="/semantic", tags=["docs-semantic"])
READ_SCOPES = (
DOCS_READ_SCOPE,
DOCS_ADMIN_READ_SCOPE,
DOCS_SEMANTIC_CREATE_SCOPE,
DOCS_SEMANTIC_EDIT_SCOPE,
DOCS_SEMANTIC_PUBLISH_SCOPE,
DOCS_SEMANTIC_SUPERSEDE_SCOPE,
DOCS_SEMANTIC_RETIRE_SCOPE,
)
EDITOR_SCOPES = READ_SCOPES[2:]
SessionDep = Annotated[Session, Depends(get_session)]
ReadPrincipal = Annotated[ApiPrincipal, Depends(require_any_scope(*READ_SCOPES))]
@router.get("/policy")
def get_policy(
response: Response,
session: SessionDep,
principal: ReadPrincipal,
) -> dict[str, str]:
_private(response)
return {"mode": publication_policy(session, principal.tenant_id)}
@router.put("/policy")
def update_policy(
payload: SemanticDocumentationPolicyUpdateRequest,
response: Response,
session: SessionDep,
principal: Annotated[
ApiPrincipal, Depends(require_any_scope(DOCS_SEMANTIC_POLICY_SCOPE))
],
) -> dict[str, str]:
_private(response)
try:
mode = set_publication_policy(
session,
tenant_id=principal.tenant_id,
mode=payload.mode,
)
_audit(session, principal, "docs.semantic.policy.updated", None, {"mode": mode})
session.commit()
return {"mode": mode}
except SemanticDocumentationError as exc:
session.rollback()
raise _http_error(exc) from exc
@router.get("/subjects")
def list_subjects(
request: Request,
response: Response,
session: SessionDep,
principal: ReadPrincipal,
query: str = Query(default="", max_length=300),
subject_kind: list[str] | None = Query(default=None, max_length=120),
module_id: str | None = Query(default=None, max_length=80),
limit: int = Query(default=50, ge=1, le=200),
cursor: str | None = Query(default=None, max_length=1000),
) -> dict[str, object]:
_private(response)
try:
pages = list_semantic_documentation_subjects(
_registry(request),
session,
principal,
request=SemanticDocumentationSubjectQuery(
tenant_id=principal.tenant_id,
query=query,
subject_kinds=tuple(subject_kind or ()),
limit=limit,
cursor=cursor,
),
)
except (SemanticDocumentationContractError, TypeError) as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
items = [
{
"module_id": provider_module_id,
"subjects": [item.to_dict() for item in page.subjects],
"next_cursor": page.next_cursor,
"has_more": page.has_more,
}
for provider_module_id, page in pages
if module_id is None or provider_module_id == module_id
]
return {"providers": items, "total": sum(len(item["subjects"]) for item in items)}
@router.get("/entries")
def list_entries(
request: Request,
response: Response,
session: SessionDep,
principal: ReadPrincipal,
locale: str = Query(default="de", min_length=2, max_length=20),
module_id: str | None = Query(default=None, max_length=80),
subject_kind: str | None = Query(default=None, max_length=120),
include_drafts: bool = Query(default=False),
) -> dict[str, object]:
_private(response)
editor = include_drafts and _has_any(principal, EDITOR_SCOPES)
entries = list_semantic_entries(
session,
principal,
module_id=module_id,
subject_kind=subject_kind,
)
selected = entries if editor else select_locale_entries(entries, locale=locale)
items = [
item
for entry in selected
if (
item := semantic_entry_payload(
session,
_registry(request),
principal,
entry=entry,
editor=editor,
requested_locale=locale,
)
)
is not None
]
return {"items": items, "total": len(items)}
@router.get("/entries/{entry_id}")
def inspect_entry(
entry_id: str,
request: Request,
response: Response,
session: SessionDep,
principal: ReadPrincipal,
include_draft: bool = Query(default=False),
) -> dict[str, object]:
_private(response)
try:
entry = get_semantic_entry(session, principal, entry_id=entry_id)
item = semantic_entry_payload(
session,
_registry(request),
principal,
entry=entry,
editor=include_draft and _has_any(principal, EDITOR_SCOPES),
)
if item is None:
raise SemanticDocumentationNotFoundError("Entry not found.")
return item
except SemanticDocumentationError as exc:
raise _http_error(exc) from exc
@router.get("/entries/{entry_id}/history")
def inspect_history(
entry_id: str,
response: Response,
session: SessionDep,
principal: Annotated[ApiPrincipal, Depends(require_any_scope(*EDITOR_SCOPES))],
) -> dict[str, object]:
_private(response)
try:
items = semantic_entry_history(session, principal, entry_id=entry_id)
return {"items": [revision_payload(item) for item in items], "total": len(items)}
except SemanticDocumentationError as exc:
raise _http_error(exc) from exc
@router.post("/entries", status_code=status.HTTP_201_CREATED)
def create_entry(
payload: SemanticDocumentationCreateRequest,
request: Request,
response: Response,
session: SessionDep,
principal: Annotated[
ApiPrincipal, Depends(require_any_scope(DOCS_SEMANTIC_CREATE_SCOPE))
],
) -> dict[str, object]:
return _mutate(
response,
session,
principal,
action="created",
callback=lambda: create_semantic_entry(
session,
_registry(request),
principal,
subject=SemanticDocumentationSubjectReference.from_mapping(
payload.subject.model_dump(mode="json")
),
locale=payload.locale,
content=payload.content.model_dump(mode="json"),
change_reason=payload.change_reason,
),
registry=_registry(request),
)
@router.put("/entries/{entry_id}")
def update_entry(
entry_id: str,
payload: SemanticDocumentationUpdateRequest,
request: Request,
response: Response,
session: SessionDep,
principal: Annotated[
ApiPrincipal, Depends(require_any_scope(DOCS_SEMANTIC_EDIT_SCOPE))
],
) -> dict[str, object]:
return _mutate(
response,
session,
principal,
action="updated",
callback=lambda: update_semantic_entry(
session,
_registry(request),
principal,
entry_id=entry_id,
expected_revision=payload.expected_revision,
content=payload.content.model_dump(mode="json"),
change_reason=payload.change_reason,
),
registry=_registry(request),
)
@router.post("/entries/{entry_id}/publish")
def publish_entry(
entry_id: str,
payload: SemanticDocumentationTransitionRequest,
request: Request,
response: Response,
session: SessionDep,
principal: Annotated[
ApiPrincipal, Depends(require_any_scope(DOCS_SEMANTIC_PUBLISH_SCOPE))
],
) -> dict[str, object]:
return _mutate(
response,
session,
principal,
action="published",
callback=lambda: publish_semantic_entry(
session,
_registry(request),
principal,
entry_id=entry_id,
expected_revision=payload.expected_revision,
change_reason=payload.change_reason,
),
registry=_registry(request),
)
@router.post("/entries/{entry_id}/supersede")
def supersede_entry(
entry_id: str,
payload: SemanticDocumentationSupersedeRequest,
request: Request,
response: Response,
session: SessionDep,
principal: Annotated[
ApiPrincipal, Depends(require_any_scope(DOCS_SEMANTIC_SUPERSEDE_SCOPE))
],
) -> dict[str, object]:
return _mutate(
response,
session,
principal,
action="superseded",
callback=lambda: supersede_semantic_entry(
session,
principal,
entry_id=entry_id,
replacement_entry_id=payload.replacement_entry_id,
expected_revision=payload.expected_revision,
change_reason=payload.change_reason,
),
registry=_registry(request),
)
@router.post("/entries/{entry_id}/retire")
def retire_entry(
entry_id: str,
payload: SemanticDocumentationTransitionRequest,
request: Request,
response: Response,
session: SessionDep,
principal: Annotated[
ApiPrincipal, Depends(require_any_scope(DOCS_SEMANTIC_RETIRE_SCOPE))
],
) -> dict[str, object]:
return _mutate(
response,
session,
principal,
action="retired",
callback=lambda: retire_semantic_entry(
session,
principal,
entry_id=entry_id,
expected_revision=payload.expected_revision,
change_reason=payload.change_reason,
),
registry=_registry(request),
)
@router.get("/export")
def export_tenant_semantics(
request: Request,
response: Response,
session: SessionDep,
principal: Annotated[
ApiPrincipal, Depends(require_any_scope(DOCS_SEMANTIC_EXPORT_SCOPE))
],
) -> dict[str, object]:
_private(response)
entries = list_semantic_entries(session, principal)
response.headers["Content-Disposition"] = (
'attachment; filename="govoplan-semantic-documentation.json"'
)
exported: list[dict[str, object]] = []
for entry in entries:
payload = semantic_entry_payload(
session,
_registry(request),
principal,
entry=entry,
editor=True,
)
if payload is None or payload.get("content_redacted"):
continue
exported.append(
{
"entry": payload,
"history": [
revision_payload(item)
for item in semantic_entry_history(
session, principal, entry_id=entry.id
)
if content_visible_to_principal(item.content, principal)
],
}
)
return {
"schema_version": "1",
"tenant_id": principal.tenant_id,
"entries": exported,
}
def _mutate(
response: Response,
session: Session,
principal: ApiPrincipal,
*,
action: str,
callback: Callable[[], Any],
registry: object,
) -> dict[str, object]:
_private(response)
try:
entry = callback()
_audit(
session,
principal,
f"docs.semantic.{action}",
entry.id,
{
"revision": entry.current_revision,
"subject_stable_key": entry.subject_stable_key,
"locale": entry.locale,
},
)
session.commit()
result = semantic_entry_payload(
session,
registry,
principal,
entry=entry,
editor=True,
)
if result is None:
raise SemanticDocumentationNotFoundError("Entry not found.")
return result
except (SemanticDocumentationError, SemanticDocumentationContractError) as exc:
session.rollback()
raise _http_error(exc) from exc
def _audit(
session: Session,
principal: ApiPrincipal,
action: str,
entry_id: str | None,
details: dict[str, object],
) -> None:
audit_from_principal(
session,
principal,
action=action,
object_type="semantic_documentation",
object_id=entry_id,
details=details,
)
def _private(response: Response) -> None:
response.headers["Cache-Control"] = "private, no-store"
def _registry(request: Request) -> object:
registry = getattr(request.app.state, "govoplan_registry", None)
if registry is None or not hasattr(registry, "capability_names"):
raise HTTPException(status_code=500, detail="Module registry is unavailable.")
return registry
def _has_any(principal: ApiPrincipal, scopes: tuple[str, ...]) -> bool:
return any(principal.has(scope) for scope in scopes)
def _http_error(exc: Exception) -> HTTPException:
if isinstance(exc, SemanticDocumentationNotFoundError):
return HTTPException(status_code=404, detail=str(exc))
if isinstance(exc, SemanticDocumentationAuthorizationError):
return HTTPException(status_code=403, detail=str(exc))
if isinstance(exc, SemanticDocumentationConflictError):
return HTTPException(status_code=409, detail=str(exc))
return HTTPException(status_code=422, detail=str(exc))
__all__ = ["router"]
+11
View File
@@ -0,0 +1,11 @@
"""Docs-owned persistence models."""
from govoplan_docs.backend.db.models import (
SemanticDocumentationEntry,
SemanticDocumentationRevision,
)
__all__ = [
"SemanticDocumentationEntry",
"SemanticDocumentationRevision",
]
+167
View File
@@ -0,0 +1,167 @@
from __future__ import annotations
import uuid
from datetime import datetime
from typing import Any
from sqlalchemy import (
Boolean,
DateTime,
ForeignKey,
Index,
Integer,
JSON,
String,
Text,
UniqueConstraint,
)
from sqlalchemy.orm import Mapped, mapped_column, relationship
from govoplan_core.db.base import Base, TimestampMixin
def new_uuid() -> str:
return str(uuid.uuid4())
class SemanticDocumentationEntry(Base, TimestampMixin):
__tablename__ = "docs_semantic_entries"
__table_args__ = (
UniqueConstraint(
"tenant_id",
"subject_stable_key",
"locale",
name="uq_docs_semantic_entry_subject_locale",
),
Index(
"ix_docs_semantic_entries_tenant_state",
"tenant_id",
"lifecycle_state",
),
Index(
"ix_docs_semantic_entries_subject",
"tenant_id",
"subject_module_id",
"subject_kind",
"subject_id",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
subject_stable_key: Mapped[str] = mapped_column(
String(80), nullable=False, index=True
)
subject_module_id: Mapped[str] = mapped_column(
String(80), nullable=False, index=True
)
subject_kind: Mapped[str] = mapped_column(String(120), nullable=False, index=True)
subject_id: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
anchor_kind: Mapped[str | None] = mapped_column(
String(120), nullable=True, index=True
)
anchor_id: Mapped[str | None] = mapped_column(
String(255), nullable=True, index=True
)
locale: Mapped[str] = mapped_column(String(20), nullable=False, index=True)
lifecycle_state: Mapped[str] = mapped_column(
String(30), nullable=False, default="draft", index=True
)
current_revision: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
current_revision_id: Mapped[str | None] = mapped_column(
String(36), nullable=True, index=True
)
published_revision_id: Mapped[str | None] = mapped_column(
String(36), nullable=True, index=True
)
superseded_by_entry_id: Mapped[str | None] = mapped_column(
ForeignKey("docs_semantic_entries.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
created_by: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
updated_by: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
published_by: Mapped[str | None] = mapped_column(
String(255), nullable=True, index=True
)
published_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, index=True
)
retired_by: Mapped[str | None] = mapped_column(
String(255), nullable=True, index=True
)
retired_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, index=True
)
revisions: Mapped[list["SemanticDocumentationRevision"]] = relationship(
back_populates="entry",
cascade="all, delete-orphan",
foreign_keys="SemanticDocumentationRevision.entry_id",
order_by="SemanticDocumentationRevision.revision",
)
superseded_by: Mapped["SemanticDocumentationEntry | None"] = relationship(
remote_side="SemanticDocumentationEntry.id",
foreign_keys=[superseded_by_entry_id],
)
class SemanticDocumentationRevision(Base, TimestampMixin):
__tablename__ = "docs_semantic_revisions"
__table_args__ = (
UniqueConstraint(
"entry_id",
"revision",
name="uq_docs_semantic_revision_number",
),
Index(
"ix_docs_semantic_revisions_entry",
"entry_id",
"revision",
),
Index(
"ix_docs_semantic_revisions_tenant_state",
"tenant_id",
"lifecycle_state",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
entry_id: Mapped[str] = mapped_column(
ForeignKey("docs_semantic_entries.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
revision: Mapped[int] = mapped_column(Integer, nullable=False)
lifecycle_state: Mapped[str] = mapped_column(String(30), nullable=False)
action: Mapped[str] = mapped_column(String(30), nullable=False)
change_reason: Mapped[str] = mapped_column(String(1000), nullable=False)
content: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
content_hash: Mapped[str] = mapped_column(String(80), nullable=False, index=True)
subject_revision: Mapped[str | None] = mapped_column(String(255), nullable=True)
subject_fingerprint: Mapped[str | None] = mapped_column(
String(80), nullable=True, index=True
)
authored_by: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
reviewed_by: Mapped[str | None] = mapped_column(
String(255), nullable=True, index=True
)
published_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, index=True
)
provenance: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict)
recoverable: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
search_text: Mapped[str] = mapped_column(Text, nullable=False, default="")
entry: Mapped[SemanticDocumentationEntry] = relationship(
back_populates="revisions",
foreign_keys=[entry_id],
)
__all__ = [
"SemanticDocumentationEntry",
"SemanticDocumentationRevision",
"new_uuid",
]
+201
View File
@@ -0,0 +1,201 @@
from __future__ import annotations
from collections.abc import Sequence
from datetime import UTC, datetime
from sqlalchemy.orm import Session
from govoplan_core.core.dsar import (
DsarErasureActionRef,
DsarExecutionResultRef,
DsarRecordRef,
DsarSubjectRef,
dsar_capability_name,
)
from govoplan_docs.backend.db.models import (
SemanticDocumentationEntry,
SemanticDocumentationRevision,
)
DOCS_DSAR_CAPABILITY = dsar_capability_name("docs")
_MAX_REVISIONS = 5_000
class DocsDsarProvider:
provider_id = "docs"
module_id = "docs"
def search_subject(
self,
session: object,
*,
tenant_id: str,
subject: DsarSubjectRef,
) -> Sequence[DsarRecordRef]:
db = _session(session)
identifiers = _identifiers(subject)
if not identifiers:
return ()
revisions = (
db.query(SemanticDocumentationRevision)
.filter(SemanticDocumentationRevision.tenant_id == tenant_id)
.order_by(SemanticDocumentationRevision.entry_id, SemanticDocumentationRevision.revision)
.limit(_MAX_REVISIONS + 1)
.all()
)
if len(revisions) > _MAX_REVISIONS:
raise ValueError(
"Docs DSAR revision limit exceeded; use an exact semantic-entry reference."
)
explicit_entry = subject.external_references.get("docs.semantic_entry")
entries = {
row.id: row
for row in db.query(SemanticDocumentationEntry).filter(
SemanticDocumentationEntry.tenant_id == tenant_id
)
}
records: list[DsarRecordRef] = []
for revision in revisions:
entry = entries.get(revision.entry_id)
if entry is None or (explicit_entry and entry.id != explicit_entry):
continue
matches = _matches(entry, revision, identifiers)
if not matches and not explicit_entry:
continue
immutable = revision.lifecycle_state in {
"published",
"superseded",
"retired",
}
records.append(
DsarRecordRef(
provider_id=self.provider_id,
module_id=self.module_id,
resource_type="semantic_documentation_revision",
resource_id=revision.id,
category="configured_semantic_documentation_attribution",
title="Semantic documentation attribution",
data={
"entry_id": entry.id,
"revision": revision.revision,
"lifecycle_state": revision.lifecycle_state,
"subject_module_id": entry.subject_module_id,
"subject_kind": entry.subject_kind,
"subject_id": entry.subject_id,
"locale": entry.locale,
"matching_reference_fields": matches,
},
observed_at=_aware(revision.created_at),
immutable_evidence=immutable,
retention_reason=(
"Published semantic-documentation authorship and review history "
"is retained as configuration-governance evidence."
if immutable
else None
),
source_path=f"/docs/semantic?entryId={entry.id}",
)
)
return tuple(records)
def plan_erasure(
self,
session: object,
*,
tenant_id: str,
subject: DsarSubjectRef,
records: Sequence[DsarRecordRef],
) -> Sequence[DsarErasureActionRef]:
del session, tenant_id, subject
return tuple(
DsarErasureActionRef(
action_id=f"docs:retain:{record.resource_id}",
provider_id=self.provider_id,
module_id=self.module_id,
kind="retain" if record.immutable_evidence else "manual_review",
resource_type=record.resource_type,
resource_id=record.resource_id,
title="Review semantic documentation attribution",
rationale=(
record.retention_reason
or "Draft attribution may be anonymized only after a configurator "
"confirms that ownership and stewardship remain accountable."
),
executable=False,
metadata={"immutable_evidence": record.immutable_evidence},
)
for record in records
)
def execute_erasure(
self,
session: object,
*,
tenant_id: str,
subject: DsarSubjectRef,
actions: Sequence[DsarErasureActionRef],
request_id: str,
) -> Sequence[DsarExecutionResultRef]:
del session, tenant_id, subject
return tuple(
DsarExecutionResultRef(
action_id=action.action_id,
status="blocked",
summary=(
"Docs semantic attribution requires governed manual review and "
"was not changed automatically."
),
evidence={"request_id": request_id},
)
for action in actions
)
def _identifiers(subject: DsarSubjectRef) -> frozenset[str]:
return frozenset(
str(value).strip()
for value in (
subject.account_id,
subject.identity_id,
subject.membership_id,
subject.external_references.get("access.account"),
)
if str(value or "").strip()
)
def _matches(
entry: SemanticDocumentationEntry,
revision: SemanticDocumentationRevision,
identifiers: frozenset[str],
) -> list[str]:
fields = {
"entry.created_by": entry.created_by,
"entry.updated_by": entry.updated_by,
"entry.published_by": entry.published_by,
"entry.retired_by": entry.retired_by,
"revision.authored_by": revision.authored_by,
"revision.reviewed_by": revision.reviewed_by,
"content.owner_account_id": revision.content.get("owner_account_id"),
"content.steward_account_id": revision.content.get("steward_account_id"),
}
return sorted(
field
for field, value in fields.items()
if str(value or "").strip() in identifiers
)
def _aware(value: datetime) -> datetime:
return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
def _session(value: object) -> Session:
if not isinstance(value, Session):
raise TypeError("Docs DSAR requires a SQLAlchemy session.")
return value
__all__ = ["DOCS_DSAR_CAPABILITY", "DocsDsarProvider"]
+261 -3
View File
@@ -1,30 +1,53 @@
from __future__ import annotations from __future__ import annotations
from pathlib import Path
from govoplan_core.core.access import ( from govoplan_core.core.access import (
CAPABILITY_AUTH_PERMISSION_EVALUATOR, CAPABILITY_AUTH_PERMISSION_EVALUATOR,
CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PRINCIPAL_RESOLVER,
) )
from govoplan_core.core.modules import ( from govoplan_core.core.modules import (
CapabilityDocumentation,
DocumentationCondition, DocumentationCondition,
DocumentationLink, DocumentationLink,
DocumentationSourceDefinition, DocumentationSourceDefinition,
DocumentationTopic, DocumentationTopic,
FrontendModule, FrontendModule,
FrontendRoute, FrontendRoute,
MigrationSpec,
ModuleContext, ModuleContext,
ModuleManifest, ModuleManifest,
ModuleInterfaceProvider,
NavItem, NavItem,
PermissionDefinition, PermissionDefinition,
RoleTemplate, RoleTemplate,
) )
from govoplan_core.core.module_guards import (
drop_table_retirement_provider,
persistent_table_uninstall_guard,
)
from govoplan_core.core.provider_governance import ( from govoplan_core.core.provider_governance import (
ModuleArchitectureDeclaration, ModuleArchitectureDeclaration,
ModuleArchitectureDocumentation, ModuleArchitectureDocumentation,
ModuleMaturityEvidence, ModuleMaturityEvidence,
) )
from govoplan_core.db.base import Base
from govoplan_core.core.search import SearchSourceProviderRegistration
from govoplan_docs.backend.db import models as docs_models
from govoplan_docs.backend.dsar_provider import DOCS_DSAR_CAPABILITY, DocsDsarProvider
from govoplan_docs.backend.search_source import (
create_semantic_documentation_search_source,
)
DOCS_READ_SCOPE = "docs:documentation:read" DOCS_READ_SCOPE = "docs:documentation:read"
DOCS_ADMIN_READ_SCOPE = "docs:documentation:admin" DOCS_ADMIN_READ_SCOPE = "docs:documentation:admin"
DOCS_SEMANTIC_CREATE_SCOPE = "docs:semantic:create"
DOCS_SEMANTIC_EDIT_SCOPE = "docs:semantic:edit"
DOCS_SEMANTIC_PUBLISH_SCOPE = "docs:semantic:publish"
DOCS_SEMANTIC_SUPERSEDE_SCOPE = "docs:semantic:supersede"
DOCS_SEMANTIC_RETIRE_SCOPE = "docs:semantic:retire"
DOCS_SEMANTIC_EXPORT_SCOPE = "docs:semantic:export"
DOCS_SEMANTIC_POLICY_SCOPE = "docs:semantic:policy"
DOCS_ADMIN_READ_SCOPES = ( DOCS_ADMIN_READ_SCOPES = (
DOCS_ADMIN_READ_SCOPE, DOCS_ADMIN_READ_SCOPE,
"system:settings:read", "system:settings:read",
@@ -42,6 +65,11 @@ ARCHITECTURE = ModuleArchitectureDeclaration(
reference="tests/test_docs_context.py", reference="tests/test_docs_context.py",
summary="Tests audience-safe configured documentation and architecture projections.", summary="Tests audience-safe configured documentation and architecture projections.",
), ),
ModuleMaturityEvidence(
kind="test",
reference="tests/test_semantic_documentation.py",
summary="Tests tenant isolation, immutable revision lifecycle, publication policy, subject reauthorization, search, localization, and DSAR projection.",
),
ModuleMaturityEvidence( ModuleMaturityEvidence(
kind="documentation", kind="documentation",
reference="docs/DOCUMENTATION_LAYER_CONCEPT.md", reference="docs/DOCUMENTATION_LAYER_CONCEPT.md",
@@ -52,11 +80,21 @@ ARCHITECTURE = ModuleArchitectureDeclaration(
reference="docs/INTERFACE_PATTERN_MIGRATION.md", reference="docs/INTERFACE_PATTERN_MIGRATION.md",
summary="Records the Core-owned Docs workspace and page-layout boundary.", summary="Records the Core-owned Docs workspace and page-layout boundary.",
), ),
ModuleMaturityEvidence(
kind="documentation",
reference="docs/SEMANTIC_DOCUMENTATION.md",
summary="Defines semantic authoring, authorization, lifecycle, export, and recovery behavior.",
),
), ),
known_limits=( known_limits=(
"Architecture declarations are in staged adoption, so undeclared modules remain visible as pending.", "Architecture declarations are in staged adoption, so undeclared modules remain visible as pending.",
), ),
owned_concepts=("configured documentation projection", "documentation audience filtering"), owned_concepts=(
"configured documentation projection",
"documentation audience filtering",
"tenant semantic documentation revisions",
"semantic documentation publication lifecycle",
),
non_owned_concepts=("module feature behavior", "module evidence generation"), non_owned_concepts=("module feature behavior", "module evidence generation"),
documentation=ModuleArchitectureDocumentation( documentation=ModuleArchitectureDocumentation(
security=("docs/DOCUMENTATION_LAYER_CONCEPT.md",), security=("docs/DOCUMENTATION_LAYER_CONCEPT.md",),
@@ -86,21 +124,82 @@ def _route_factory(context: ModuleContext):
return router return router
def _dsar_provider(_context: ModuleContext) -> DocsDsarProvider:
return DocsDsarProvider()
manifest = ModuleManifest( manifest = ModuleManifest(
id="docs", id="docs",
name="Docs", name="Docs",
version="0.1.18", version="0.1.19",
required_capabilities=( required_capabilities=(
CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PRINCIPAL_RESOLVER,
CAPABILITY_AUTH_PERMISSION_EVALUATOR, CAPABILITY_AUTH_PERMISSION_EVALUATOR,
), ),
optional_dependencies=("policy", "audit", "ops", "workflow_engine", "search"), provides_interfaces=(
ModuleInterfaceProvider(name=DOCS_DSAR_CAPABILITY, version="0.1.0"),
),
capability_factories={DOCS_DSAR_CAPABILITY: _dsar_provider},
capability_documentation={
DOCS_DSAR_CAPABILITY: CapabilityDocumentation(
label="Docs data-subject request provider",
summary=(
"Exports minimized tenant semantic-documentation authorship, "
"review, ownership, and stewardship references while retaining "
"published configuration-governance evidence."
),
contract_version="0.1.0",
),
},
optional_dependencies=(
"policy",
"audit",
"ops",
"workflow",
"forms",
"search",
),
permissions=( permissions=(
_permission( _permission(
DOCS_READ_SCOPE, DOCS_READ_SCOPE,
"View configured documentation", "View configured documentation",
"Read user documentation generated for the current actor from installed modules and effective configuration.", "Read user documentation generated for the current actor from installed modules and effective configuration.",
), ),
_permission(
DOCS_SEMANTIC_CREATE_SCOPE,
"Create semantic documentation",
"Create tenant-owned semantic documentation for configured subjects.",
),
_permission(
DOCS_SEMANTIC_EDIT_SCOPE,
"Edit semantic documentation",
"Edit drafts using immutable revisions and optimistic concurrency.",
),
_permission(
DOCS_SEMANTIC_PUBLISH_SCOPE,
"Publish semantic documentation",
"Review and publish semantic documentation under tenant policy.",
),
_permission(
DOCS_SEMANTIC_SUPERSEDE_SCOPE,
"Supersede semantic documentation",
"Replace semantic documentation with another published entry.",
),
_permission(
DOCS_SEMANTIC_RETIRE_SCOPE,
"Retire semantic documentation",
"Retire semantic documentation while retaining its revision history.",
),
_permission(
DOCS_SEMANTIC_EXPORT_SCOPE,
"Export tenant semantic documentation",
"Export tenant-owned semantic entries and immutable history.",
),
_permission(
DOCS_SEMANTIC_POLICY_SCOPE,
"Configure semantic publication policy",
"Choose direct publication or independent reviewer publication.",
),
_permission( _permission(
DOCS_ADMIN_READ_SCOPE, DOCS_ADMIN_READ_SCOPE,
"View administrative documentation", "View administrative documentation",
@@ -115,6 +214,43 @@ manifest = ModuleManifest(
permissions=(DOCS_READ_SCOPE,), permissions=(DOCS_READ_SCOPE,),
default_authenticated=True, default_authenticated=True,
), ),
RoleTemplate(
slug="semantic_documentation_author",
name="Semantic documentation author",
description="Discover configured subjects and create or revise their semantic documentation.",
permissions=(
DOCS_READ_SCOPE,
DOCS_SEMANTIC_CREATE_SCOPE,
DOCS_SEMANTIC_EDIT_SCOPE,
),
),
RoleTemplate(
slug="semantic_documentation_reviewer",
name="Semantic documentation reviewer",
description="Review, publish, supersede, and retire semantic documentation.",
permissions=(
DOCS_READ_SCOPE,
DOCS_SEMANTIC_PUBLISH_SCOPE,
DOCS_SEMANTIC_SUPERSEDE_SCOPE,
DOCS_SEMANTIC_RETIRE_SCOPE,
),
),
RoleTemplate(
slug="semantic_documentation_manager",
name="Semantic documentation manager",
description="Administer semantic authoring, review, lifecycle, policy, and tenant export.",
permissions=(
DOCS_READ_SCOPE,
DOCS_ADMIN_READ_SCOPE,
DOCS_SEMANTIC_CREATE_SCOPE,
DOCS_SEMANTIC_EDIT_SCOPE,
DOCS_SEMANTIC_PUBLISH_SCOPE,
DOCS_SEMANTIC_SUPERSEDE_SCOPE,
DOCS_SEMANTIC_RETIRE_SCOPE,
DOCS_SEMANTIC_EXPORT_SCOPE,
DOCS_SEMANTIC_POLICY_SCOPE,
),
),
RoleTemplate( RoleTemplate(
slug="docs_admin", slug="docs_admin",
name="Documentation administrator", name="Documentation administrator",
@@ -142,6 +278,18 @@ manifest = ModuleManifest(
required_any=DOCS_READ_SCOPES, required_any=DOCS_READ_SCOPES,
order=880, order=880,
), ),
FrontendRoute(
path="/docs/semantic",
component="SemanticDocumentationPage",
required_any=(
DOCS_SEMANTIC_CREATE_SCOPE,
DOCS_SEMANTIC_EDIT_SCOPE,
DOCS_SEMANTIC_PUBLISH_SCOPE,
DOCS_SEMANTIC_SUPERSEDE_SCOPE,
DOCS_SEMANTIC_RETIRE_SCOPE,
),
order=881,
),
), ),
nav_items=( nav_items=(
NavItem( NavItem(
@@ -154,6 +302,88 @@ manifest = ModuleManifest(
), ),
), ),
documentation=( documentation=(
DocumentationTopic(
id="docs.semantic-documentation",
title="Tenant semantic documentation",
summary="Explain what configured forms, fields, workflows, steps, and other stable subjects mean in this tenant.",
body=(
"Authors select an authorized subject supplied by its owning module and create locale-specific plain-text guidance. "
"Every save creates an immutable revision. Tenant policy chooses direct publication or an independent reviewer. "
"Published content remains subject to the subject's current authorization, the documentation audience and classification, tenant isolation, and locale selection. "
"Changed, missing, superseded, or temporarily unavailable subjects are shown explicitly; direct links, contextual help, search, caches, and tenant exports apply the same read-time authorization. "
"Retirement and supersession preserve history. Generic public documentation exports never include tenant semantic entries; administrators use the separately authorized tenant export."
),
layer="always",
documentation_types=("admin", "user"),
audience=("tenant_admin", "module_admin", "documentation_author"),
order=11,
conditions=(
DocumentationCondition(
any_scopes=(
DOCS_SEMANTIC_CREATE_SCOPE,
DOCS_SEMANTIC_EDIT_SCOPE,
DOCS_SEMANTIC_PUBLISH_SCOPE,
DOCS_SEMANTIC_POLICY_SCOPE,
),
),
),
links=(
DocumentationLink(
label="Semantic documentation administration",
href="/docs/semantic",
kind="runtime",
),
DocumentationLink(
label="Semantic documentation operations",
href="govoplan-docs/docs/SEMANTIC_DOCUMENTATION.md",
kind="repository",
),
),
metadata={"kind": "workflow"},
),
DocumentationTopic(
id="docs.data-subject-requests",
title="Review Docs semantic attribution in a data-subject request",
summary="Export minimized author, reviewer, owner, and steward references without disclosing unrelated tenant-authored guidance.",
body=(
"Docs matches exact account and namespaced semantic-entry references within the active tenant. "
"The projection identifies the entry, revision, subject, locale, lifecycle state, and fields that matched, but excludes authored body content. "
"Published, superseded, and retired attribution is immutable configuration-governance evidence and is retained with a reason. "
"Draft attribution requires manual governance review so ownership or stewardship can be reassigned before any anonymization; Docs performs no automatic erasure."
),
layer="configured",
documentation_types=("admin",),
audience=("privacy_officer", "documentation_administrator", "operator"),
order=12,
conditions=(
DocumentationCondition(
required_modules=("docs", "access"),
any_scopes=(
"access:privacy:read",
"access:privacy:manage",
"access:privacy:erase",
),
),
),
links=(
DocumentationLink(
label="Data-subject requests",
href="/admin?section=tenant-data-subject-requests",
kind="runtime",
),
DocumentationLink(
label="Semantic documentation operations",
href="govoplan-docs/docs/SEMANTIC_DOCUMENTATION.md",
kind="repository",
),
),
related_modules=("access", "audit", "policy"),
metadata={
"kind": "workflow",
"route": "/admin?section=tenant-data-subject-requests",
"help_contexts": ["admin.privacy.data-subject-requests"],
},
),
DocumentationTopic( DocumentationTopic(
id="docs.configured-system-documentation", id="docs.configured-system-documentation",
title="Configured system documentation", title="Configured system documentation",
@@ -434,6 +664,34 @@ manifest = ModuleManifest(
}, },
), ),
), ),
migration_spec=MigrationSpec(
module_id="docs",
metadata=Base.metadata,
script_location=str(Path(__file__).with_name("migrations") / "versions"),
retirement_supported=True,
retirement_provider=drop_table_retirement_provider(
docs_models.SemanticDocumentationRevision,
docs_models.SemanticDocumentationEntry,
label="Docs semantic documentation",
),
retirement_notes=(
"Destructive retirement removes tenant semantic entries and immutable "
"revision history after the installer captures a database snapshot."
),
),
uninstall_guard_providers=(
persistent_table_uninstall_guard(
docs_models.SemanticDocumentationEntry,
docs_models.SemanticDocumentationRevision,
label="Docs semantic documentation",
),
),
search_sources=(
SearchSourceProviderRegistration(
id="docs.semantic_documentation",
factory=create_semantic_documentation_search_source,
),
),
architecture=ARCHITECTURE, architecture=ARCHITECTURE,
) )
@@ -0,0 +1 @@
"""Docs-owned database migrations."""
@@ -0,0 +1 @@
"""Docs migration revisions."""
@@ -0,0 +1,170 @@
"""Add tenant semantic documentation and immutable revisions.
Revision ID: d3e7a1c5f9b2
Revises: None
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d3e7a1c5f9b2"
down_revision = None
branch_labels = None
depends_on = "4f2a9c8e7b6d"
def upgrade() -> None:
op.create_table(
"docs_semantic_entries",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=255), nullable=False),
sa.Column("subject_stable_key", sa.String(length=80), nullable=False),
sa.Column("subject_module_id", sa.String(length=80), nullable=False),
sa.Column("subject_kind", sa.String(length=120), nullable=False),
sa.Column("subject_id", sa.String(length=255), nullable=False),
sa.Column("anchor_kind", sa.String(length=120), nullable=True),
sa.Column("anchor_id", sa.String(length=255), nullable=True),
sa.Column("locale", sa.String(length=20), nullable=False),
sa.Column("lifecycle_state", sa.String(length=30), nullable=False),
sa.Column("current_revision", sa.Integer(), nullable=False),
sa.Column("current_revision_id", sa.String(length=36), nullable=True),
sa.Column("published_revision_id", sa.String(length=36), nullable=True),
sa.Column("superseded_by_entry_id", sa.String(length=36), nullable=True),
sa.Column("created_by", sa.String(length=255), nullable=False),
sa.Column("updated_by", sa.String(length=255), nullable=False),
sa.Column("published_by", sa.String(length=255), nullable=True),
sa.Column("published_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("retired_by", sa.String(length=255), nullable=True),
sa.Column("retired_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["superseded_by_entry_id"],
["docs_semantic_entries.id"],
name=op.f(
"fk_docs_semantic_entries_superseded_by_entry_id_docs_semantic_entries"
),
ondelete="SET NULL",
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_docs_semantic_entries")),
sa.UniqueConstraint(
"tenant_id",
"subject_stable_key",
"locale",
name="uq_docs_semantic_entry_subject_locale",
),
)
_entry_indexes()
op.create_table(
"docs_semantic_revisions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=255), nullable=False),
sa.Column("entry_id", sa.String(length=36), nullable=False),
sa.Column("revision", sa.Integer(), nullable=False),
sa.Column("lifecycle_state", sa.String(length=30), nullable=False),
sa.Column("action", sa.String(length=30), nullable=False),
sa.Column("change_reason", sa.String(length=1000), nullable=False),
sa.Column("content", sa.JSON(), nullable=False),
sa.Column("content_hash", sa.String(length=80), nullable=False),
sa.Column("subject_revision", sa.String(length=255), nullable=True),
sa.Column("subject_fingerprint", sa.String(length=80), nullable=True),
sa.Column("authored_by", sa.String(length=255), nullable=False),
sa.Column("reviewed_by", sa.String(length=255), nullable=True),
sa.Column("published_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("provenance", sa.JSON(), nullable=False),
sa.Column("recoverable", sa.Boolean(), nullable=False),
sa.Column("search_text", sa.Text(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["entry_id"],
["docs_semantic_entries.id"],
name=op.f("fk_docs_semantic_revisions_entry_id_docs_semantic_entries"),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_docs_semantic_revisions")),
sa.UniqueConstraint(
"entry_id",
"revision",
name="uq_docs_semantic_revision_number",
),
)
_revision_indexes()
def downgrade() -> None:
op.drop_table("docs_semantic_revisions")
op.drop_table("docs_semantic_entries")
def _entry_indexes() -> None:
for column in (
"tenant_id",
"subject_stable_key",
"subject_module_id",
"subject_kind",
"subject_id",
"anchor_kind",
"anchor_id",
"locale",
"lifecycle_state",
"current_revision_id",
"published_revision_id",
"superseded_by_entry_id",
"created_by",
"updated_by",
"published_by",
"published_at",
"retired_by",
"retired_at",
):
op.create_index(
op.f(f"ix_docs_semantic_entries_{column}"),
"docs_semantic_entries",
[column],
unique=False,
)
op.create_index(
"ix_docs_semantic_entries_tenant_state",
"docs_semantic_entries",
["tenant_id", "lifecycle_state"],
unique=False,
)
op.create_index(
"ix_docs_semantic_entries_subject",
"docs_semantic_entries",
["tenant_id", "subject_module_id", "subject_kind", "subject_id"],
unique=False,
)
def _revision_indexes() -> None:
for column in (
"tenant_id",
"entry_id",
"content_hash",
"subject_fingerprint",
"authored_by",
"reviewed_by",
"published_at",
):
op.create_index(
op.f(f"ix_docs_semantic_revisions_{column}"),
"docs_semantic_revisions",
[column],
unique=False,
)
op.create_index(
"ix_docs_semantic_revisions_entry",
"docs_semantic_revisions",
["entry_id", "revision"],
unique=False,
)
op.create_index(
"ix_docs_semantic_revisions_tenant_state",
"docs_semantic_revisions",
["tenant_id", "lifecycle_state"],
unique=False,
)
+263
View File
@@ -0,0 +1,263 @@
from __future__ import annotations
from collections.abc import Mapping, Sequence
from urllib.parse import quote
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.events import PlatformEvent
from govoplan_core.core.modules import ModuleContext
from govoplan_core.core.search import (
SearchAuthorizationRequest,
SearchBackfillPage,
SearchBackfillRequest,
SearchDocument,
SearchIndexChange,
SearchResourceReference,
SearchResourceType,
)
from govoplan_docs.backend.db.models import (
SemanticDocumentationEntry,
SemanticDocumentationRevision,
)
from govoplan_docs.backend.semantic_service import semantic_entry_payload
PROVIDER_ID = "docs.semantic_documentation"
RESOURCE_TYPE = "semantic_documentation"
DOCS_READ_SCOPE = "docs:documentation:read"
class SemanticDocumentationSearchSource:
def __init__(self, registry: object) -> None:
self._registry = registry
def resource_types(self) -> Sequence[SearchResourceType]:
return (
SearchResourceType(
provider_id=PROVIDER_ID,
module_id="docs",
resource_type=RESOURCE_TYPE,
label="Semantic documentation",
requires_authorization_recheck=True,
),
)
def backfill(
self,
session: object,
*,
request: SearchBackfillRequest,
) -> SearchBackfillPage:
_assert_source(request.provider_id, request.resource_type)
db = _session(session)
statement = (
select(SemanticDocumentationEntry, SemanticDocumentationRevision)
.join(
SemanticDocumentationRevision,
SemanticDocumentationRevision.id
== SemanticDocumentationEntry.published_revision_id,
)
.where(
SemanticDocumentationEntry.tenant_id == request.tenant_id,
SemanticDocumentationEntry.lifecycle_state.in_(("draft", "published")),
)
)
if request.cursor:
statement = statement.where(
SemanticDocumentationEntry.id > request.cursor
)
rows = list(
db.execute(
statement.order_by(SemanticDocumentationEntry.id).limit(
request.limit + 1
)
).all()
)
has_more = len(rows) > request.limit
selected = rows[: request.limit]
high_watermark = db.scalar(
select(func.max(SemanticDocumentationEntry.updated_at)).where(
SemanticDocumentationEntry.tenant_id == request.tenant_id,
SemanticDocumentationEntry.published_revision_id.is_not(None),
)
)
return SearchBackfillPage(
documents=tuple(_document(entry, revision) for entry, revision in selected),
next_cursor=selected[-1][0].id if has_more and selected else None,
complete=not has_more,
high_watermark=(
high_watermark.isoformat() if high_watermark is not None else None
),
)
def authorize(
self,
session: object,
principal: object,
*,
requests: Sequence[SearchAuthorizationRequest],
) -> Mapping[str, bool]:
decisions = {item.reference.key: False for item in requests}
if not isinstance(principal, ApiPrincipal) or not principal.has(DOCS_READ_SCOPE):
return decisions
db = _session(session)
for item in requests:
reference = item.reference
if (
reference.tenant_id != principal.tenant_id
or reference.module_id != "docs"
or reference.resource_type != RESOURCE_TYPE
):
continue
entry = db.get(SemanticDocumentationEntry, reference.resource_id)
if (
entry is None
or entry.tenant_id != principal.tenant_id
or entry.published_revision_id != item.source_revision
or entry.lifecycle_state not in {"draft", "published"}
):
continue
payload = semantic_entry_payload(
db,
self._registry,
principal,
entry=entry,
editor=False,
)
decisions[reference.key] = bool(
payload
and payload["subject_resolution"]["availability"]
in {"available", "changed"}
)
return decisions
def index_changes_for_event(
self,
session: object,
*,
event: PlatformEvent,
delivery_key: str,
) -> Sequence[SearchIndexChange]:
if (
event.module_id != "docs"
or event.tenant is None
or event.resource is None
or event.resource.type != RESOURCE_TYPE
or event.resource.id is None
):
return ()
db = _session(session)
entry = db.get(SemanticDocumentationEntry, event.resource.id)
revision = (
db.get(SemanticDocumentationRevision, entry.published_revision_id)
if entry is not None and entry.published_revision_id
else None
)
visible = bool(
entry is not None
and entry.tenant_id == event.tenant.id
and entry.lifecycle_state in {"draft", "published"}
and revision is not None
)
cursor = event.event_id
document = _document(entry, revision, change_cursor=cursor) if visible else None
reference = SearchResourceReference(
tenant_id=event.tenant.id,
module_id="docs",
resource_type=RESOURCE_TYPE,
resource_id=event.resource.id,
)
return (
SearchIndexChange(
change_id=f"{delivery_key}:{PROVIDER_ID}",
provider_id=PROVIDER_ID,
kind="upsert" if document is not None else "delete",
reference=reference,
source_revision=(
document.source_revision if document is not None else cursor
),
cursor=cursor,
document=document,
occurred_at=event.occurred_at,
),
)
def create_semantic_documentation_search_source(
context: ModuleContext,
) -> SemanticDocumentationSearchSource:
return SemanticDocumentationSearchSource(context.registry)
def _document(
entry: SemanticDocumentationEntry,
revision: SemanticDocumentationRevision,
*,
change_cursor: str | None = None,
) -> SearchDocument:
content = revision.content
title = str(content.get("title") or "Semantic documentation")
summary = str(content.get("summary") or "")[:4000] or None
keywords = tuple(
str(value)[:200]
for value in (
entry.subject_module_id,
entry.subject_kind,
entry.locale,
"semantic documentation",
)
)
return SearchDocument(
tenant_id=entry.tenant_id,
module_id="docs",
provider_id=PROVIDER_ID,
resource_type=RESOURCE_TYPE,
resource_id=entry.id,
title=title,
url=(
f"/docs/semantic?entryId={quote(entry.id, safe='')}"
f"&locale={quote(entry.locale, safe='')}"
),
summary=summary,
body=revision.search_text[:200_000] or None,
keywords=keywords,
visibility="restricted",
acl_tokens=(f"scope:{DOCS_READ_SCOPE}",),
metadata={
"source_badge": "tenant_semantic",
"subject_module_id": entry.subject_module_id,
"subject_kind": entry.subject_kind,
"subject_id": entry.subject_id,
"anchor_kind": entry.anchor_kind,
"anchor_id": entry.anchor_id,
"locale": entry.locale,
"classification": content.get("classification", "internal"),
},
source_revision=revision.id,
change_cursor=change_cursor,
source_updated_at=entry.updated_at or entry.created_at,
requires_authorization_recheck=True,
)
def _assert_source(provider_id: str, resource_type: str) -> None:
if provider_id != PROVIDER_ID or resource_type != RESOURCE_TYPE:
raise ValueError("Unsupported Docs search source.")
def _session(value: object) -> Session:
if not isinstance(value, Session):
raise TypeError("Semantic documentation search requires a SQLAlchemy session.")
return value
__all__ = [
"PROVIDER_ID",
"RESOURCE_TYPE",
"SemanticDocumentationSearchSource",
"create_semantic_documentation_search_source",
]
@@ -0,0 +1,137 @@
from __future__ import annotations
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
SemanticLifecycleState = Literal["draft", "published", "superseded", "retired"]
SemanticSubjectAvailability = Literal[
"available",
"changed",
"superseded",
"missing",
"temporarily_unavailable",
]
class _StrictModel(BaseModel):
model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
class SemanticSubjectAnchorPayload(_StrictModel):
kind: str = Field(min_length=1, max_length=120)
id: str = Field(min_length=1, max_length=255)
class SemanticSubjectReferencePayload(_StrictModel):
module_id: str = Field(min_length=1, max_length=80)
tenant_id: str = Field(min_length=1, max_length=255)
subject_kind: str = Field(min_length=1, max_length=120)
subject_id: str = Field(min_length=1, max_length=255)
anchor: SemanticSubjectAnchorPayload | None = None
observed_revision: str | None = Field(default=None, max_length=255)
observed_fingerprint: str | None = Field(default=None, max_length=80)
class SemanticDocumentationLinkPayload(_StrictModel):
label: str = Field(min_length=1, max_length=300)
href: str = Field(min_length=1, max_length=2000)
@field_validator("href")
@classmethod
def validate_href(cls, value: str) -> str:
if value.startswith("/") and not value.startswith("//"):
return value
if value.startswith("https://"):
return value
raise ValueError("Semantic documentation links must use HTTPS or a local path.")
class SemanticDocumentationContentPayload(_StrictModel):
title: str = Field(min_length=1, max_length=500)
summary: str = Field(default="", max_length=4000)
body: str = Field(default="", max_length=200_000)
meaning: str = Field(default="", max_length=20_000)
intended_use: str = Field(default="", max_length=20_000)
non_intended_use: str = Field(default="", max_length=20_000)
examples: list[str] = Field(default_factory=list, max_length=50)
owner_account_id: str | None = Field(default=None, max_length=255)
steward_account_id: str | None = Field(default=None, max_length=255)
audience: list[str] = Field(default_factory=list, max_length=100)
classification: Literal["internal", "restricted"] = "internal"
links: list[SemanticDocumentationLinkPayload] = Field(
default_factory=list,
max_length=50,
)
@field_validator("examples")
@classmethod
def validate_examples(cls, values: list[str]) -> list[str]:
if any(not value.strip() or len(value) > 4000 for value in values):
raise ValueError("Examples must contain bounded non-empty text.")
return list(dict.fromkeys(value.strip() for value in values))
@field_validator("audience")
@classmethod
def validate_audience(cls, values: list[str]) -> list[str]:
prefixes = ("account:", "group:", "role:", "function:", "scope:")
normalized = list(dict.fromkeys(value.strip() for value in values))
if any(
not value
or len(value) > 500
or (value != "authenticated" and not value.startswith(prefixes))
for value in normalized
):
raise ValueError(
"Audience entries must be authenticated or typed account, group, "
"role, function, or scope selectors."
)
return normalized
@model_validator(mode="after")
def validate_restricted_audience(self):
if self.classification == "restricted" and not self.audience:
raise ValueError("Restricted semantic documentation requires an audience.")
return self
class SemanticDocumentationCreateRequest(_StrictModel):
subject: SemanticSubjectReferencePayload
locale: str = Field(min_length=2, max_length=20)
content: SemanticDocumentationContentPayload
change_reason: str = Field(min_length=1, max_length=1000)
class SemanticDocumentationUpdateRequest(_StrictModel):
expected_revision: int = Field(ge=1)
content: SemanticDocumentationContentPayload
change_reason: str = Field(min_length=1, max_length=1000)
class SemanticDocumentationTransitionRequest(_StrictModel):
expected_revision: int = Field(ge=1)
change_reason: str = Field(min_length=1, max_length=1000)
class SemanticDocumentationSupersedeRequest(SemanticDocumentationTransitionRequest):
replacement_entry_id: str = Field(min_length=1, max_length=36)
class SemanticDocumentationPolicyUpdateRequest(_StrictModel):
mode: Literal["direct", "reviewer_required"]
__all__ = [
"SemanticDocumentationContentPayload",
"SemanticDocumentationCreateRequest",
"SemanticDocumentationLinkPayload",
"SemanticDocumentationPolicyUpdateRequest",
"SemanticDocumentationSupersedeRequest",
"SemanticDocumentationTransitionRequest",
"SemanticDocumentationUpdateRequest",
"SemanticLifecycleState",
"SemanticSubjectAnchorPayload",
"SemanticSubjectAvailability",
"SemanticSubjectReferencePayload",
]
@@ -0,0 +1,980 @@
from __future__ import annotations
import re
from collections.abc import Mapping, Sequence
from datetime import UTC, datetime
from typing import Literal
from sqlalchemy.orm import Session
from govoplan_core.core.semantic_documentation import (
SemanticDocumentationSubjectReference,
SemanticDocumentationSubjectResolution,
resolve_semantic_documentation_subject,
semantic_documentation_fingerprint,
)
from govoplan_core.tenancy.scope import Tenant
from govoplan_docs.backend.db.models import (
SemanticDocumentationEntry,
SemanticDocumentationRevision,
)
SEMANTIC_PUBLICATION_POLICY_KEY = "docs.semantic_publication_policy"
SEMANTIC_PUBLICATION_MODES = frozenset({"direct", "reviewer_required"})
SEMANTIC_LIFECYCLE_STATES = frozenset(
{"draft", "published", "superseded", "retired"}
)
_LOCALE_RE = re.compile(r"^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{2,8})*$")
class SemanticDocumentationError(ValueError):
"""Base semantic-documentation service error."""
class SemanticDocumentationNotFoundError(SemanticDocumentationError):
pass
class SemanticDocumentationConflictError(SemanticDocumentationError):
pass
class SemanticDocumentationAuthorizationError(SemanticDocumentationError):
pass
class SemanticDocumentationSubjectError(SemanticDocumentationError):
pass
def publication_policy(session: Session, tenant_id: str) -> str:
tenant = session.get(Tenant, tenant_id)
settings = tenant.settings if tenant is not None else {}
raw = settings.get(SEMANTIC_PUBLICATION_POLICY_KEY) if settings else None
return str(raw) if raw in SEMANTIC_PUBLICATION_MODES else "reviewer_required"
def set_publication_policy(
session: Session,
*,
tenant_id: str,
mode: str,
) -> str:
if mode not in SEMANTIC_PUBLICATION_MODES:
raise SemanticDocumentationError(
"Semantic publication policy must be direct or reviewer_required."
)
tenant = session.get(Tenant, tenant_id)
if tenant is None:
raise SemanticDocumentationNotFoundError("The active tenant is unavailable.")
settings = dict(tenant.settings or {})
settings[SEMANTIC_PUBLICATION_POLICY_KEY] = mode
tenant.settings = settings
session.flush()
return mode
def create_semantic_entry(
session: Session,
registry: object,
principal: object,
*,
subject: SemanticDocumentationSubjectReference,
locale: str,
content: Mapping[str, object],
change_reason: str,
now: datetime | None = None,
) -> SemanticDocumentationEntry:
tenant_id = _principal_tenant_id(principal)
_require_tenant(subject, tenant_id)
clean_locale = _locale(locale)
current_subject = _resolved_current_subject(
session,
registry,
principal,
subject,
)
stable_key = current_subject.stable_key
existing = (
session.query(SemanticDocumentationEntry)
.filter(
SemanticDocumentationEntry.tenant_id == tenant_id,
SemanticDocumentationEntry.subject_stable_key == stable_key,
SemanticDocumentationEntry.locale == clean_locale,
)
.one_or_none()
)
if existing is not None:
raise SemanticDocumentationConflictError(
"Semantic documentation already exists for this subject and locale."
)
actor_id = _principal_account_id(principal)
anchor = current_subject.anchor
entry = SemanticDocumentationEntry(
tenant_id=tenant_id,
subject_stable_key=stable_key,
subject_module_id=current_subject.module_id,
subject_kind=current_subject.subject_kind,
subject_id=current_subject.subject_id,
anchor_kind=anchor.kind if anchor else None,
anchor_id=anchor.id if anchor else None,
locale=clean_locale,
lifecycle_state="draft",
current_revision=1,
created_by=actor_id,
updated_by=actor_id,
)
session.add(entry)
session.flush()
revision = _new_revision(
entry,
revision=1,
lifecycle_state="draft",
action="create",
content=content,
subject=current_subject,
actor_id=actor_id,
change_reason=change_reason,
now=now,
)
session.add(revision)
session.flush()
entry.current_revision_id = revision.id
session.flush()
return entry
def update_semantic_entry(
session: Session,
registry: object,
principal: object,
*,
entry_id: str,
expected_revision: int,
content: Mapping[str, object],
change_reason: str,
now: datetime | None = None,
) -> SemanticDocumentationEntry:
entry = get_semantic_entry(session, principal, entry_id=entry_id)
_require_editable(entry)
_require_expected_revision(entry, expected_revision)
subject = _resolved_current_subject(
session,
registry,
principal,
entry_subject_reference(entry),
)
actor_id = _principal_account_id(principal)
revision = _new_revision(
entry,
revision=entry.current_revision + 1,
lifecycle_state="draft",
action="edit",
content=content,
subject=subject,
actor_id=actor_id,
change_reason=change_reason,
now=now,
)
session.add(revision)
session.flush()
entry.current_revision += 1
entry.current_revision_id = revision.id
entry.lifecycle_state = "draft"
entry.updated_by = actor_id
session.flush()
return entry
def publish_semantic_entry(
session: Session,
registry: object,
principal: object,
*,
entry_id: str,
expected_revision: int,
change_reason: str,
now: datetime | None = None,
) -> SemanticDocumentationEntry:
entry = get_semantic_entry(session, principal, entry_id=entry_id)
_require_editable(entry)
_require_expected_revision(entry, expected_revision)
current = current_revision(session, entry)
actor_id = _principal_account_id(principal)
mode = publication_policy(session, entry.tenant_id)
if mode == "reviewer_required" and current.authored_by == actor_id:
raise SemanticDocumentationAuthorizationError(
"The configured publication policy requires another reviewer."
)
subject = _resolved_current_subject(
session,
registry,
principal,
entry_subject_reference(entry, revision=current),
)
timestamp = _utc(now)
revision = _new_revision(
entry,
revision=entry.current_revision + 1,
lifecycle_state="published",
action="publish",
content=current.content,
subject=subject,
actor_id=current.authored_by,
reviewer_id=actor_id,
change_reason=change_reason,
now=timestamp,
)
revision.published_at = timestamp
revision.provenance = {
**revision.provenance,
"publication_policy": mode,
"published_by": actor_id,
}
session.add(revision)
session.flush()
entry.current_revision += 1
entry.current_revision_id = revision.id
entry.published_revision_id = revision.id
entry.lifecycle_state = "published"
entry.updated_by = actor_id
entry.published_by = actor_id
entry.published_at = timestamp
session.flush()
return entry
def supersede_semantic_entry(
session: Session,
principal: object,
*,
entry_id: str,
replacement_entry_id: str,
expected_revision: int,
change_reason: str,
now: datetime | None = None,
) -> SemanticDocumentationEntry:
entry = get_semantic_entry(session, principal, entry_id=entry_id)
_require_editable(entry)
_require_expected_revision(entry, expected_revision)
if entry.id == replacement_entry_id:
raise SemanticDocumentationError("An entry cannot supersede itself.")
replacement = get_semantic_entry(
session,
principal,
entry_id=replacement_entry_id,
)
if replacement.published_revision_id is None:
raise SemanticDocumentationError(
"The replacement semantic documentation must be published."
)
current = current_revision(session, entry)
actor_id = _principal_account_id(principal)
revision = _lifecycle_revision(
entry,
current=current,
lifecycle_state="superseded",
action="supersede",
actor_id=actor_id,
change_reason=change_reason,
now=now,
provenance={"replacement_entry_id": replacement.id},
)
session.add(revision)
session.flush()
entry.current_revision += 1
entry.current_revision_id = revision.id
entry.lifecycle_state = "superseded"
entry.superseded_by_entry_id = replacement.id
entry.updated_by = actor_id
session.flush()
return entry
def retire_semantic_entry(
session: Session,
principal: object,
*,
entry_id: str,
expected_revision: int,
change_reason: str,
now: datetime | None = None,
) -> SemanticDocumentationEntry:
entry = get_semantic_entry(session, principal, entry_id=entry_id)
_require_editable(entry)
_require_expected_revision(entry, expected_revision)
current = current_revision(session, entry)
actor_id = _principal_account_id(principal)
timestamp = _utc(now)
revision = _lifecycle_revision(
entry,
current=current,
lifecycle_state="retired",
action="retire",
actor_id=actor_id,
change_reason=change_reason,
now=timestamp,
)
session.add(revision)
session.flush()
entry.current_revision += 1
entry.current_revision_id = revision.id
entry.lifecycle_state = "retired"
entry.retired_by = actor_id
entry.retired_at = timestamp
entry.updated_by = actor_id
session.flush()
return entry
def get_semantic_entry(
session: Session,
principal: object,
*,
entry_id: str,
) -> SemanticDocumentationEntry:
tenant_id = _principal_tenant_id(principal)
entry = (
session.query(SemanticDocumentationEntry)
.filter(
SemanticDocumentationEntry.id == entry_id,
SemanticDocumentationEntry.tenant_id == tenant_id,
)
.one_or_none()
)
if entry is None:
raise SemanticDocumentationNotFoundError(
"Semantic documentation entry not found."
)
return entry
def list_semantic_entries(
session: Session,
principal: object,
*,
module_id: str | None = None,
subject_kind: str | None = None,
) -> tuple[SemanticDocumentationEntry, ...]:
query = session.query(SemanticDocumentationEntry).filter(
SemanticDocumentationEntry.tenant_id == _principal_tenant_id(principal)
)
if module_id:
query = query.filter(SemanticDocumentationEntry.subject_module_id == module_id)
if subject_kind:
query = query.filter(SemanticDocumentationEntry.subject_kind == subject_kind)
return tuple(
query.order_by(
SemanticDocumentationEntry.subject_module_id.asc(),
SemanticDocumentationEntry.subject_kind.asc(),
SemanticDocumentationEntry.subject_id.asc(),
SemanticDocumentationEntry.locale.asc(),
).all()
)
def semantic_entry_history(
session: Session,
principal: object,
*,
entry_id: str,
) -> tuple[SemanticDocumentationRevision, ...]:
entry = get_semantic_entry(session, principal, entry_id=entry_id)
return tuple(
session.query(SemanticDocumentationRevision)
.filter(
SemanticDocumentationRevision.entry_id == entry.id,
SemanticDocumentationRevision.tenant_id == entry.tenant_id,
)
.order_by(SemanticDocumentationRevision.revision.desc())
.all()
)
def current_revision(
session: Session,
entry: SemanticDocumentationEntry,
) -> SemanticDocumentationRevision:
revision = (
session.get(SemanticDocumentationRevision, entry.current_revision_id)
if entry.current_revision_id
else None
)
if revision is None or revision.entry_id != entry.id:
raise SemanticDocumentationError(
"Semantic documentation current revision is unavailable."
)
return revision
def published_revision(
session: Session,
entry: SemanticDocumentationEntry,
) -> SemanticDocumentationRevision | None:
if not entry.published_revision_id:
return None
revision = session.get(
SemanticDocumentationRevision,
entry.published_revision_id,
)
return revision if revision is not None and revision.entry_id == entry.id else None
def entry_subject_reference(
entry: SemanticDocumentationEntry,
*,
revision: SemanticDocumentationRevision | None = None,
) -> SemanticDocumentationSubjectReference:
from govoplan_core.core.semantic_documentation import (
SemanticDocumentationSubjectAnchor,
)
return SemanticDocumentationSubjectReference(
module_id=entry.subject_module_id,
tenant_id=entry.tenant_id,
subject_kind=entry.subject_kind,
subject_id=entry.subject_id,
anchor=(
SemanticDocumentationSubjectAnchor(
kind=entry.anchor_kind,
id=entry.anchor_id,
)
if entry.anchor_kind and entry.anchor_id
else None
),
observed_revision=revision.subject_revision if revision else None,
observed_fingerprint=revision.subject_fingerprint if revision else None,
)
def resolve_entry_subject(
session: Session,
registry: object,
principal: object,
*,
entry: SemanticDocumentationEntry,
revision: SemanticDocumentationRevision,
) -> SemanticDocumentationSubjectResolution | None:
return resolve_semantic_documentation_subject(
registry,
session,
principal,
reference=entry_subject_reference(entry, revision=revision),
)
def content_visible_to_principal(
content: Mapping[str, object],
principal: object,
) -> bool:
audience = tuple(str(item) for item in content.get("audience", ()) or ())
classification = str(content.get("classification") or "internal")
if classification == "restricted" and not audience:
return False
if not audience or "authenticated" in audience:
return True
tokens = _principal_audience_tokens(principal)
return any(selector in tokens for selector in audience)
def semantic_entry_payload(
session: Session,
registry: object,
principal: object,
*,
entry: SemanticDocumentationEntry,
editor: bool,
requested_locale: str | None = None,
) -> dict[str, object] | None:
current = current_revision(session, entry)
published = published_revision(session, entry)
selected = current if editor else published
if selected is None:
return None
resolution = resolve_entry_subject(
session,
registry,
principal,
entry=entry,
revision=selected,
)
if resolution is None:
return None
unavailable = resolution.availability == "temporarily_unavailable"
if unavailable and not editor:
return None
if not content_visible_to_principal(selected.content, principal):
return None
subject = resolution.subject
required_scopes = subject.required_scopes if subject is not None else ()
if any(not _principal_has(principal, scope) for scope in required_scopes):
return None
return {
"id": entry.id,
"tenant_id": entry.tenant_id,
"subject": entry_subject_reference(entry, revision=selected).to_dict(),
"subject_stable_key": entry.subject_stable_key,
"subject_resolution": resolution.to_dict(),
"locale": entry.locale,
"requested_locale": requested_locale or entry.locale,
"locale_fallback": bool(requested_locale and requested_locale != entry.locale),
"lifecycle_state": entry.lifecycle_state,
"effective_state": selected.lifecycle_state,
"pending_draft": bool(
published is not None and current.id != published.id
),
"current_revision": entry.current_revision,
"selected_revision": selected.revision,
"published_revision": published.revision if published else None,
"content": {} if unavailable else dict(selected.content),
"content_redacted": unavailable,
"content_hash": selected.content_hash,
"subject_revision": selected.subject_revision,
"subject_fingerprint": selected.subject_fingerprint,
"authorship": {
"created_by": entry.created_by,
"updated_by": entry.updated_by,
"authored_by": selected.authored_by,
"reviewed_by": selected.reviewed_by,
"published_by": entry.published_by,
},
"published_at": entry.published_at.isoformat() if entry.published_at else None,
"retired_at": entry.retired_at.isoformat() if entry.retired_at else None,
"superseded_by_entry_id": entry.superseded_by_entry_id,
"created_at": entry.created_at.isoformat(),
"updated_at": entry.updated_at.isoformat(),
}
def select_locale_entries(
entries: Sequence[SemanticDocumentationEntry],
*,
locale: str,
) -> tuple[SemanticDocumentationEntry, ...]:
requested = _locale(locale)
language = requested.split("-", 1)[0]
grouped: dict[str, list[SemanticDocumentationEntry]] = {}
for entry in entries:
grouped.setdefault(entry.subject_stable_key, []).append(entry)
selected: list[SemanticDocumentationEntry] = []
for candidates in grouped.values():
order = (
requested,
language,
"de",
"en",
)
candidate = next(
(
item
for target in order
for item in candidates
if item.locale.casefold() == target.casefold()
),
sorted(candidates, key=lambda item: item.locale)[0],
)
selected.append(candidate)
return tuple(
sorted(
selected,
key=lambda item: (
item.subject_module_id,
item.subject_kind,
item.subject_id,
item.anchor_kind or "",
item.anchor_id or "",
),
)
)
def revision_payload(revision: SemanticDocumentationRevision) -> dict[str, object]:
return {
"id": revision.id,
"entry_id": revision.entry_id,
"revision": revision.revision,
"lifecycle_state": revision.lifecycle_state,
"action": revision.action,
"change_reason": revision.change_reason,
"content": dict(revision.content),
"content_hash": revision.content_hash,
"subject_revision": revision.subject_revision,
"subject_fingerprint": revision.subject_fingerprint,
"authored_by": revision.authored_by,
"reviewed_by": revision.reviewed_by,
"published_at": (
revision.published_at.isoformat() if revision.published_at else None
),
"provenance": dict(revision.provenance or {}),
"recoverable": revision.recoverable,
"created_at": revision.created_at.isoformat(),
}
def _new_revision(
entry: SemanticDocumentationEntry,
*,
revision: int,
lifecycle_state: str,
action: str,
content: Mapping[str, object],
subject: SemanticDocumentationSubjectReference,
actor_id: str,
change_reason: str,
now: datetime | None,
reviewer_id: str | None = None,
) -> SemanticDocumentationRevision:
normalized = _normalize_content(content)
timestamp = _utc(now)
return SemanticDocumentationRevision(
tenant_id=entry.tenant_id,
entry_id=entry.id,
revision=revision,
lifecycle_state=lifecycle_state,
action=action,
change_reason=_bounded_required(change_reason, "Change reason", 1000),
content=normalized,
content_hash=semantic_documentation_fingerprint(normalized),
subject_revision=subject.observed_revision,
subject_fingerprint=subject.observed_fingerprint,
authored_by=actor_id,
reviewed_by=reviewer_id,
provenance={
"subject_stable_key": subject.stable_key,
"recorded_at": timestamp.isoformat(),
"action_by": reviewer_id or actor_id,
},
recoverable=True,
search_text=_search_text(normalized),
)
def _lifecycle_revision(
entry: SemanticDocumentationEntry,
*,
current: SemanticDocumentationRevision,
lifecycle_state: Literal["superseded", "retired"],
action: str,
actor_id: str,
change_reason: str,
now: datetime | None,
provenance: Mapping[str, object] | None = None,
) -> SemanticDocumentationRevision:
timestamp = _utc(now)
return SemanticDocumentationRevision(
tenant_id=entry.tenant_id,
entry_id=entry.id,
revision=entry.current_revision + 1,
lifecycle_state=lifecycle_state,
action=action,
change_reason=_bounded_required(change_reason, "Change reason", 1000),
content=dict(current.content),
content_hash=current.content_hash,
subject_revision=current.subject_revision,
subject_fingerprint=current.subject_fingerprint,
authored_by=current.authored_by,
reviewed_by=actor_id,
provenance={
**dict(current.provenance or {}),
**dict(provenance or {}),
"recorded_at": timestamp.isoformat(),
"action_by": actor_id,
},
recoverable=True,
search_text=current.search_text,
)
def _resolved_current_subject(
session: Session,
registry: object,
principal: object,
reference: SemanticDocumentationSubjectReference,
) -> SemanticDocumentationSubjectReference:
resolution = resolve_semantic_documentation_subject(
registry,
session,
principal,
reference=reference,
)
if resolution is None:
raise SemanticDocumentationNotFoundError(
"Semantic documentation subject not found."
)
if resolution.availability not in {"available", "changed"} or resolution.subject is None:
raise SemanticDocumentationSubjectError(
"Semantic documentation subject is not currently available."
)
return resolution.subject.reference
def _normalize_content(content: Mapping[str, object]) -> dict[str, object]:
allowed = {
"title",
"summary",
"body",
"meaning",
"intended_use",
"non_intended_use",
"examples",
"owner_account_id",
"steward_account_id",
"audience",
"classification",
"links",
}
unknown = sorted(str(key) for key in content if key not in allowed)
if unknown:
raise SemanticDocumentationError(
"Unsupported semantic content fields: " + ", ".join(unknown)
)
normalized = {
"title": _plain_text(content.get("title"), "Title", 500, required=True),
"summary": _plain_text(content.get("summary"), "Summary", 4000),
"body": _plain_text(content.get("body"), "Body", 200_000),
"meaning": _plain_text(content.get("meaning"), "Meaning", 20_000),
"intended_use": _plain_text(
content.get("intended_use"), "Intended use", 20_000
),
"non_intended_use": _plain_text(
content.get("non_intended_use"), "Non-intended use", 20_000
),
"examples": _string_list(content.get("examples"), "Examples", 50, 4000),
"owner_account_id": _optional_id(content.get("owner_account_id")),
"steward_account_id": _optional_id(content.get("steward_account_id")),
"audience": _audience(content.get("audience")),
"classification": str(content.get("classification") or "internal"),
"links": _links(content.get("links")),
}
if normalized["classification"] not in {"internal", "restricted"}:
raise SemanticDocumentationError(
"Semantic content classification must be internal or restricted."
)
if normalized["classification"] == "restricted" and not normalized["audience"]:
raise SemanticDocumentationError(
"Restricted semantic documentation requires an audience."
)
return normalized
def _plain_text(
value: object,
label: str,
maximum: int,
*,
required: bool = False,
) -> str:
text = str(value or "").strip()
if required and not text:
raise SemanticDocumentationError(f"{label} is required.")
if len(text) > maximum:
raise SemanticDocumentationError(
f"{label} is limited to {maximum} characters."
)
if any(ord(character) < 32 and character not in "\n\t" for character in text):
raise SemanticDocumentationError(f"{label} contains control characters.")
if "<script" in text.casefold() or "javascript:" in text.casefold():
raise SemanticDocumentationError(f"{label} contains unsafe active content.")
return text
def _string_list(
value: object,
label: str,
maximum_items: int,
maximum_length: int,
) -> list[str]:
if value is None:
return []
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
raise SemanticDocumentationError(f"{label} must be a list.")
if len(value) > maximum_items:
raise SemanticDocumentationError(
f"{label} are limited to {maximum_items} items."
)
return list(
dict.fromkeys(
_plain_text(item, label, maximum_length, required=True) for item in value
)
)
def _audience(value: object) -> list[str]:
selectors = _string_list(value, "Audience", 100, 500)
prefixes = ("account:", "group:", "role:", "function:", "scope:")
if any(
selector != "authenticated" and not selector.startswith(prefixes)
for selector in selectors
):
raise SemanticDocumentationError(
"Semantic audience selectors must be typed."
)
return selectors
def _links(value: object) -> list[dict[str, str]]:
if value is None:
return []
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
raise SemanticDocumentationError("Links must be a list.")
if len(value) > 50:
raise SemanticDocumentationError("Links are limited to 50 items.")
result: list[dict[str, str]] = []
for item in value:
if not isinstance(item, Mapping) or set(item) != {"label", "href"}:
raise SemanticDocumentationError(
"Semantic links require only label and href."
)
label = _plain_text(item.get("label"), "Link label", 300, required=True)
href = _plain_text(item.get("href"), "Link href", 2000, required=True)
if not (
(href.startswith("/") and not href.startswith("//"))
or href.startswith("https://")
):
raise SemanticDocumentationError(
"Semantic links must use HTTPS or a local path."
)
if href.startswith("https://") and "@" in href.split("/", 3)[2]:
raise SemanticDocumentationError("Semantic links cannot contain credentials.")
result.append({"label": label, "href": href})
return result
def _search_text(content: Mapping[str, object]) -> str:
values = [
content.get("title"),
content.get("summary"),
content.get("body"),
content.get("meaning"),
content.get("intended_use"),
content.get("non_intended_use"),
*(content.get("examples") or ()),
]
return "\n".join(str(value) for value in values if value).strip()
def _principal_audience_tokens(principal: object) -> frozenset[str]:
return frozenset(
{
"authenticated",
f"account:{_principal_account_id(principal)}",
*(f"group:{value}" for value in getattr(principal, "group_ids", ())),
*(f"role:{value}" for value in getattr(principal, "role_ids", ())),
*(
f"function:{value}"
for value in getattr(principal, "function_assignment_ids", ())
),
*(f"scope:{value}" for value in getattr(principal, "scopes", ())),
}
)
def _principal_has(principal: object, scope: str) -> bool:
checker = getattr(principal, "has", None)
return bool(checker(scope)) if callable(checker) else scope in getattr(
principal, "scopes", ()
)
def _principal_tenant_id(principal: object) -> str:
tenant_id = str(getattr(principal, "tenant_id", "") or "").strip()
if not tenant_id:
raise SemanticDocumentationAuthorizationError(
"Semantic documentation requires an active tenant."
)
return tenant_id
def _principal_account_id(principal: object) -> str:
account_id = str(getattr(principal, "account_id", "") or "").strip()
if not account_id:
raise SemanticDocumentationAuthorizationError(
"Semantic documentation requires an authenticated account."
)
return account_id
def _require_tenant(
reference: SemanticDocumentationSubjectReference,
tenant_id: str,
) -> None:
if reference.tenant_id != tenant_id:
raise SemanticDocumentationNotFoundError(
"Semantic documentation subject not found."
)
def _require_expected_revision(
entry: SemanticDocumentationEntry,
expected_revision: int,
) -> None:
if entry.current_revision != expected_revision:
raise SemanticDocumentationConflictError(
"Semantic documentation changed; reload before saving."
)
def _require_editable(entry: SemanticDocumentationEntry) -> None:
if entry.lifecycle_state in {"superseded", "retired"}:
raise SemanticDocumentationConflictError(
f"{entry.lifecycle_state.title()} semantic documentation cannot be edited."
)
def _locale(value: str) -> str:
clean = str(value or "").strip()
if not _LOCALE_RE.fullmatch(clean):
raise SemanticDocumentationError("Semantic documentation locale is invalid.")
return clean
def _optional_id(value: object) -> str | None:
clean = str(value or "").strip()
if not clean:
return None
if len(clean) > 255 or any(ord(character) < 32 for character in clean):
raise SemanticDocumentationError("Account references must be bounded text.")
return clean
def _bounded_required(value: object, label: str, maximum: int) -> str:
return _plain_text(value, label, maximum, required=True)
def _utc(value: datetime | None) -> datetime:
timestamp = value or datetime.now(UTC)
if timestamp.tzinfo is None:
return timestamp.replace(tzinfo=UTC)
return timestamp.astimezone(UTC)
__all__ = [
"SEMANTIC_LIFECYCLE_STATES",
"SEMANTIC_PUBLICATION_MODES",
"SEMANTIC_PUBLICATION_POLICY_KEY",
"SemanticDocumentationAuthorizationError",
"SemanticDocumentationConflictError",
"SemanticDocumentationError",
"SemanticDocumentationNotFoundError",
"SemanticDocumentationSubjectError",
"content_visible_to_principal",
"create_semantic_entry",
"current_revision",
"entry_subject_reference",
"get_semantic_entry",
"list_semantic_entries",
"publication_policy",
"publish_semantic_entry",
"published_revision",
"resolve_entry_subject",
"retire_semantic_entry",
"revision_payload",
"select_locale_entries",
"semantic_entry_history",
"semantic_entry_payload",
"set_publication_policy",
"supersede_semantic_entry",
"update_semantic_entry",
]
+373
View File
@@ -0,0 +1,373 @@
from __future__ import annotations
from types import SimpleNamespace
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.search import (
SearchAuthorizationRequest,
SearchBackfillRequest,
)
from govoplan_core.core.semantic_documentation import (
SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION,
SemanticDocumentationSubjectDescriptor,
SemanticDocumentationSubjectPage,
SemanticDocumentationSubjectReference,
SemanticDocumentationSubjectResolution,
semantic_documentation_fingerprint,
semantic_documentation_subject_capability,
)
from govoplan_core.db.base import Base
from govoplan_core.tenancy.scope import Tenant, create_scope_tables
from govoplan_docs.backend.dsar_provider import DocsDsarProvider
from govoplan_docs.backend.search_source import SemanticDocumentationSearchSource
from govoplan_docs.backend.semantic_service import (
SemanticDocumentationAuthorizationError,
SemanticDocumentationConflictError,
create_semantic_entry,
publication_policy,
publish_semantic_entry,
select_locale_entries,
semantic_entry_payload,
set_publication_policy,
update_semantic_entry,
)
from govoplan_core.core.dsar import DsarSubjectRef
class _SubjectProvider:
provider_id = "forms.semantic_subjects"
module_id = "forms"
contract_version = SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION
def __init__(self) -> None:
self.revision = "7"
self.denied_accounts: set[str] = set()
def reference(self, tenant_id: str) -> SemanticDocumentationSubjectReference:
fingerprint = semantic_documentation_fingerprint(
{"id": "form-1", "revision": self.revision}
)
return SemanticDocumentationSubjectReference(
module_id="forms",
tenant_id=tenant_id,
subject_kind="form",
subject_id="form-1",
observed_revision=self.revision,
observed_fingerprint=fingerprint,
)
def descriptor(self, tenant_id: str) -> SemanticDocumentationSubjectDescriptor:
return SemanticDocumentationSubjectDescriptor(
reference=self.reference(tenant_id),
labels={"de": "Anwohnerparkausweis", "en": "Resident permit"},
descriptions={"de": "Konfiguriertes Formular"},
route="/forms/form-1",
required_scopes=("forms:form:read",),
)
def list_subjects(self, session, principal, *, request):
del session
if principal.account_id in self.denied_accounts:
return SemanticDocumentationSubjectPage()
return SemanticDocumentationSubjectPage(
subjects=(self.descriptor(request.tenant_id),)
)
def resolve_subject(self, session, principal, *, reference):
del session
if principal.account_id in self.denied_accounts:
return None
descriptor = self.descriptor(reference.tenant_id)
availability = (
"changed"
if reference.observed_revision
and reference.observed_revision != descriptor.reference.observed_revision
else "available"
)
return SemanticDocumentationSubjectResolution(
requested_reference=reference,
availability=availability,
subject=descriptor,
)
class _Registry:
def __init__(self, provider: _SubjectProvider) -> None:
self.provider = provider
self.name = semantic_documentation_subject_capability("forms")
def capability_names(self):
return (self.name,)
def has_capability(self, name):
return name == self.name
def capability(self, name):
return self.provider if name == self.name else None
def _principal(
account_id: str,
*,
tenant_id: str = "tenant-1",
scopes: frozenset[str] | None = None,
) -> ApiPrincipal:
principal = PrincipalRef(
account_id=account_id,
membership_id=f"membership-{account_id}",
tenant_id=tenant_id,
scopes=scopes or frozenset({"docs:documentation:read", "forms:form:read"}),
)
return ApiPrincipal(
principal=principal,
account=SimpleNamespace(id=account_id),
user=SimpleNamespace(id=account_id),
)
def _content(title: str = "Permit form") -> dict[str, object]:
return {
"title": title,
"summary": "Tenant meaning",
"body": "Use this form for residents.",
"meaning": "Configured resident permit request.",
"intended_use": "Resident permits",
"non_intended_use": "Visitor permits",
"examples": [],
"owner_account_id": "author",
"steward_account_id": "reviewer",
"audience": [],
"classification": "internal",
"links": [],
}
class SemanticDocumentationTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine("sqlite+pysqlite:///:memory:")
create_scope_tables(self.engine)
Base.metadata.create_all(self.engine)
self.session = Session(self.engine)
self.session.add(
Tenant(id="tenant-1", slug="tenant-1", name="Tenant One", settings={})
)
self.session.commit()
self.provider = _SubjectProvider()
self.registry = _Registry(self.provider)
self.author = _principal("author")
self.reviewer = _principal("reviewer")
def tearDown(self) -> None:
self.session.close()
self.engine.dispose()
def test_revision_review_policy_visibility_and_concurrency(self) -> None:
entry = create_semantic_entry(
self.session,
self.registry,
self.author,
subject=self.provider.reference("tenant-1"),
locale="de",
content=_content(),
change_reason="Initial tenant semantics",
)
self.assertEqual("reviewer_required", publication_policy(self.session, "tenant-1"))
with self.assertRaises(SemanticDocumentationAuthorizationError):
publish_semantic_entry(
self.session,
self.registry,
self.author,
entry_id=entry.id,
expected_revision=1,
change_reason="Self publish",
)
entry = publish_semantic_entry(
self.session,
self.registry,
self.reviewer,
entry_id=entry.id,
expected_revision=1,
change_reason="Independent review",
)
self.assertEqual(2, entry.current_revision)
reader_payload = semantic_entry_payload(
self.session,
self.registry,
self.author,
entry=entry,
editor=False,
)
self.assertEqual("Permit form", reader_payload["content"]["title"])
update_semantic_entry(
self.session,
self.registry,
self.author,
entry_id=entry.id,
expected_revision=2,
content=_content("Updated permit form"),
change_reason="Clarify meaning",
)
with self.assertRaises(SemanticDocumentationConflictError):
update_semantic_entry(
self.session,
self.registry,
self.author,
entry_id=entry.id,
expected_revision=2,
content=_content("Stale edit"),
change_reason="Stale",
)
reader_payload = semantic_entry_payload(
self.session,
self.registry,
self.author,
entry=entry,
editor=False,
)
self.assertEqual("Permit form", reader_payload["content"]["title"])
self.assertTrue(reader_payload["pending_draft"])
def test_subject_reauthorization_tenant_and_changed_state(self) -> None:
entry = create_semantic_entry(
self.session,
self.registry,
self.author,
subject=self.provider.reference("tenant-1"),
locale="de",
content=_content(),
change_reason="Initial",
)
publish_semantic_entry(
self.session,
self.registry,
self.reviewer,
entry_id=entry.id,
expected_revision=1,
change_reason="Review",
)
self.provider.revision = "8"
payload = semantic_entry_payload(
self.session,
self.registry,
self.author,
entry=entry,
editor=False,
)
self.assertEqual("changed", payload["subject_resolution"]["availability"])
self.provider.denied_accounts.add("author")
self.assertIsNone(
semantic_entry_payload(
self.session,
self.registry,
self.author,
entry=entry,
editor=False,
)
)
with self.assertRaises(Exception):
create_semantic_entry(
self.session,
self.registry,
_principal("other", tenant_id="tenant-2"),
subject=self.provider.reference("tenant-1"),
locale="de",
content=_content(),
change_reason="Foreign",
)
def test_locale_search_and_dsar_projection(self) -> None:
entries = []
for locale, title in (("de", "Deutsch"), ("en", "English")):
entry = create_semantic_entry(
self.session,
self.registry,
self.author,
subject=self.provider.reference("tenant-1"),
locale=locale,
content=_content(title),
change_reason="Initial",
)
publish_semantic_entry(
self.session,
self.registry,
self.reviewer,
entry_id=entry.id,
expected_revision=1,
change_reason="Review",
)
entries.append(entry)
self.assertEqual("de", select_locale_entries(entries, locale="de-AT")[0].locale)
source = SemanticDocumentationSearchSource(self.registry)
page = source.backfill(
self.session,
request=SearchBackfillRequest(
tenant_id="tenant-1",
provider_id="docs.semantic_documentation",
resource_type="semantic_documentation",
rebuild_id="rebuild-1",
),
)
self.assertEqual(2, len(page.documents))
request = SearchAuthorizationRequest(
reference=page.documents[0].reference,
source_revision=page.documents[0].source_revision,
)
self.assertTrue(source.authorize(self.session, self.author, requests=(request,))[request.reference.key])
self.provider.denied_accounts.add("author")
self.assertFalse(source.authorize(self.session, self.author, requests=(request,))[request.reference.key])
records = DocsDsarProvider().search_subject(
self.session,
tenant_id="tenant-1",
subject=DsarSubjectRef(account_id="author"),
)
self.assertGreaterEqual(len(records), 2)
self.assertNotIn("body", records[0].data)
actions = DocsDsarProvider().plan_erasure(
self.session,
tenant_id="tenant-1",
subject=DsarSubjectRef(account_id="author"),
records=records,
)
self.assertTrue(all(not action.executable for action in actions))
def test_direct_publication_policy_is_explicit(self) -> None:
self.assertEqual(
"direct",
set_publication_policy(
self.session,
tenant_id="tenant-1",
mode="direct",
),
)
entry = create_semantic_entry(
self.session,
self.registry,
self.author,
subject=self.provider.reference("tenant-1"),
locale="de",
content=_content(),
change_reason="Initial",
)
published = publish_semantic_entry(
self.session,
self.registry,
self.author,
entry_id=entry.id,
expected_revision=1,
change_reason="Direct publication",
)
self.assertEqual("published", published.lifecycle_state)
if __name__ == "__main__":
unittest.main()
+4 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@govoplan/docs-webui", "name": "@govoplan/docs-webui",
"version": "0.1.18", "version": "0.1.19",
"private": true, "private": true,
"type": "module", "type": "module",
"main": "src/index.ts", "main": "src/index.ts",
@@ -12,6 +12,9 @@
"import": "./src/index.ts" "import": "./src/index.ts"
} }
}, },
"scripts": {
"typecheck": "tsc --noEmit"
},
"peerDependencies": { "peerDependencies": {
"@govoplan/core-webui": "^0.1.18", "@govoplan/core-webui": "^0.1.18",
"lucide-react": "^1.23.0", "lucide-react": "^1.23.0",
+115
View File
@@ -275,3 +275,118 @@ export function fetchDocsSource(
`/api/v1/docs/sources/${encodeURIComponent(sourceId)}${query ? `?${query}` : ""}` `/api/v1/docs/sources/${encodeURIComponent(sourceId)}${query ? `?${query}` : ""}`
); );
} }
export type SemanticSubjectReference = {
module_id: string;
tenant_id: string;
subject_kind: string;
subject_id: string;
anchor?: { kind: string; id: string } | null;
observed_revision?: string | null;
observed_fingerprint?: string | null;
};
export type SemanticSubjectDescriptor = {
reference: SemanticSubjectReference;
labels: Record<string, string>;
descriptions: Record<string, string>;
route?: string | null;
route_anchor?: string | null;
};
export type SemanticContent = {
title: string;
summary: string;
body: string;
meaning: string;
intended_use: string;
non_intended_use: string;
examples: string[];
owner_account_id: string | null;
steward_account_id: string | null;
audience: string[];
classification: "internal" | "restricted";
links: Array<{ label: string; href: string }>;
};
export type SemanticEntry = {
id: string;
subject: SemanticSubjectReference;
subject_resolution: {
availability: "available" | "changed" | "superseded" | "missing" | "temporarily_unavailable";
reason_code?: string | null;
};
locale: string;
requested_locale: string;
locale_fallback: boolean;
lifecycle_state: "draft" | "published" | "superseded" | "retired";
pending_draft: boolean;
current_revision: number;
published_revision?: number | null;
content: SemanticContent;
content_redacted: boolean;
updated_at: string;
};
export async function fetchSemanticSubjects(
settings: ApiSettings,
query = ""
): Promise<SemanticSubjectDescriptor[]> {
const params = new URLSearchParams({ query });
const response = await apiFetch<{ providers: Array<{ subjects: SemanticSubjectDescriptor[] }> }>(
settings,
`/api/v1/docs/semantic/subjects?${params}`
);
return response.providers.flatMap((provider) => provider.subjects);
}
export async function fetchSemanticEntries(
settings: ApiSettings,
locale: string,
includeDrafts = true
): Promise<SemanticEntry[]> {
const params = new URLSearchParams({ locale, include_drafts: String(includeDrafts) });
const response = await apiFetch<{ items: SemanticEntry[] }>(
settings,
`/api/v1/docs/semantic/entries?${params}`
);
return response.items;
}
export function createSemanticEntry(
settings: ApiSettings,
payload: { subject: SemanticSubjectReference; locale: string; content: SemanticContent; change_reason: string }
): Promise<SemanticEntry> {
return apiFetch(settings, "/api/v1/docs/semantic/entries", {
method: "POST",
body: JSON.stringify(payload)
});
}
export function updateSemanticEntry(
settings: ApiSettings,
entryId: string,
payload: { expected_revision: number; content: SemanticContent; change_reason: string }
): Promise<SemanticEntry> {
return apiFetch(settings, `/api/v1/docs/semantic/entries/${encodeURIComponent(entryId)}`, {
method: "PUT",
body: JSON.stringify(payload)
});
}
export function transitionSemanticEntry(
settings: ApiSettings,
entryId: string,
transition: "publish" | "retire",
expectedRevision: number,
changeReason: string
): Promise<SemanticEntry> {
return apiFetch(
settings,
`/api/v1/docs/semantic/entries/${encodeURIComponent(entryId)}/${transition}`,
{
method: "POST",
body: JSON.stringify({ expected_revision: expectedRevision, change_reason: changeReason })
}
);
}
+1 -1
View File
@@ -1,7 +1,7 @@
import { DescriptionList } from "@govoplan/core-webui"; import { DescriptionList } from "@govoplan/core-webui";
import { useEffect, useMemo, useRef, useState } from "react"; import { useEffect, useMemo, useRef, useState } from "react";
import { Link, useLocation } from "react-router"; import { Link, useLocation } from "react-router";
import { ChevronDown, ChevronRight, Eye, RefreshCw } from "lucide-react"; import { ChevronDown, ChevronRight, Eye } from "lucide-react";
import { import {
Button, Button,
DataGrid, DataGrid,
@@ -0,0 +1,307 @@
import { useEffect, useMemo, useState } from "react";
import { useSearchParams } from "react-router";
import { Archive, Check, Plus } from "lucide-react";
import {
Button,
Card,
ContentGrid,
DismissibleAlert,
FormField,
FormGrid,
PageActionBar,
PageLayout,
StatePanel,
StatusBadge,
WorkspaceLayout,
adminErrorMessage,
usePlatformLanguage,
useUnsavedDraftGuard,
type ApiSettings
} from "@govoplan/core-webui";
import {
createSemanticEntry,
fetchSemanticEntries,
fetchSemanticSubjects,
transitionSemanticEntry,
updateSemanticEntry,
type SemanticContent,
type SemanticEntry,
type SemanticSubjectDescriptor
} from "../../api/docs";
const emptyContent = (): SemanticContent => ({
title: "",
summary: "",
body: "",
meaning: "",
intended_use: "",
non_intended_use: "",
examples: [],
owner_account_id: null,
steward_account_id: null,
audience: [],
classification: "internal",
links: []
});
export default function SemanticDocumentationPage({ settings }: { settings: ApiSettings }) {
const { language } = usePlatformLanguage();
const [searchParams, setSearchParams] = useSearchParams();
const [subjects, setSubjects] = useState<SemanticSubjectDescriptor[]>([]);
const [entries, setEntries] = useState<SemanticEntry[]>([]);
const [selectedSubject, setSelectedSubject] = useState<SemanticSubjectDescriptor | null>(null);
const [selectedEntry, setSelectedEntry] = useState<SemanticEntry | null>(null);
const [content, setContent] = useState<SemanticContent>(emptyContent);
const [locale, setLocale] = useState(searchParams.get("locale") || language || "de");
const [reason, setReason] = useState("Document configured meaning");
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [error, setError] = useState("");
const [success, setSuccess] = useState("");
const baseline = selectedEntry?.content ?? emptyContent();
const dirty = Boolean(selectedSubject) && (
JSON.stringify(content) !== JSON.stringify(baseline)
|| (!selectedEntry && Boolean(content.title || content.body || content.meaning))
);
const valid = Boolean(selectedSubject && content.title.trim() && reason.trim());
useUnsavedDraftGuard({
dirty,
onSave: save,
onDiscard: resetDraft
});
async function load() {
setLoading(true);
setError("");
try {
const [nextSubjects, nextEntries] = await Promise.all([
fetchSemanticSubjects(settings),
fetchSemanticEntries(settings, locale, true)
]);
setSubjects(nextSubjects);
setEntries(nextEntries);
const requestedEntry = searchParams.get("entryId");
const entry = nextEntries.find((item) => item.id === requestedEntry) ?? selectedEntry;
if (entry) selectExisting(entry, nextSubjects);
} catch (reason) {
setError(adminErrorMessage(reason));
} finally {
setLoading(false);
}
}
useEffect(() => { void load(); }, [settings.apiBaseUrl, settings.apiKey, settings.accessToken]);
async function save(): Promise<boolean> {
if (!selectedSubject || !valid) return false;
setSaving(true);
setError("");
try {
const next = selectedEntry
? await updateSemanticEntry(settings, selectedEntry.id, {
expected_revision: selectedEntry.current_revision,
content,
change_reason: reason
})
: await createSemanticEntry(settings, {
subject: selectedSubject.reference,
locale,
content,
change_reason: reason
});
setSelectedEntry(next);
setContent(next.content);
setEntries((current) => [next, ...current.filter((item) => item.id !== next.id)]);
setSearchParams({ entryId: next.id, locale: next.locale }, { replace: true });
setSuccess(`Saved immutable revision ${next.current_revision}.`);
return true;
} catch (reason) {
setError(adminErrorMessage(reason));
return false;
} finally {
setSaving(false);
}
}
async function transition(kind: "publish" | "retire") {
if (!selectedEntry || dirty) return;
setSaving(true);
setError("");
try {
const next = await transitionSemanticEntry(
settings,
selectedEntry.id,
kind,
selectedEntry.current_revision,
reason
);
setSelectedEntry(next);
setEntries((current) => [next, ...current.filter((item) => item.id !== next.id)]);
setSuccess(kind === "publish" ? "Semantic documentation published." : "Semantic documentation retired.");
} catch (reason) {
setError(adminErrorMessage(reason));
} finally {
setSaving(false);
}
}
function selectExisting(entry: SemanticEntry, availableSubjects = subjects) {
const subject = availableSubjects.find((item) => sameSubject(item.reference, entry.subject));
setSelectedEntry(entry);
setSelectedSubject(subject ?? descriptorFromEntry(entry));
setContent(entry.content_redacted ? emptyContent() : entry.content);
setLocale(entry.locale);
setSearchParams({ entryId: entry.id, locale: entry.locale }, { replace: true });
}
function selectNew(subject: SemanticSubjectDescriptor) {
const existing = entries.find((entry) => entry.locale === locale && sameSubject(entry.subject, subject.reference));
if (existing) {
selectExisting(existing);
return;
}
setSelectedSubject(subject);
setSelectedEntry(null);
setContent({ ...emptyContent(), title: localized(subject.labels, locale) });
setSearchParams({ locale }, { replace: true });
}
function resetDraft() {
setContent(selectedEntry?.content ?? emptyContent());
}
const sourceStatus = selectedEntry?.subject_resolution.availability ?? "available";
const sortedEntries = useMemo(
() => [...entries].sort((left, right) => left.content.title.localeCompare(right.content.title)),
[entries]
);
return (
<WorkspaceLayout
className="module-workspace docs-semantic-workspace"
primaryLabel="Semantic documentation"
contentLabel="Semantic documentation editor"
primary={(
<aside className="section-sidebar" aria-label="Semantic documentation subjects">
<div className="section-title">Documented subjects</div>
<nav className="section-nav">
{sortedEntries.map((entry) => (
<button
type="button"
key={entry.id}
className={selectedEntry?.id === entry.id ? "active" : ""}
onClick={() => selectExisting(entry)}
>
<span>{entry.content.title}</span>
<small>{entry.subject.subject_kind} · {entry.locale}</small>
</button>
))}
</nav>
<div className="section-title">Available configured subjects</div>
<nav className="section-nav">
{subjects.map((subject) => (
<button type="button" key={subjectKey(subject)} onClick={() => selectNew(subject)}>
<Plus size={14} aria-hidden="true" />
<span>{localized(subject.labels, locale)}</span>
<small>{subject.reference.module_id} · {subject.reference.subject_kind}</small>
</button>
))}
</nav>
</aside>
)}
>
<PageLayout
archetype="editor"
title="Semantic documentation"
description="Explain the tenant-specific meaning and intended use of stable configured subjects."
loading={loading}
error={error}
success={success}
actions={(
<PageActionBar
variant="editor"
state={saving ? "saving" : !valid && dirty ? "invalid" : dirty ? "dirty" : "clean"}
refreshable
reloadAction={{ onReload: () => void load(), loading }}
primaryActions={selectedEntry?.lifecycle_state === "draft" && !dirty ? (
<Button onClick={() => void transition("publish")}>
<Check size={16} /> Publish
</Button>
) : null}
destructiveActions={selectedEntry && !["retired", "superseded"].includes(selectedEntry.lifecycle_state) && !dirty ? (
<Button variant="danger" onClick={() => void transition("retire")}>
<Archive size={16} /> Retire
</Button>
) : null}
discardAction={{ label: "Discard", onClick: resetDraft }}
saveAction={{ label: selectedEntry ? "Save revision" : "Create entry", onClick: () => void save() }}
/>
)}
>
{error ? <DismissibleAlert tone="danger" resetKey={error}>{error}</DismissibleAlert> : null}
{!selectedSubject ? (
<StatePanel size="fill" title="Select a configured subject" description="Choose an existing entry or an authorized subject from an installed module." />
) : (
<ContentGrid columns={2} collapseAt="workspace">
<Card
title={localized(selectedSubject.labels, locale)}
actions={<><StatusBadge status={selectedEntry?.lifecycle_state ?? "draft"} /><StatusBadge status={sourceStatus} /></>}
>
<FormGrid columns={1}>
<FormField label="Locale" help="Entries are locale-specific. Visible fallback is applied at read time.">
<input value={locale} onChange={(event) => setLocale(event.target.value)} maxLength={20} />
</FormField>
<FormField label="Title"><input value={content.title} onChange={(event) => setContent({ ...content, title: event.target.value })} /></FormField>
<FormField label="Summary"><textarea value={content.summary} onChange={(event) => setContent({ ...content, summary: event.target.value })} rows={3} /></FormField>
<FormField label="Meaning"><textarea value={content.meaning} onChange={(event) => setContent({ ...content, meaning: event.target.value })} rows={5} /></FormField>
<FormField label="Body"><textarea value={content.body} onChange={(event) => setContent({ ...content, body: event.target.value })} rows={10} /></FormField>
</FormGrid>
</Card>
<Card title="Governance and use">
<FormGrid columns={1}>
<FormField label="Intended use"><textarea value={content.intended_use} onChange={(event) => setContent({ ...content, intended_use: event.target.value })} rows={5} /></FormField>
<FormField label="Not intended for"><textarea value={content.non_intended_use} onChange={(event) => setContent({ ...content, non_intended_use: event.target.value })} rows={5} /></FormField>
<FormField label="Classification">
<select value={content.classification} onChange={(event) => setContent({ ...content, classification: event.target.value as SemanticContent["classification"] })}>
<option value="internal">Internal</option>
<option value="restricted">Restricted</option>
</select>
</FormField>
<FormField label="Audience selectors" help="One per line: authenticated, account:, group:, role:, function:, or scope:.">
<textarea value={content.audience.join("\n")} onChange={(event) => setContent({ ...content, audience: event.target.value.split("\n").map((item) => item.trim()).filter(Boolean) })} rows={5} />
</FormField>
<FormField label="Change reason"><input value={reason} onChange={(event) => setReason(event.target.value)} maxLength={1000} /></FormField>
</FormGrid>
</Card>
</ContentGrid>
)}
</PageLayout>
</WorkspaceLayout>
);
}
function sameSubject(left: SemanticEntry["subject"], right: SemanticEntry["subject"]): boolean {
return left.module_id === right.module_id
&& left.subject_kind === right.subject_kind
&& left.subject_id === right.subject_id
&& left.anchor?.kind === right.anchor?.kind
&& left.anchor?.id === right.anchor?.id;
}
function subjectKey(subject: SemanticSubjectDescriptor): string {
const reference = subject.reference;
return [reference.module_id, reference.subject_kind, reference.subject_id, reference.anchor?.kind, reference.anchor?.id].filter(Boolean).join(":");
}
function localized(values: Record<string, string>, locale: string): string {
return values[locale] ?? values[locale.split("-")[0]] ?? values.de ?? values.en ?? Object.values(values)[0] ?? "Configured subject";
}
function descriptorFromEntry(entry: SemanticEntry): SemanticSubjectDescriptor {
return {
reference: entry.subject,
labels: { [entry.locale]: entry.content.title },
descriptions: {}
};
}
+5 -2
View File
@@ -3,8 +3,10 @@ import type { PlatformWebModule } from "@govoplan/core-webui";
import { generatedTranslations } from "./i18n/generatedTranslations"; import { generatedTranslations } from "./i18n/generatedTranslations";
const DocsPage = lazy(() => import("./features/docs/DocsPage")); const DocsPage = lazy(() => import("./features/docs/DocsPage"));
const SemanticDocumentationPage = lazy(() => import("./features/docs/SemanticDocumentationPage"));
const docsReadScopes = ["docs:documentation:read", "docs:documentation:admin", "system:settings:read", "admin:settings:read"]; const docsReadScopes = ["docs:documentation:read", "docs:documentation:admin", "system:settings:read", "admin:settings:read"];
const semanticEditorScopes = ["docs:semantic:create", "docs:semantic:edit", "docs:semantic:publish", "docs:semantic:supersede", "docs:semantic:retire"];
const translations = { const translations = {
en: generatedTranslations.en, en: generatedTranslations.en,
@@ -14,13 +16,14 @@ const translations = {
export const docsModule: PlatformWebModule = { export const docsModule: PlatformWebModule = {
id: "docs", id: "docs",
label: "i18n:govoplan-docs.docs.68a41942", label: "i18n:govoplan-docs.docs.68a41942",
version: "0.1.10", version: "0.1.19",
dependencies: ["access"], dependencies: ["access"],
optionalDependencies: ["policy", "audit", "ops", "workflow", "search"], optionalDependencies: ["policy", "audit", "ops", "workflow", "search"],
translations, translations,
navItems: [{ to: "/docs", label: "i18n:govoplan-docs.docs.68a41942", iconName: "reports", anyOf: docsReadScopes, order: 880 }], navItems: [{ to: "/docs", label: "i18n:govoplan-docs.docs.68a41942", iconName: "reports", anyOf: docsReadScopes, order: 880 }],
routes: [ routes: [
{ path: "/docs", anyOf: docsReadScopes, order: 880, render: ({ settings }) => createElement(DocsPage, { settings }) }] { path: "/docs", anyOf: docsReadScopes, order: 880, render: ({ settings }) => createElement(DocsPage, { settings }) },
{ path: "/docs/semantic", anyOf: semanticEditorScopes, order: 881, render: ({ settings }) => createElement(SemanticDocumentationPage, { settings }) }]
}; };
+32
View File
@@ -0,0 +1,32 @@
{
"compilerOptions": {
"target": "ES2022",
"useDefineForClassFields": true,
"lib": ["DOM", "DOM.Iterable", "ES2022"],
"allowJs": false,
"skipLibCheck": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"forceConsistentCasingInFileNames": true,
"module": "ESNext",
"moduleResolution": "Bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"preserveSymlinks": true,
"baseUrl": ".",
"paths": {
"@govoplan/core-webui": ["../../govoplan-core/webui/src/index.ts"],
"@govoplan/core-webui/*": ["../../govoplan-core/webui/src/*"],
"lucide-react": ["../../govoplan-core/webui/node_modules/lucide-react/dist/lucide-react.d.ts"],
"react": ["../../govoplan-core/webui/node_modules/@types/react/index.d.ts"],
"react/jsx-runtime": ["../../govoplan-core/webui/node_modules/@types/react/jsx-runtime.d.ts"],
"react-router": ["../../govoplan-core/webui/node_modules/react-router/dist/production/index.d.ts"]
}
},
"include": ["src", "../../govoplan-core/webui/src/vite-env.d.ts"]
}