From b5a49151ca6eb2a9a152bc7187c995d1ca95598f Mon Sep 17 00:00:00 2001
From: zemion <albrecht@degering.name>
Date: Mon, 6 Jul 2026 14:36:15 +0200
Subject: [PATCH] Sync Repo-README from project files

---
 Repo-README.-.md | 73 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 Repo-README.-.md

diff --git a/Repo-README.-.md b/Repo-README.-.md
new file mode 100644
index 0000000..fd192bd
--- /dev/null
+++ b/Repo-README.-.md
@@ -0,0 +1,73 @@
+<!-- codex-wiki-sync:6c4087efb0469647c0d1bc35 -->
+
+> Mirrored from `/mnt/DATA/git/llm-repro-ui/README.md`.
+> Origin: `repository`.
+> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
+
+---
+# LLM Reproducibility Harness
+
+A local Streamlit UI for testing how repeatable OpenAI Chat Completions are when you hold the prompt, context, model, seed, and generation parameters constant.
+
+The goal is to demonstrate operational reproducibility: identical request payloads, identical outputs, and identical backend metadata where available. It is not a proof that the hosted model is mathematically deterministic.
+
+## What it records
+
+Each run writes a JSON bundle under `runs/` containing:
+
+- exact request payload sent to `v1/chat/completions`
+- SHA-256 hash of the canonicalized request payload
+- repeated trial outputs
+- SHA-256 hash of each output string
+- `system_fingerprint` returned by the API, when available
+- returned model id, finish reason, usage, response id, and raw response
+
+The API key is not stored in bundles.
+
+## Setup
+
+```bash
+python -m venv .venv
+source .venv/bin/activate  # Windows: .venv\Scripts\activate
+pip install -r requirements.txt
+cp .env.example .env
+# Edit .env and set OPENAI_API_KEY, or export it in your shell.
+streamlit run app.py
+```
+
+## Recommended first experiment
+
+Use the default prompt and these settings:
+
+- model: a dated snapshot, for example `gpt-4.1-mini-2025-04-14`
+- repetitions: `3` or `5`
+- seed: `42`
+- temperature: `0`
+- top_p: `1`
+- max_completion_tokens: keep modest, for example `200`
+- streaming: off; tools: not used by this harness
+
+Then change only the seed and run again. You should usually see a different output hash while the request hash changes by exactly the seed field.
+
+## Baseline comparison
+
+After a run is saved, select it in the sidebar as a baseline and run the same request again. The comparison checks:
+
+- whether the canonical request hash is identical
+- whether output hashes are identical in the same order
+- whether the set of returned system fingerprints is identical
+
+## Offline verification
+
+To recompute bundle and output hashes without calling the API:
+
+```bash
+python verify_bundle.py runs/run_*.json
+```
+
+## Caveats
+
+- Hosted LLM reproducibility is best-effort. Matching `seed`, prompt, parameters, and `system_fingerprint` improves repeatability but does not guarantee identical text forever.
+- Rolling model aliases can change. Use snapshot model identifiers where your account and model family support them.
+- Longer completions provide more opportunities for divergence.
+- Tool calls, retrieval, web access, time-dependent prompts, and streaming add more variables. This scaffold deliberately avoids them.