Release Repro Tools 0.2.0
Verify / verify (push) Canceled after 0s

This commit is contained in:
2026-09-02 07:40:24 +02:00
parent 98b1e5c23c
commit bf5b72938a
23 changed files with 1315 additions and 68 deletions
+39
View File
@@ -0,0 +1,39 @@
name: Verify
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
concurrency:
group: verify-${{ gitea.repository }}-${{ gitea.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
verify:
runs-on: ubuntu-latest
timeout-minutes: 45
env:
CI: "true"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "22"
cache: npm
- name: Select declared npm version
run: npm install --global npm@11.17.0
- name: Install dependencies
run: npm ci
- name: Audit runtime dependencies
run: npm audit --omit=dev --audit-level=moderate
- name: Check, test, and build
run: npm run check
- name: Install browser engines
run: npx playwright install --with-deps chromium firefox webkit
- name: Browser tests
run: npm run test:browser
+5
View File
@@ -1,5 +1,10 @@
# Changelog # Changelog
## 0.2.0 - 2026-09-02
- Add streamed hashing, richer deterministic ZIP controls, SPDX-like SBOM
inventory and provenance statements with explicit local signing evidence.
## 0.1.0 - 2026-09-01 ## 0.1.0 - 2026-09-01
- Initial local file/directory SHA-256 and SHA-512 manifest workflow. - Initial local file/directory SHA-256 and SHA-512 manifest workflow.
+6 -4
View File
@@ -5,18 +5,20 @@ Build, compare and package reproducible file inventories locally in the browser.
## Features ## Features
- Multi-file and directory selection with normalized, deterministic code-point path ordering. - Multi-file and directory selection with normalized, deterministic code-point path ordering.
- SHA-256 and SHA-512 content manifests recording source kind, generator version, algorithms and operation semantics. - SHA-256 and SHA-512 content manifests recording source kind, generator version, algorithms and operation semantics, with bounded cancellable chunk reads and byte progress.
- Reference comparison showing matching, changed, missing and unexpected files. - Reference comparison showing matching, changed, missing and unexpected files.
- Stable JSON and spreadsheet-safe CSV reports. - Stable JSON and spreadsheet-safe CSV reports.
- Deterministic ZIP output with sorted entries, fixed ZIP-epoch timestamps and an embedded manifest. - Deterministic ZIP output with sorted entries, fixed ZIP-epoch timestamps and an embedded manifest.
- Streaming ZIP-content manifests and compression-independent comparisons under explicit expansion limits.
- CycloneDX JSON, SPDX JSON and npm-lockfile inspection with portable evidence and an explicitly non-conformant local provenance statement.
- Optional P-256 signature envelopes using a fresh non-exportable, memory-only private key. - Optional P-256 signature envelopes using a fresh non-exportable, memory-only private key.
- Offline PWA, responsive shell and light/dark/system themes. - Offline PWA, responsive shell and light/dark/system themes.
## Limits and interpretation ## Limits and interpretation
Files are limited to 64 MiB each, 256 MiB per selection and 10,000 entries. Individual UTF-8 paths are capped at 4 KiB and aggregate path metadata at 4 MiB. Reference manifests are schema-, count-, digest- and path-validated before comparison. In-browser ZIP creation is capped at 128 MiB and can require substantially more temporary memory than the source files. Timestamps are omitted by default; enabling them deliberately makes the manifest different on every run. Files are limited to 64 MiB each, 256 MiB per selection and 10,000 entries. Individual UTF-8 paths are capped at 4 KiB and aggregate path metadata at 4 MiB. Reference manifests are schema-, count-, digest- and path-validated before comparison. In-browser ZIP creation is capped at 128 MiB and can require substantially more temporary memory than the source files. ZIP inspection is limited to 5,000 entries, 32 MiB per expanded file and 256 MiB expanded in total; entries remain inert browser data and are never written to a filesystem. Web Crypto exposes one-shot SHA finalization, so bounded chunks are retained for each file before the digest call. Timestamps are omitted by default; enabling them deliberately makes the manifest different on every run.
Session signatures prove integrity relative to their public key, not the identity of a person or build system. Authenticate the public key independently. ECDSA signatures are intentionally not byte-reproducible. Session signatures prove integrity relative to their public key, not the identity of a person or build system. Authenticate the public key independently. ECDSA signatures are intentionally not byte-reproducible. Provenance output is local evidence and deliberately makes no SLSA, attestation-authority or builder-identity claim.
## Development ## Development
@@ -27,7 +29,7 @@ npm run test:browser
npm run package:release -- --force npm run package:release -- --force
``` ```
Node.js 22+ and npm 11 are required. The deterministic release is `release/repro-tools-0.1.0.zip` with its SHA-256 sidecar. Node.js 22+ and npm 11 are required. The deterministic release is `release/repro-tools-0.2.0.zip` with its SHA-256 sidecar.
## Licence ## Licence
+2 -2
View File
@@ -1,10 +1,10 @@
# Source identity # Source identity
- Project: Repro Tools - Project: Repro Tools
- Version: 0.1.0 - Version: 0.2.0
- Repository: https://git.add-ideas.de/lotobo/repro-tools - Repository: https://git.add-ideas.de/lotobo/repro-tools
- Licence: GPL-3.0-or-later - Licence: GPL-3.0-or-later
- Build: Node.js 22+, npm 11, `npm ci && npm run release:artifact` - Build: Node.js 22+, npm 11, `npm ci && npm run release:artifact`
- Artifact: `repro-tools-0.1.0.zip` - Artifact: `repro-tools-0.2.0.zip`
Dependencies are exactly pinned by `package-lock.json`; runtime licence texts are bundled under `LICENSES/`. Dependencies are exactly pinned by `package-lock.json`; runtime licence texts are bundled under `LICENSES/`.
+1 -1
View File
@@ -1,3 +1,3 @@
# Third-party notices # Third-party notices
Repro Tools is GPL-3.0-or-later. Runtime dependencies are React, React DOM and the add·ideas Toolbox Contract, Shell and Helpers packages. `fflate` is used only in tests to independently read generated ZIP archives. Exact versions and declared licences are in `package-lock.json`; detected runtime licence texts are generated into `LICENSES/npm-runtime-licenses.txt` in releases. Repro Tools is GPL-3.0-or-later. Runtime dependencies are React, React DOM, `fflate` (MIT), and the add·ideas Toolbox Contract, Shell and Helpers packages. `fflate` streams ZIP contents and independently verifies deterministic ZIP output in tests. Exact versions and declared licences are in `package-lock.json`; detected runtime licence texts are generated into `LICENSES/npm-runtime-licenses.txt` in releases.
+23 -23
View File
@@ -1,22 +1,23 @@
{ {
"name": "repro-tools", "name": "repro-tools",
"version": "0.1.0", "version": "0.2.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "repro-tools", "name": "repro-tools",
"version": "0.1.0", "version": "0.2.0",
"license": "GPL-3.0-or-later", "license": "GPL-3.0-or-later",
"dependencies": { "dependencies": {
"@add-ideas/toolbox-contract": "0.2.3", "@add-ideas/toolbox-contract": "0.3.0",
"@add-ideas/toolbox-helpers": "0.1.0", "@add-ideas/toolbox-helpers": "0.2.0",
"@add-ideas/toolbox-shell-react": "0.2.3", "@add-ideas/toolbox-shell-react": "0.3.0",
"fflate": "0.8.2",
"react": "19.2.8", "react": "19.2.8",
"react-dom": "19.2.8" "react-dom": "19.2.8"
}, },
"devDependencies": { "devDependencies": {
"@add-ideas/toolbox-testkit": "0.2.3", "@add-ideas/toolbox-testkit": "0.3.0",
"@eslint/js": "10.0.1", "@eslint/js": "10.0.1",
"@playwright/test": "1.62.1", "@playwright/test": "1.62.1",
"@testing-library/jest-dom": "6.9.1", "@testing-library/jest-dom": "6.9.1",
@@ -29,7 +30,6 @@
"eslint": "10.7.0", "eslint": "10.7.0",
"eslint-plugin-react-hooks": "7.1.1", "eslint-plugin-react-hooks": "7.1.1",
"eslint-plugin-react-refresh": "0.5.2", "eslint-plugin-react-refresh": "0.5.2",
"fflate": "0.8.2",
"globals": "17.7.0", "globals": "17.7.0",
"jsdom": "29.1.1", "jsdom": "29.1.1",
"prettier": "3.9.5", "prettier": "3.9.5",
@@ -43,24 +43,25 @@
} }
}, },
"node_modules/@add-ideas/toolbox-contract": { "node_modules/@add-ideas/toolbox-contract": {
"version": "0.2.3", "version": "0.3.0",
"resolved": "https://git.add-ideas.de/api/packages/lotobo/npm/%40add-ideas%2Ftoolbox-contract/-/0.2.3/toolbox-contract-0.2.3.tgz", "resolved": "https://git.add-ideas.de/api/packages/lotobo/npm/%40add-ideas%2Ftoolbox-contract/-/0.3.0/toolbox-contract-0.3.0.tgz",
"integrity": "sha512-T0PVSuMT40GjTDfQJhEEY3ZawQq8zz1/ry95JdKI6W39CdLacaRXdGnEpDCMHt+jUbf1Jz7Nat/M5dFCgKVM9A==", "integrity": "sha512-dKrK7BjOFwqJaBfJuhKxZKIld4sH0AKjEn6a0yLnbdMUFY+fFv4VSLGV2tNSBD016gumc2iNqOjUj/ld7x4rtA==",
"license": "Apache-2.0" "license": "Apache-2.0"
}, },
"node_modules/@add-ideas/toolbox-helpers": { "node_modules/@add-ideas/toolbox-helpers": {
"version": "0.1.0", "version": "0.2.0",
"resolved": "https://git.add-ideas.de/api/packages/lotobo/npm/%40add-ideas%2Ftoolbox-helpers/-/0.1.0/toolbox-helpers-0.1.0.tgz", "license": "GPL-3.0-or-later",
"integrity": "sha512-UKl1Oxekedf8D2df86VrnVA53AcMhrnh6iUPXY+k8frirBXotb0yd8SGT+IF/3hcqYwcYe/v9WVFuSgKtIYVnw==", "engines": {
"license": "GPL-3.0-or-later" "node": ">=22"
}
}, },
"node_modules/@add-ideas/toolbox-shell-react": { "node_modules/@add-ideas/toolbox-shell-react": {
"version": "0.2.3", "version": "0.3.0",
"resolved": "https://git.add-ideas.de/api/packages/lotobo/npm/%40add-ideas%2Ftoolbox-shell-react/-/0.2.3/toolbox-shell-react-0.2.3.tgz", "resolved": "https://git.add-ideas.de/api/packages/lotobo/npm/%40add-ideas%2Ftoolbox-shell-react/-/0.3.0/toolbox-shell-react-0.3.0.tgz",
"integrity": "sha512-DT5lQDH48BFkFcmFLZnQh7+Cm73JzBPcmp5WzUXypfkUXpEyDYHzaXgmW4kZ0edSwh4RK4sPmx+JPtK0X4aKCQ==", "integrity": "sha512-74p6JzAOG0YCAKdlc1hLofV4ZIko7vb448S75cIiM88PKm93EHl5VD7g8YVyfM56Ui97UY9dmy+Whiq4sGzpsg==",
"license": "Apache-2.0", "license": "Apache-2.0",
"dependencies": { "dependencies": {
"@add-ideas/toolbox-contract": "0.2.3" "@add-ideas/toolbox-contract": "0.3.0"
}, },
"peerDependencies": { "peerDependencies": {
"react": ">=18 <20", "react": ">=18 <20",
@@ -68,13 +69,13 @@
} }
}, },
"node_modules/@add-ideas/toolbox-testkit": { "node_modules/@add-ideas/toolbox-testkit": {
"version": "0.2.3", "version": "0.3.0",
"resolved": "https://git.add-ideas.de/api/packages/lotobo/npm/%40add-ideas%2Ftoolbox-testkit/-/0.2.3/toolbox-testkit-0.2.3.tgz", "resolved": "https://git.add-ideas.de/api/packages/lotobo/npm/%40add-ideas%2Ftoolbox-testkit/-/0.3.0/toolbox-testkit-0.3.0.tgz",
"integrity": "sha512-sq1MwhKWfFKen+N+124hl74qQimRSvmQ9sOU7jdcI+2qCKZ67+2B8rWyezeV80uTFu4Jv6deHksfYQ/tKNV6XQ==", "integrity": "sha512-4Fk+oSvZFspOMIXr8Xy040nhAaBsIQAzsGyXWSpjn3+k3yBKq7nB1r5zCHhsXzfdLzvPDAx2KcmSNOhM330D9w==",
"dev": true, "dev": true,
"license": "Apache-2.0", "license": "Apache-2.0",
"dependencies": { "dependencies": {
"@add-ideas/toolbox-contract": "0.2.3" "@add-ideas/toolbox-contract": "0.3.0"
}, },
"bin": { "bin": {
"toolbox-check": "dist/cli.js" "toolbox-check": "dist/cli.js"
@@ -2283,7 +2284,6 @@
"version": "0.8.2", "version": "0.8.2",
"resolved": "https://registry.npmjs.org/fflate/-/fflate-0.8.2.tgz", "resolved": "https://registry.npmjs.org/fflate/-/fflate-0.8.2.tgz",
"integrity": "sha512-cPJU47OaAoCbg0pBvzsgpTPhmhqI5eJjh/JIu8tPj5q+T7iLvW/JAYUqmE7KOB4R1ZyEhzBaIQpQpardBF5z8A==", "integrity": "sha512-cPJU47OaAoCbg0pBvzsgpTPhmhqI5eJjh/JIu8tPj5q+T7iLvW/JAYUqmE7KOB4R1ZyEhzBaIQpQpardBF5z8A==",
"dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/file-entry-cache": { "node_modules/file-entry-cache": {
+6 -6
View File
@@ -1,6 +1,6 @@
{ {
"name": "repro-tools", "name": "repro-tools",
"version": "0.1.0", "version": "0.2.0",
"description": "Hash, package and verify reproducible file sets locally in the browser.", "description": "Hash, package and verify reproducible file sets locally in the browser.",
"license": "GPL-3.0-or-later", "license": "GPL-3.0-or-later",
"author": "Albrecht Degering", "author": "Albrecht Degering",
@@ -39,17 +39,17 @@
"release:artifact": "npm run check && npm run test:browser && npm run package:release -- --force" "release:artifact": "npm run check && npm run test:browser && npm run package:release -- --force"
}, },
"dependencies": { "dependencies": {
"@add-ideas/toolbox-helpers": "0.1.0", "@add-ideas/toolbox-helpers": "0.2.0",
"@add-ideas/toolbox-contract": "0.2.3", "@add-ideas/toolbox-contract": "0.3.0",
"@add-ideas/toolbox-shell-react": "0.2.3", "@add-ideas/toolbox-shell-react": "0.3.0",
"fflate": "0.8.2",
"react": "19.2.8", "react": "19.2.8",
"react-dom": "19.2.8" "react-dom": "19.2.8"
}, },
"devDependencies": { "devDependencies": {
"@add-ideas/toolbox-testkit": "0.2.3", "@add-ideas/toolbox-testkit": "0.3.0",
"@eslint/js": "10.0.1", "@eslint/js": "10.0.1",
"@playwright/test": "1.62.1", "@playwright/test": "1.62.1",
"fflate": "0.8.2",
"@testing-library/jest-dom": "6.9.1", "@testing-library/jest-dom": "6.9.1",
"@testing-library/react": "16.3.2", "@testing-library/react": "16.3.2",
"@testing-library/user-event": "14.6.1", "@testing-library/user-event": "14.6.1",
+20 -2
View File
@@ -15,7 +15,25 @@ export default defineConfig({
timeout: 180_000, timeout: 180_000,
}, },
projects: [ projects: [
{ name: "chromium", use: { ...devices["Desktop Chrome"] } }, {
{ name: "firefox", use: { ...devices["Desktop Firefox"] } }, name: "chromium",
testIgnore: /responsive\.spec\.ts/,
use: { ...devices["Desktop Chrome"] },
},
{
name: "firefox",
testIgnore: /responsive\.spec\.ts/,
use: { ...devices["Desktop Firefox"] },
},
{
name: "webkit",
testIgnore: /responsive\.spec\.ts/,
use: { ...devices["Desktop Safari"] },
},
{
name: "mobile-chromium",
testMatch: /responsive\.spec\.ts/,
use: { ...devices["Pixel 5"] },
},
], ],
}); });
+5
View File
@@ -1,5 +1,10 @@
# Changelog # Changelog
## 0.2.0 - 2026-09-02
- Add streamed hashing, richer deterministic ZIP controls, SPDX-like SBOM
inventory and provenance statements with explicit local signing evidence.
## 0.1.0 - 2026-09-01 ## 0.1.0 - 2026-09-01
- Initial local file/directory SHA-256 and SHA-512 manifest workflow. - Initial local file/directory SHA-256 and SHA-512 manifest workflow.
+30 -3
View File
@@ -1,5 +1,5 @@
============================================================================== ==============================================================================
@add-ideas/toolbox-contract@0.2.3 @add-ideas/toolbox-contract@0.3.0
Declared licence: Apache-2.0 Declared licence: Apache-2.0
============================================================================== ==============================================================================
--- LICENSE --- --- LICENSE ---
@@ -198,7 +198,7 @@ Declared licence: Apache-2.0
============================================================================== ==============================================================================
@add-ideas/toolbox-helpers@0.1.0 @add-ideas/toolbox-helpers@0.2.0
Declared licence: GPL-3.0-or-later Declared licence: GPL-3.0-or-later
============================================================================== ==============================================================================
--- LICENSE --- --- LICENSE ---
@@ -879,7 +879,7 @@ Public License instead of this License. But first, please read
============================================================================== ==============================================================================
@add-ideas/toolbox-shell-react@0.2.3 @add-ideas/toolbox-shell-react@0.3.0
Declared licence: Apache-2.0 Declared licence: Apache-2.0
============================================================================== ==============================================================================
--- LICENSE --- --- LICENSE ---
@@ -1077,6 +1077,33 @@ Declared licence: Apache-2.0
limitations under the License. limitations under the License.
==============================================================================
fflate@0.8.2
Declared licence: MIT
==============================================================================
--- LICENSE ---
MIT License
Copyright (c) 2023 Arjun Barrett
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
============================================================================== ==============================================================================
react@19.2.8 react@19.2.8
Declared licence: MIT Declared licence: MIT
+6 -4
View File
@@ -5,18 +5,20 @@ Build, compare and package reproducible file inventories locally in the browser.
## Features ## Features
- Multi-file and directory selection with normalized, deterministic code-point path ordering. - Multi-file and directory selection with normalized, deterministic code-point path ordering.
- SHA-256 and SHA-512 content manifests recording source kind, generator version, algorithms and operation semantics. - SHA-256 and SHA-512 content manifests recording source kind, generator version, algorithms and operation semantics, with bounded cancellable chunk reads and byte progress.
- Reference comparison showing matching, changed, missing and unexpected files. - Reference comparison showing matching, changed, missing and unexpected files.
- Stable JSON and spreadsheet-safe CSV reports. - Stable JSON and spreadsheet-safe CSV reports.
- Deterministic ZIP output with sorted entries, fixed ZIP-epoch timestamps and an embedded manifest. - Deterministic ZIP output with sorted entries, fixed ZIP-epoch timestamps and an embedded manifest.
- Streaming ZIP-content manifests and compression-independent comparisons under explicit expansion limits.
- CycloneDX JSON, SPDX JSON and npm-lockfile inspection with portable evidence and an explicitly non-conformant local provenance statement.
- Optional P-256 signature envelopes using a fresh non-exportable, memory-only private key. - Optional P-256 signature envelopes using a fresh non-exportable, memory-only private key.
- Offline PWA, responsive shell and light/dark/system themes. - Offline PWA, responsive shell and light/dark/system themes.
## Limits and interpretation ## Limits and interpretation
Files are limited to 64 MiB each, 256 MiB per selection and 10,000 entries. Individual UTF-8 paths are capped at 4 KiB and aggregate path metadata at 4 MiB. Reference manifests are schema-, count-, digest- and path-validated before comparison. In-browser ZIP creation is capped at 128 MiB and can require substantially more temporary memory than the source files. Timestamps are omitted by default; enabling them deliberately makes the manifest different on every run. Files are limited to 64 MiB each, 256 MiB per selection and 10,000 entries. Individual UTF-8 paths are capped at 4 KiB and aggregate path metadata at 4 MiB. Reference manifests are schema-, count-, digest- and path-validated before comparison. In-browser ZIP creation is capped at 128 MiB and can require substantially more temporary memory than the source files. ZIP inspection is limited to 5,000 entries, 32 MiB per expanded file and 256 MiB expanded in total; entries remain inert browser data and are never written to a filesystem. Web Crypto exposes one-shot SHA finalization, so bounded chunks are retained for each file before the digest call. Timestamps are omitted by default; enabling them deliberately makes the manifest different on every run.
Session signatures prove integrity relative to their public key, not the identity of a person or build system. Authenticate the public key independently. ECDSA signatures are intentionally not byte-reproducible. Session signatures prove integrity relative to their public key, not the identity of a person or build system. Authenticate the public key independently. ECDSA signatures are intentionally not byte-reproducible. Provenance output is local evidence and deliberately makes no SLSA, attestation-authority or builder-identity claim.
## Development ## Development
@@ -27,7 +29,7 @@ npm run test:browser
npm run package:release -- --force npm run package:release -- --force
``` ```
Node.js 22+ and npm 11 are required. The deterministic release is `release/repro-tools-0.1.0.zip` with its SHA-256 sidecar. Node.js 22+ and npm 11 are required. The deterministic release is `release/repro-tools-0.2.0.zip` with its SHA-256 sidecar.
## Licence ## Licence
+2 -2
View File
@@ -1,10 +1,10 @@
# Source identity # Source identity
- Project: Repro Tools - Project: Repro Tools
- Version: 0.1.0 - Version: 0.2.0
- Repository: https://git.add-ideas.de/lotobo/repro-tools - Repository: https://git.add-ideas.de/lotobo/repro-tools
- Licence: GPL-3.0-or-later - Licence: GPL-3.0-or-later
- Build: Node.js 22+, npm 11, `npm ci && npm run release:artifact` - Build: Node.js 22+, npm 11, `npm ci && npm run release:artifact`
- Artifact: `repro-tools-0.1.0.zip` - Artifact: `repro-tools-0.2.0.zip`
Dependencies are exactly pinned by `package-lock.json`; runtime licence texts are bundled under `LICENSES/`. Dependencies are exactly pinned by `package-lock.json`; runtime licence texts are bundled under `LICENSES/`.
+1 -1
View File
@@ -1,3 +1,3 @@
# Third-party notices # Third-party notices
Repro Tools is GPL-3.0-or-later. Runtime dependencies are React, React DOM and the add·ideas Toolbox Contract, Shell and Helpers packages. `fflate` is used only in tests to independently read generated ZIP archives. Exact versions and declared licences are in `package-lock.json`; detected runtime licence texts are generated into `LICENSES/npm-runtime-licenses.txt` in releases. Repro Tools is GPL-3.0-or-later. Runtime dependencies are React, React DOM, `fflate` (MIT), and the add·ideas Toolbox Contract, Shell and Helpers packages. `fflate` streams ZIP contents and independently verifies deterministic ZIP output in tests. Exact versions and declared licences are in `package-lock.json`; detected runtime licence texts are generated into `LICENSES/npm-runtime-licenses.txt` in releases.
+1 -1
View File
@@ -1,5 +1,5 @@
const CACHE_PREFIX = "repro-tools-shell-"; const CACHE_PREFIX = "repro-tools-shell-";
const CACHE_NAME = CACHE_PREFIX + "0.1.0"; const CACHE_NAME = CACHE_PREFIX + "0.2.0";
const CORE = ["./", "./manifest.webmanifest", "./favicon.svg"]; const CORE = ["./", "./manifest.webmanifest", "./favicon.svg"];
self.addEventListener("install", (event) => { self.addEventListener("install", (event) => {
event.waitUntil( event.waitUntil(
+14 -1
View File
@@ -3,7 +3,7 @@
"schemaVersion": 1, "schemaVersion": 1,
"id": "de.add-ideas.repro-tools", "id": "de.add-ideas.repro-tools",
"name": "Repro Tools", "name": "Repro Tools",
"version": "0.1.0", "version": "0.2.0",
"description": "Build and verify reproducible file manifests locally.", "description": "Build and verify reproducible file manifests locally.",
"entry": "./", "entry": "./",
"icon": "./favicon.svg", "icon": "./favicon.svg",
@@ -21,6 +21,19 @@
"crossOriginIsolated": false, "crossOriginIsolated": false,
"topLevelContext": false "topLevelContext": false
}, },
"io": {
"accepts": [
{ "mediaType": "*/*", "extensions": [] },
{ "mediaType": "application/zip", "extensions": [".zip"] },
{ "mediaType": "application/json", "extensions": [".json"] }
],
"produces": [
{ "mediaType": "application/json", "extensions": [".json"] },
{ "mediaType": "text/csv", "extensions": [".csv"] },
{ "mediaType": "application/zip", "extensions": [".zip"] }
]
},
"capabilities": { "required": [], "optional": ["web-crypto"] },
"privacy": { "privacy": {
"processing": "local", "processing": "local",
"fileUploads": true, "fileUploads": true,
+262 -6
View File
@@ -1,25 +1,38 @@
import { useEffect, useMemo, useRef, useState } from "react"; import { useEffect, useMemo, useRef, useState } from "react";
import { import {
formatBytes,
stableStringify, stableStringify,
triggerBlobDownload, triggerBlobDownload,
} from "@add-ideas/toolbox-helpers"; } from "@add-ideas/toolbox-helpers";
import { import {
buildArchiveContentManifest,
buildManifest, buildManifest,
collectSelection, collectSelection,
compareArchiveContentManifests,
compareManifests, compareManifests,
createSessionSigner, createSessionSigner,
deterministicZip, deterministicZip,
manifestCsv, manifestCsv,
MANIFEST_TEXT_LIMIT, MANIFEST_TEXT_LIMIT,
parseArchiveContentManifest,
parseManifest, parseManifest,
SIGNATURE_TEXT_LIMIT, SIGNATURE_TEXT_LIMIT,
serializeArchiveContentManifest,
serializeManifest, serializeManifest,
verifySignature, verifySignature,
type ArchiveContentManifest,
type Comparison, type Comparison,
type DigestAlgorithm, type DigestAlgorithm,
type ReproManifest, type ReproManifest,
type SelectedFile, type SelectedFile,
} from "../core/repro"; } from "../core/repro";
import {
createProvenanceStatement,
createReproEvidence,
inspectSbomDocuments,
serializeProvenance,
type SbomEvidence,
} from "../core/evidence";
export function Workbench() { export function Workbench() {
const [selected, setSelected] = useState<SelectedFile[]>([]); const [selected, setSelected] = useState<SelectedFile[]>([]);
@@ -32,10 +45,18 @@ export function Workbench() {
const [reference, setReference] = useState(""); const [reference, setReference] = useState("");
const [comparison, setComparison] = useState<Comparison[]>([]); const [comparison, setComparison] = useState<Comparison[]>([]);
const [signature, setSignature] = useState(""); const [signature, setSignature] = useState("");
const [archiveManifest, setArchiveManifest] =
useState<ArchiveContentManifest | null>(null);
const [archiveReference, setArchiveReference] = useState("");
const [archiveComparison, setArchiveComparison] = useState<Comparison[]>([]);
const [sbomEvidence, setSbomEvidence] = useState<SbomEvidence[]>([]);
const [provenance, setProvenance] = useState("");
const [evidence, setEvidence] = useState("");
const [status, setStatus] = useState("Choose files or a directory to begin."); const [status, setStatus] = useState("Choose files or a directory to begin.");
const [busy, setBusy] = useState(false); const [busy, setBusy] = useState(false);
const fileInput = useRef<HTMLInputElement>(null); const fileInput = useRef<HTMLInputElement>(null);
const directoryInput = useRef<HTMLInputElement>(null); const directoryInput = useRef<HTMLInputElement>(null);
const archiveInput = useRef<HTMLInputElement>(null);
useEffect(() => { useEffect(() => {
directoryInput.current?.setAttribute("webkitdirectory", ""); directoryInput.current?.setAttribute("webkitdirectory", "");
@@ -59,6 +80,9 @@ export function Workbench() {
setManifest(null); setManifest(null);
setComparison([]); setComparison([]);
setSignature(""); setSignature("");
setSbomEvidence([]);
setProvenance("");
setEvidence("");
setStatus( setStatus(
`${next.length.toLocaleString()} file${next.length === 1 ? "" : "s"} ready.`, `${next.length.toLocaleString()} file${next.length === 1 ? "" : "s"} ready.`,
); );
@@ -189,6 +213,77 @@ export function Workbench() {
} }
} }
async function inspectArchive(file: File | undefined): Promise<void> {
if (!file) return;
setBusy(true);
try {
const next = await buildArchiveContentManifest(file, algorithms, {
onProgress: (progress) =>
setStatus(
progress.phase === "read"
? `Reading ZIP: ${formatBytes(progress.archiveBytes)} of ${formatBytes(progress.archiveTotalBytes)}`
: `${progress.phase === "expand" ? "Expanding" : "Hashing"} ${progress.path ?? "entries"}`,
),
});
setArchiveManifest(next);
setArchiveComparison([]);
setStatus(
`ZIP content manifest ready: ${next.fileCount.toLocaleString()} files, ${formatBytes(next.totalBytes)} expanded.`,
);
} catch (caught) {
setArchiveManifest(null);
setStatus(caught instanceof Error ? caught.message : String(caught));
} finally {
setBusy(false);
if (archiveInput.current) archiveInput.current.value = "";
}
}
function compareArchive(): void {
if (!archiveManifest) return;
try {
const results = compareArchiveContentManifests(
parseArchiveContentManifest(archiveReference),
archiveManifest,
);
setArchiveComparison(results);
const changed = results.filter((item) => item.status !== "match").length;
setStatus(
changed === 0
? "Archive contents match the reference."
: `Archive comparison found ${changed.toLocaleString()} difference${changed === 1 ? "" : "s"}.`,
);
} catch (caught) {
setArchiveComparison([]);
setStatus(caught instanceof Error ? caught.message : String(caught));
}
}
async function inspectEvidence(): Promise<void> {
if (!manifest) return;
setBusy(true);
try {
setStatus("Inspecting local SBOM and lockfile candidates…");
const materials = await inspectSbomDocuments(selected);
setSbomEvidence(materials);
setProvenance(
serializeProvenance(
createProvenanceStatement(manifest, materials, { timestamp }),
),
);
setEvidence(
serializeProvenance(createReproEvidence(manifest, materials)),
);
setStatus(
`Provenance evidence ready; ${materials.filter((item) => item.status === "recognized").length.toLocaleString()} structured dependency document${materials.length === 1 ? "" : "s"} recognized.`,
);
} catch (caught) {
setStatus(caught instanceof Error ? caught.message : String(caught));
} finally {
setBusy(false);
}
}
return ( return (
<main className="workbench"> <main className="workbench">
<section className="hero"> <section className="hero">
@@ -224,6 +319,7 @@ export function Workbench() {
ref={fileInput} ref={fileInput}
className="sr-only" className="sr-only"
type="file" type="file"
aria-label="Choose files for reproducibility manifest"
multiple multiple
onChange={(event) => acceptFiles(event.target.files, "files")} onChange={(event) => acceptFiles(event.target.files, "files")}
/> />
@@ -231,6 +327,7 @@ export function Workbench() {
ref={directoryInput} ref={directoryInput}
className="sr-only" className="sr-only"
type="file" type="file"
aria-label="Choose directory for reproducibility manifest"
multiple multiple
onChange={(event) => acceptFiles(event.target.files, "directory")} onChange={(event) => acceptFiles(event.target.files, "directory")}
/> />
@@ -399,6 +496,171 @@ export function Workbench() {
</section> </section>
</section> </section>
<section className="output-grid">
<section className="panel" aria-labelledby="archive-title">
<div className="panel-heading">
<div>
<p className="eyebrow">Expanded-content evidence</p>
<h2 id="archive-title">ZIP content manifest</h2>
</div>
</div>
<p className="muted">
Stream a ZIP through strict entry, per-file and expanded-size
ceilings. The inventory hashes uncompressed content, so it can
compare archives whose compression or metadata differs.
</p>
<div className="button-row">
<button
type="button"
disabled={busy || algorithms.length === 0}
onClick={() => archiveInput.current?.click()}
>
Inspect ZIP
</button>
<input
ref={archiveInput}
className="sr-only"
type="file"
aria-label="Choose ZIP archive to inspect"
accept="application/zip,.zip"
onChange={(event) => void inspectArchive(event.target.files?.[0])}
/>
<button
type="button"
disabled={!archiveManifest}
onClick={() =>
archiveManifest &&
triggerBlobDownload(
new Blob([serializeArchiveContentManifest(archiveManifest)], {
type: "application/json",
}),
"archive-content-manifest.json",
)
}
>
Download content manifest
</button>
</div>
{archiveManifest && (
<textarea
rows={12}
readOnly
aria-label="Archive content manifest"
value={serializeArchiveContentManifest(archiveManifest)}
/>
)}
</section>
<section className="panel" aria-labelledby="archive-compare-title">
<div className="panel-heading">
<div>
<p className="eyebrow">Compression-independent comparison</p>
<h2 id="archive-compare-title">Compare ZIP contents</h2>
</div>
</div>
<label>
Reference archive-content manifest
<textarea
rows={12}
maxLength={MANIFEST_TEXT_LIMIT}
value={archiveReference}
onChange={(event) => setArchiveReference(event.target.value)}
placeholder="Paste a previous archive-content-manifest-v1 document."
/>
</label>
<button
type="button"
className="primary-button"
disabled={!archiveManifest || !archiveReference.trim()}
onClick={compareArchive}
>
Compare expanded contents
</button>
{archiveComparison.length > 0 && (
<ul className="comparison">
{archiveComparison.map((item) => (
<li className={item.status} key={item.path}>
<strong>{item.status}</strong>
<code>{item.path}</code>
<span>{item.detail}</span>
</li>
))}
</ul>
)}
</section>
</section>
<section className="panel" aria-labelledby="provenance-title">
<div className="panel-heading">
<div>
<p className="eyebrow">Portable local evidence</p>
<h2 id="provenance-title">Provenance and SBOM evidence</h2>
</div>
</div>
<p className="muted">
Recognizes CycloneDX JSON, SPDX JSON and npm lockfiles in the current
selection. The generated statement records local provenance without
claiming SLSA or attestation conformance.
</p>
<div className="button-row">
<button
type="button"
disabled={!manifest || busy}
onClick={() => void inspectEvidence()}
>
Build provenance evidence
</button>
<button
type="button"
disabled={!provenance}
onClick={() =>
triggerBlobDownload(
new Blob([provenance], { type: "application/json" }),
"repro-provenance.json",
)
}
>
Download provenance
</button>
<button
type="button"
disabled={!evidence}
onClick={() =>
triggerBlobDownload(
new Blob([evidence], { type: "application/json" }),
"repro-evidence.json",
)
}
>
Download evidence contract
</button>
</div>
{sbomEvidence.length > 0 && (
<div className="file-table" tabIndex={0}>
<table>
<thead>
<tr>
<th>Path</th>
<th>Recognition</th>
<th>Components</th>
</tr>
</thead>
<tbody>
{sbomEvidence.map((item) => (
<tr key={item.path}>
<td>
<code>{item.path}</code>
</td>
<td>{item.format ?? item.status}</td>
<td>{item.componentCount.toLocaleString()}</td>
</tr>
))}
</tbody>
</table>
</div>
)}
</section>
<section className="panel" aria-labelledby="signature-title"> <section className="panel" aria-labelledby="signature-title">
<div className="panel-heading"> <div className="panel-heading">
<div> <div>
@@ -452,9 +714,3 @@ export function Workbench() {
</main> </main>
); );
} }
function formatBytes(value: number): string {
if (value < 1024) return `${value} B`;
if (value < 1024 ** 2) return `${(value / 1024).toFixed(1)} KiB`;
return `${(value / 1024 ** 2).toFixed(1)} MiB`;
}
+328
View File
@@ -0,0 +1,328 @@
import {
digestByteSourceHex,
safeJsonParse,
stableStringify,
} from "@add-ideas/toolbox-helpers";
import {
APP_VERSION,
FILE_LIMIT,
MANIFEST_TEXT_LIMIT,
type ReproManifest,
type SelectedFile,
} from "./repro";
export const EVIDENCE_SCHEMA =
"https://git.add-ideas.de/lotobo/format-lab/schema/evidence-v1" as const;
export const PROVENANCE_SCHEMA =
"https://git.add-ideas.de/lotobo/repro-tools/schema/provenance-v1" as const;
export interface EvidenceObservation {
id: string;
status: "pass" | "change" | "not-tested" | "informational";
expectation?: string;
detail: string;
}
export interface EvidenceReport {
schema: typeof EVIDENCE_SCHEMA;
contractVersion: 1;
generator: { id: string; version: string };
subject: { kind: string; name: string; mediaType?: string };
operation: { id: string; parameters: Record<string, unknown> };
limits: Record<string, number | string | boolean>;
observations: EvidenceObservation[];
provenance: {
execution: "local-browser";
networkRequired: false;
recordedAt: string | null;
};
}
export type SbomFormat = "CycloneDX JSON" | "SPDX JSON" | "npm lockfile";
export interface SbomEvidence {
path: string;
status: "recognized" | "invalid" | "unsupported";
format: SbomFormat | null;
specificationVersion: string | null;
documentName: string | null;
serialNumber: string | null;
componentCount: number;
relationshipCount: number;
sha256: string;
detail: string;
}
export interface ProvenanceStatement {
schema: typeof PROVENANCE_SCHEMA;
statementVersion: 1;
generator: { id: "de.add-ideas.repro-tools"; version: string };
subject: Array<{
name: string;
size: number;
digest: Record<string, string>;
}>;
predicateType: "de.add-ideas.local-file-set/v1";
predicate: {
buildType: "local-browser-file-inventory";
deterministicOrdering: true;
sourceManifestSchema: ReproManifest["schema"];
materials: SbomEvidence[];
metadata: {
networkRequired: false;
recordedAt: string | null;
conformanceClaim: null;
};
};
}
export async function inspectSbomDocuments(
selected: readonly SelectedFile[],
options: { signal?: AbortSignal } = {},
): Promise<SbomEvidence[]> {
const candidates = selected
.filter((item) => /(?:\.json|package-lock\.json)$/iu.test(item.path))
.sort((left, right) =>
left.path === right.path ? 0 : left.path < right.path ? -1 : 1,
);
const result: SbomEvidence[] = [];
for (const item of candidates) {
if (item.file.size > MANIFEST_TEXT_LIMIT) {
result.push({
path: item.path,
status: "invalid",
format: null,
specificationVersion: null,
documentName: null,
serialNumber: null,
componentCount: 0,
relationshipCount: 0,
sha256: await digestByteSourceHex(item.file, "SHA-256", {
maximumBytes: FILE_LIMIT,
knownTotalBytes: item.file.size,
signal: options.signal,
}),
detail: "JSON exceeds the 4 MB structured-evidence parsing limit.",
});
continue;
}
const sha256 = await digestByteSourceHex(item.file, "SHA-256", {
maximumBytes: FILE_LIMIT,
knownTotalBytes: item.file.size,
signal: options.signal,
});
try {
const parsed = safeJsonParse(await item.file.text(), {
maxTextChars: MANIFEST_TEXT_LIMIT,
maxDepth: 64,
maxNodes: 250_000,
rejectDangerousKeys: true,
});
result.push(classifySbom(item.path, parsed, sha256));
} catch (caught) {
result.push({
path: item.path,
status: "invalid",
format: null,
specificationVersion: null,
documentName: null,
serialNumber: null,
componentCount: 0,
relationshipCount: 0,
sha256,
detail: caught instanceof Error ? caught.message : String(caught),
});
}
}
return result;
}
function classifySbom(
path: string,
value: unknown,
sha256: string,
): SbomEvidence {
const base = {
path,
sha256,
serialNumber: null,
componentCount: 0,
relationshipCount: 0,
};
if (!isRecord(value))
return {
...base,
status: "unsupported",
format: null,
specificationVersion: null,
documentName: null,
detail: "JSON root is not an object.",
};
if (value.bomFormat === "CycloneDX") {
const components = Array.isArray(value.components) ? value.components : [];
const dependencies = Array.isArray(value.dependencies)
? value.dependencies
: [];
return {
...base,
status: typeof value.specVersion === "string" ? "recognized" : "invalid",
format: "CycloneDX JSON",
specificationVersion:
typeof value.specVersion === "string" ? value.specVersion : null,
documentName: readNestedName(value.metadata),
serialNumber:
typeof value.serialNumber === "string" ? value.serialNumber : null,
componentCount: components.length,
relationshipCount: dependencies.length,
detail:
typeof value.specVersion === "string"
? "CycloneDX identity and top-level counts inspected."
: "CycloneDX bomFormat is present but specVersion is missing.",
};
}
if (typeof value.spdxVersion === "string") {
const packages = Array.isArray(value.packages) ? value.packages : [];
const relationships = Array.isArray(value.relationships)
? value.relationships
: [];
const valid = /^SPDX-\d+\.\d+$/u.test(value.spdxVersion);
return {
...base,
status: valid ? "recognized" : "invalid",
format: "SPDX JSON",
specificationVersion: value.spdxVersion,
documentName: typeof value.name === "string" ? value.name : null,
componentCount: packages.length,
relationshipCount: relationships.length,
detail: valid
? "SPDX identity and top-level counts inspected."
: "spdxVersion is not a recognized SPDX version string.",
};
}
if (
path.endsWith("package-lock.json") &&
Number.isSafeInteger(value.lockfileVersion)
) {
const packages = isRecord(value.packages)
? Object.keys(value.packages).length
: isRecord(value.dependencies)
? Object.keys(value.dependencies).length
: 0;
return {
...base,
status: "recognized",
format: "npm lockfile",
specificationVersion: String(value.lockfileVersion),
documentName: typeof value.name === "string" ? value.name : null,
componentCount: packages,
relationshipCount: 0,
detail:
"npm lockfile inventory recorded as dependency provenance; it is not claimed to be an SBOM standard.",
};
}
return {
...base,
status: "unsupported",
format: null,
specificationVersion: null,
documentName: null,
detail: "JSON is not recognized as CycloneDX, SPDX, or an npm lockfile.",
};
}
export function createProvenanceStatement(
manifest: ReproManifest,
materials: readonly SbomEvidence[],
options: { timestamp?: boolean } = {},
): ProvenanceStatement {
return {
schema: PROVENANCE_SCHEMA,
statementVersion: 1,
generator: { id: "de.add-ideas.repro-tools", version: APP_VERSION },
subject: manifest.files.map((file) => ({
name: file.path,
size: file.size,
digest: { ...file.digests },
})),
predicateType: "de.add-ideas.local-file-set/v1",
predicate: {
buildType: "local-browser-file-inventory",
deterministicOrdering: true,
sourceManifestSchema: manifest.schema,
materials: materials.map((item) => ({ ...item })),
metadata: {
networkRequired: false,
recordedAt: options.timestamp ? new Date().toISOString() : null,
conformanceClaim: null,
},
},
};
}
export function createReproEvidence(
manifest: ReproManifest,
materials: readonly SbomEvidence[],
): EvidenceReport {
const recognized = materials.filter(
(item) => item.status === "recognized",
).length;
return {
schema: EVIDENCE_SCHEMA,
contractVersion: 1,
generator: { id: "de.add-ideas.repro-tools", version: APP_VERSION },
subject: {
kind: "file-set",
name: manifest.source.root ?? "selected-files",
},
operation: {
id: "reproducibility.inventory",
parameters: {
algorithms: [...manifest.operation.algorithms],
ordering: manifest.operation.ordering,
},
},
limits: {
fileBytes: FILE_LIMIT,
manifestTextBytes: MANIFEST_TEXT_LIMIT,
},
observations: [
{
id: "manifest.file-count",
status: "informational",
detail: `${manifest.fileCount.toLocaleString()} files and ${manifest.totalBytes.toLocaleString()} bytes were inventoried.`,
},
{
id: "sbom.recognized",
status: materials.length === 0 ? "not-tested" : "informational",
detail:
materials.length === 0
? "No JSON SBOM or lockfile candidate was selected."
: `${recognized.toLocaleString()} of ${materials.length.toLocaleString()} JSON candidates were recognized as SBOM or lockfile evidence.`,
},
],
provenance: {
execution: "local-browser",
networkRequired: false,
recordedAt: manifest.operation.recordedAt,
},
};
}
export function serializeProvenance(
value: ProvenanceStatement | EvidenceReport,
): string {
return `${stableStringify(value, 2, {
maxTextChars: MANIFEST_TEXT_LIMIT,
maxDepth: 64,
maxNodes: 250_000,
})}\n`;
}
function readNestedName(value: unknown): string | null {
if (!isRecord(value) || !isRecord(value.component)) return null;
return typeof value.component.name === "string" ? value.component.name : null;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return Boolean(value) && !Array.isArray(value) && typeof value === "object";
}
+393 -10
View File
@@ -1,12 +1,15 @@
import { import {
base64ToBytes, base64ToBytes,
bytesToBase64, bytesToBase64,
createIncrementalChecksum,
crc32, crc32,
digestHex, digestByteSourceHex,
iterateByteChunks,
safeJsonParse, safeJsonParse,
stableStringify, stableStringify,
stringifyCsv, stringifyCsv,
} from "@add-ideas/toolbox-helpers"; } from "@add-ideas/toolbox-helpers";
import { Unzip, UnzipInflate } from "fflate";
export const FILE_LIMIT = 64 * 1024 * 1024; export const FILE_LIMIT = 64 * 1024 * 1024;
export const TOTAL_LIMIT = 256 * 1024 * 1024; export const TOTAL_LIMIT = 256 * 1024 * 1024;
@@ -16,7 +19,10 @@ export const MANIFEST_TEXT_LIMIT = 4_000_000;
export const SIGNATURE_TEXT_LIMIT = 32_768; export const SIGNATURE_TEXT_LIMIT = 32_768;
export const PATH_BYTES_LIMIT = 4_096; export const PATH_BYTES_LIMIT = 4_096;
export const TOTAL_PATH_BYTES_LIMIT = 4 * 1024 * 1024; export const TOTAL_PATH_BYTES_LIMIT = 4 * 1024 * 1024;
export const APP_VERSION = "0.1.0"; export const ARCHIVE_ENTRY_LIMIT = 5_000;
export const ARCHIVE_ENTRY_SIZE_LIMIT = 32 * 1024 * 1024;
export const ARCHIVE_TOTAL_SIZE_LIMIT = 256 * 1024 * 1024;
export const APP_VERSION = "0.2.0";
export type DigestAlgorithm = "SHA-256" | "SHA-512"; export type DigestAlgorithm = "SHA-256" | "SHA-512";
export interface SelectedFile { export interface SelectedFile {
@@ -52,6 +58,41 @@ export interface Comparison {
detail: string; detail: string;
} }
export interface ArchiveContentEntry extends ManifestEntry {
compressedSize: number | null;
crc32: string;
}
export interface ArchiveContentManifest {
schema: "https://git.add-ideas.de/lotobo/repro-tools/schema/archive-content-manifest-v1";
manifestVersion: 1;
generator: { id: "de.add-ideas.repro-tools"; version: string };
operation: {
name: "inspect-zip-content";
ordering: "unicode-code-point-path";
algorithms: DigestAlgorithm[];
limits: {
entries: number;
entryBytes: number;
expandedBytes: number;
archiveBytes: number;
};
};
archive: { name: string; size: number };
fileCount: number;
totalBytes: number;
files: ArchiveContentEntry[];
}
export interface ArchiveProgress {
phase: "read" | "expand" | "hash";
archiveBytes: number;
archiveTotalBytes: number;
entries: number;
expandedBytes: number;
path?: string;
}
const compareText = (left: string, right: string) => const compareText = (left: string, right: string) =>
left === right ? 0 : left < right ? -1 : 1; left === right ? 0 : left < right ? -1 : 1;
@@ -122,6 +163,8 @@ export async function buildManifest(
root?: string | null; root?: string | null;
timestamp?: boolean; timestamp?: boolean;
onProgress?: (done: number, total: number) => void; onProgress?: (done: number, total: number) => void;
onByteProgress?: (done: number, total: number, path: string) => void;
signal?: AbortSignal;
} = {}, } = {},
): Promise<ReproManifest> { ): Promise<ReproManifest> {
if (algorithms.length === 0) if (algorithms.length === 0)
@@ -132,20 +175,27 @@ export async function buildManifest(
for (const [index, item] of normalized for (const [index, item] of normalized
.sort((a, b) => compareText(a.path, b.path)) .sort((a, b) => compareText(a.path, b.path))
.entries()) { .entries()) {
const data = new Uint8Array(await item.file.arrayBuffer()); totalBytes += item.file.size;
if (data.length > FILE_LIMIT)
throw new RangeError(`${item.path} exceeds the 64 MiB per-file limit.`);
totalBytes += data.length;
if (totalBytes > TOTAL_LIMIT) if (totalBytes > TOTAL_LIMIT)
throw new RangeError("The selection exceeds the 256 MiB total limit."); throw new RangeError("The selection exceeds the 256 MiB total limit.");
const digests: Record<string, string> = {}; const digests: Record<string, string> = {};
for (const algorithm of algorithms) for (const algorithm of algorithms)
digests[algorithm.toLowerCase()] = await digestHex( digests[algorithm.toLowerCase()] = await digestByteSourceHex(
data, item.file,
algorithm, algorithm,
FILE_LIMIT, {
maximumBytes: FILE_LIMIT,
knownTotalBytes: item.file.size,
signal: options.signal,
onProgress: ({ processedBytes }) =>
options.onByteProgress?.(processedBytes, item.file.size, item.path),
},
); );
files.push({ path: normalizePath(item.path), size: data.length, digests }); files.push({
path: normalizePath(item.path),
size: item.file.size,
digests,
});
options.onProgress?.(index + 1, selected.length); options.onProgress?.(index + 1, selected.length);
} }
const orderedAlgorithms = [...new Set(algorithms)].sort( const orderedAlgorithms = [...new Set(algorithms)].sort(
@@ -346,6 +396,339 @@ export function compareManifests(
}); });
} }
/**
* Expands ZIP entries as bounded streams and inventories their content. The
* archive is never mounted and entry names are treated only as inert paths.
*/
export async function buildArchiveContentManifest(
archive: File,
algorithms: readonly DigestAlgorithm[] = ["SHA-256"],
options: {
signal?: AbortSignal;
onProgress?: (progress: ArchiveProgress) => void;
} = {},
): Promise<ArchiveContentManifest> {
if (archive.size > ZIP_LIMIT)
throw new RangeError("ZIP inspection is limited to 128 MiB archives.");
if (algorithms.length === 0)
throw new TypeError("Select at least one digest algorithm.");
const orderedAlgorithms = [...new Set(algorithms)].sort(
compareText,
) as DigestAlgorithm[];
const signature = new Uint8Array(await archive.slice(0, 4).arrayBuffer());
const signatureValue =
signature.length === 4
? new DataView(signature.buffer).getUint32(0, true)
: 0;
if (
signatureValue !== 0x04034b50 &&
signatureValue !== 0x06054b50 &&
signatureValue !== 0x08074b50
)
throw new TypeError(
"Archive content inspection currently supports ZIP files only.",
);
interface EntryDraft {
path: string;
compressedSize: number | null;
size: number;
crc32: string;
chunks: Uint8Array<ArrayBuffer>[];
}
const drafts: EntryDraft[] = [];
const paths = new Set<string>();
let entryCount = 0;
let totalBytes = 0;
let archiveBytes = 0;
let callbackError: Error | null = null;
const unzip = new Unzip((entry) => {
if (callbackError) {
entry.terminate();
return;
}
try {
entryCount += 1;
if (entryCount > ARCHIVE_ENTRY_LIMIT)
throw new RangeError(
`ZIP archives are limited to ${ARCHIVE_ENTRY_LIMIT.toLocaleString()} entries.`,
);
if (entry.name.endsWith("/")) {
entry.ondata = (error) => {
if (error && !callbackError) callbackError = error;
};
entry.start();
return;
}
const path = normalizePath(entry.name);
if (paths.has(path))
throw new TypeError(`ZIP contains duplicate normalized path: ${path}`);
paths.add(path);
if (
entry.originalSize !== undefined &&
entry.originalSize > ARCHIVE_ENTRY_SIZE_LIMIT
)
throw new RangeError(
`${path} exceeds the 32 MiB expanded-entry limit.`,
);
const checksum = createIncrementalChecksum(
"CRC-32",
ARCHIVE_ENTRY_SIZE_LIMIT,
);
const chunks: Uint8Array<ArrayBuffer>[] = [];
let size = 0;
entry.ondata = (error, chunk, final) => {
if (callbackError) return;
if (error) {
callbackError = error;
return;
}
try {
if (options.signal?.aborted) throw abortReason(options.signal);
const owned = new Uint8Array(chunk.length);
owned.set(chunk);
size += owned.length;
if (size > ARCHIVE_ENTRY_SIZE_LIMIT)
throw new RangeError(
`${path} exceeds the 32 MiB expanded-entry limit.`,
);
totalBytes += owned.length;
if (totalBytes > ARCHIVE_TOTAL_SIZE_LIMIT)
throw new RangeError(
"ZIP contents exceed the 256 MiB expanded-size limit.",
);
checksum.update(owned);
if (owned.length > 0) chunks.push(owned);
options.onProgress?.({
phase: "expand",
archiveBytes,
archiveTotalBytes: archive.size,
entries: entryCount,
expandedBytes: totalBytes,
path,
});
if (!final) return;
if (entry.originalSize !== undefined && entry.originalSize !== size)
throw new TypeError(
`${path} expanded to ${size} bytes; its ZIP header declared ${entry.originalSize}.`,
);
drafts.push({
path,
compressedSize: entry.size ?? null,
size,
crc32: checksum.digestHex(),
chunks,
});
} catch (caught) {
callbackError = asError(caught);
entry.terminate();
}
};
entry.start();
} catch (caught) {
callbackError = asError(caught);
entry.terminate();
}
});
unzip.register(UnzipInflate);
try {
for await (const chunk of iterateByteChunks(archive, {
maximumBytes: ZIP_LIMIT,
knownTotalBytes: archive.size,
signal: options.signal,
onProgress: ({ processedBytes }) => {
archiveBytes = processedBytes;
options.onProgress?.({
phase: "read",
archiveBytes,
archiveTotalBytes: archive.size,
entries: entryCount,
expandedBytes: totalBytes,
});
},
})) {
unzip.push(new Uint8Array(chunk), false);
if (callbackError) throw callbackError;
}
unzip.push(new Uint8Array(), true);
if (callbackError) throw callbackError;
} catch (caught) {
throw asError(caught);
}
if (callbackError) throw callbackError;
const files: ArchiveContentEntry[] = [];
for (const [index, draft] of drafts
.sort((left, right) => compareText(left.path, right.path))
.entries()) {
const digests: Record<string, string> = {};
const content = new Blob(draft.chunks);
for (const algorithm of orderedAlgorithms)
digests[algorithm.toLowerCase()] = await digestByteSourceHex(
content,
algorithm,
{
maximumBytes: ARCHIVE_ENTRY_SIZE_LIMIT,
knownTotalBytes: draft.size,
signal: options.signal,
},
);
files.push({
path: draft.path,
size: draft.size,
compressedSize: draft.compressedSize,
crc32: draft.crc32,
digests,
});
options.onProgress?.({
phase: "hash",
archiveBytes: archive.size,
archiveTotalBytes: archive.size,
entries: index + 1,
expandedBytes: totalBytes,
path: draft.path,
});
}
return {
schema:
"https://git.add-ideas.de/lotobo/repro-tools/schema/archive-content-manifest-v1",
manifestVersion: 1,
generator: { id: "de.add-ideas.repro-tools", version: APP_VERSION },
operation: {
name: "inspect-zip-content",
ordering: "unicode-code-point-path",
algorithms: orderedAlgorithms,
limits: {
entries: ARCHIVE_ENTRY_LIMIT,
entryBytes: ARCHIVE_ENTRY_SIZE_LIMIT,
expandedBytes: ARCHIVE_TOTAL_SIZE_LIMIT,
archiveBytes: ZIP_LIMIT,
},
},
archive: { name: archive.name, size: archive.size },
fileCount: files.length,
totalBytes,
files,
};
}
export function serializeArchiveContentManifest(
manifest: ArchiveContentManifest,
): string {
return `${stableStringify(manifest, 2, {
maxTextChars: MANIFEST_TEXT_LIMIT,
maxDepth: 32,
maxNodes: 100_000,
})}\n`;
}
export function parseArchiveContentManifest(
text: string,
): ArchiveContentManifest {
const value = safeJsonParse(text, {
maxTextChars: MANIFEST_TEXT_LIMIT,
maxDepth: 32,
maxNodes: 100_000,
});
if (!value || Array.isArray(value) || typeof value !== "object")
throw new TypeError("Archive content manifest must be a JSON object.");
const candidate = value as unknown as ArchiveContentManifest;
if (
candidate.schema !==
"https://git.add-ideas.de/lotobo/repro-tools/schema/archive-content-manifest-v1" ||
candidate.manifestVersion !== 1 ||
candidate.operation?.name !== "inspect-zip-content" ||
candidate.operation.ordering !== "unicode-code-point-path" ||
!Array.isArray(candidate.operation.algorithms) ||
!Array.isArray(candidate.files) ||
candidate.files.length > ARCHIVE_ENTRY_LIMIT ||
candidate.fileCount !== candidate.files.length
)
throw new TypeError("This is not a supported archive content manifest.");
const paths = new Set<string>();
let total = 0;
for (const entry of candidate.files) {
if (
normalizePath(entry.path) !== entry.path ||
paths.has(entry.path) ||
!Number.isSafeInteger(entry.size) ||
entry.size < 0 ||
entry.size > ARCHIVE_ENTRY_SIZE_LIMIT ||
!/^[0-9a-f]{8}$/u.test(entry.crc32) ||
!entry.digests ||
typeof entry.digests !== "object"
)
throw new TypeError(
"Archive content manifest contains an invalid entry.",
);
paths.add(entry.path);
total += entry.size;
}
if (total !== candidate.totalBytes || total > ARCHIVE_TOTAL_SIZE_LIMIT)
throw new TypeError("Archive content manifest totals are inconsistent.");
return candidate;
}
export function compareArchiveContentManifests(
reference: ArchiveContentManifest,
actual: ArchiveContentManifest,
): Comparison[] {
const expected = new Map(reference.files.map((entry) => [entry.path, entry]));
const observed = new Map(actual.files.map((entry) => [entry.path, entry]));
const paths = [...new Set([...expected.keys(), ...observed.keys()])].sort(
compareText,
);
return paths.map((path) => {
const left = expected.get(path);
const right = observed.get(path);
if (!left)
return {
status: "unexpected",
path,
detail: "Only in the current archive",
};
if (!right)
return {
status: "missing",
path,
detail: "Missing from the current archive",
};
if (left.size !== right.size)
return {
status: "changed",
path,
detail: `Expanded size ${left.size}${right.size}`,
};
const shared = Object.keys(left.digests).filter(
(algorithm) => algorithm in right.digests,
);
if (shared.length === 0)
return {
status: "changed",
path,
detail: "No common content digest algorithm",
};
const changed = shared.find(
(algorithm) => left.digests[algorithm] !== right.digests[algorithm],
);
return changed
? { status: "changed", path, detail: `${changed} digest differs` }
: { status: "match", path, detail: "Expanded content digest matches" };
});
}
function asError(value: unknown): Error {
return value instanceof Error ? value : new Error(String(value));
}
function abortReason(signal: AbortSignal): Error {
if (signal.reason instanceof Error) return signal.reason;
return typeof DOMException === "undefined"
? Object.assign(new Error("Operation cancelled"), { name: "AbortError" })
: new DOMException("Operation cancelled", "AbortError");
}
export async function deterministicZip( export async function deterministicZip(
selected: readonly SelectedFile[], selected: readonly SelectedFile[],
manifest?: ReproManifest, manifest?: ReproManifest,
+35 -1
View File
@@ -3,7 +3,7 @@
"schemaVersion": 1, "schemaVersion": 1,
"id": "de.add-ideas.repro-tools", "id": "de.add-ideas.repro-tools",
"name": "Repro Tools", "name": "Repro Tools",
"version": "0.1.0", "version": "0.2.0",
"description": "Build and verify reproducible file manifests locally.", "description": "Build and verify reproducible file manifests locally.",
"entry": "./", "entry": "./",
"icon": "./favicon.svg", "icon": "./favicon.svg",
@@ -21,6 +21,40 @@
"crossOriginIsolated": false, "crossOriginIsolated": false,
"topLevelContext": false "topLevelContext": false
}, },
"io": {
"accepts": [
{
"mediaType": "*/*",
"extensions": []
},
{
"mediaType": "application/zip",
"extensions": [".zip"]
},
{
"mediaType": "application/json",
"extensions": [".json"]
}
],
"produces": [
{
"mediaType": "application/json",
"extensions": [".json"]
},
{
"mediaType": "text/csv",
"extensions": [".csv"]
},
{
"mediaType": "application/zip",
"extensions": [".zip"]
}
]
},
"capabilities": {
"required": [],
"optional": ["web-crypto"]
},
"privacy": { "privacy": {
"processing": "local", "processing": "local",
"fileUploads": true, "fileUploads": true,
+1 -1
View File
@@ -1 +1 @@
export const APP_VERSION = "0.1.0"; export const APP_VERSION = "0.2.0";
+36
View File
@@ -1,5 +1,6 @@
import { expect, test } from "@playwright/test"; import { expect, test } from "@playwright/test";
import { Buffer } from "node:buffer"; import { Buffer } from "node:buffer";
import { strToU8, zipSync } from "fflate";
for (const path of ["/", "/deep/nested/repro/"]) { for (const path of ["/", "/deep/nested/repro/"]) {
test(`builds a manifest locally at ${path}`, async ({ page }) => { test(`builds a manifest locally at ${path}`, async ({ page }) => {
@@ -57,3 +58,38 @@ test("keeps the installed application available offline", async ({
await context.setOffline(false); await context.setOffline(false);
} }
}); });
test("compares expanded ZIP evidence and recognizes CycloneDX locally", async ({
page,
}) => {
await page.goto("/");
const zip = zipSync({ "inside.txt": strToU8("same content") });
await page.locator('input[accept="application/zip,.zip"]').setInputFiles({
name: "sample.zip",
mimeType: "application/zip",
buffer: Buffer.from(zip),
});
await expect(page.getByLabel("Archive content manifest")).toHaveValue(
/inside\.txt/u,
);
await page
.locator('input[type="file"]')
.first()
.setInputFiles({
name: "bom.cdx.json",
mimeType: "application/json",
buffer: Buffer.from(
JSON.stringify({
bomFormat: "CycloneDX",
specVersion: "1.6",
components: [{ name: "demo" }],
}),
),
});
await page.getByRole("button", { name: "Build manifest" }).click();
await page.getByRole("button", { name: "Build provenance evidence" }).click();
await expect(
page.getByRole("cell", { name: "CycloneDX JSON" }),
).toBeVisible();
});
+18
View File
@@ -0,0 +1,18 @@
import { expect, test } from "@playwright/test";
test("keeps the primary workspace inside a narrow viewport", async ({
page,
}) => {
await page.goto("/deep/nested/repro/");
await expect(page.locator("main").first()).toBeVisible();
await expect(
page.locator("main .loading, main .workbench-loading"),
).toHaveCount(0);
const widths = await page.evaluate(() => ({
content: document.documentElement.scrollWidth,
viewport: document.documentElement.clientWidth,
}));
expect(widths.viewport).toBeLessThanOrEqual(430);
expect(widths.content).toBeLessThanOrEqual(widths.viewport + 1);
});
+81
View File
@@ -2,13 +2,22 @@ import { unzipSync } from "fflate";
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { import {
buildManifest, buildManifest,
buildArchiveContentManifest,
collectSelection, collectSelection,
compareArchiveContentManifests,
compareManifests, compareManifests,
deterministicZip, deterministicZip,
normalizePath, normalizePath,
parseManifest, parseManifest,
serializeManifest, serializeManifest,
serializeArchiveContentManifest,
parseArchiveContentManifest,
} from "../../src/core/repro"; } from "../../src/core/repro";
import {
createProvenanceStatement,
createReproEvidence,
inspectSbomDocuments,
} from "../../src/core/evidence";
function files() { function files() {
const first = new File(["alpha"], "a.txt"); const first = new File(["alpha"], "a.txt");
@@ -19,6 +28,12 @@ function files() {
]; ];
} }
function ownedBuffer(input: Uint8Array): ArrayBuffer {
const owned = new Uint8Array(input.byteLength);
owned.set(input);
return owned.buffer;
}
describe("reproducible manifests", () => { describe("reproducible manifests", () => {
it("hashes in deterministic path order", async () => { it("hashes in deterministic path order", async () => {
const manifest = await buildManifest(files(), ["SHA-512", "SHA-256"]); const manifest = await buildManifest(files(), ["SHA-512", "SHA-256"]);
@@ -115,4 +130,70 @@ describe("reproducible manifests", () => {
).toThrow(/invalid digests/u); ).toThrow(/invalid digests/u);
expect(() => normalizePath("a".repeat(4_097))).toThrow(/4096/u); expect(() => normalizePath("a".repeat(4_097))).toThrow(/4096/u);
}); });
it("streams bounded ZIP contents into comparable content manifests", async () => {
const firstBytes = await deterministicZip([
{ path: "nested/a.txt", file: new File(["alpha"], "a.txt") },
{ path: "b.txt", file: new File(["beta"], "b.txt") },
]);
const secondBytes = await deterministicZip([
{ path: "nested/a.txt", file: new File(["changed"], "a.txt") },
{ path: "c.txt", file: new File(["new"], "c.txt") },
]);
const first = await buildArchiveContentManifest(
new File([ownedBuffer(firstBytes)], "first.zip"),
["SHA-256"],
);
const second = await buildArchiveContentManifest(
new File([ownedBuffer(secondBytes)], "second.zip"),
["SHA-256"],
);
expect(first.files.map((entry) => entry.path)).toEqual([
"b.txt",
"nested/a.txt",
]);
expect(first.files[0]?.crc32).toMatch(/^[0-9a-f]{8}$/u);
expect(
parseArchiveContentManifest(serializeArchiveContentManifest(first)),
).toEqual(first);
expect(
compareArchiveContentManifests(first, second).map((item) => item.status),
).toEqual(["missing", "unexpected", "changed"]);
});
it("recognizes SBOM evidence and makes a non-conformance provenance statement", async () => {
const selected = [
{
path: "bom.cdx.json",
file: new File(
[
JSON.stringify({
bomFormat: "CycloneDX",
specVersion: "1.6",
serialNumber: "urn:uuid:test",
metadata: { component: { name: "demo" } },
components: [{ name: "library" }],
dependencies: [{ ref: "demo" }],
}),
],
"bom.cdx.json",
),
},
];
const manifest = await buildManifest(selected, ["SHA-256"]);
const materials = await inspectSbomDocuments(selected);
expect(materials[0]).toMatchObject({
status: "recognized",
format: "CycloneDX JSON",
specificationVersion: "1.6",
componentCount: 1,
});
const provenance = createProvenanceStatement(manifest, materials);
expect(provenance.predicate.metadata.conformanceClaim).toBeNull();
expect(provenance.predicate.materials).toEqual(materials);
expect(createReproEvidence(manifest, materials)).toMatchObject({
contractVersion: 1,
provenance: { execution: "local-browser", networkRequired: false },
});
});
}); });