av-tools/README.md

# Audio & Video Tools

av-tools is a standalone, local-first browser application for converting and
lightly editing audio/video. Media is processed by self-hosted FFmpeg
WebAssembly in a browser worker: there is no upload API, server transcoder,
telemetry, runtime CDN, remote-media fetch or third-party script.

It is also a Toolbox SDK application. The same static build runs directly or
under Toolbox Portal with shared shell, theme, help and app switching.

> **Release status:** 0.1.0 is licensed GPL-3.0-or-later. Its canonical
> release consists of the exact reviewed `v0.1.0` source, production
> ZIP/checksum and Corresponding Source bundle published together. See
> [Licensing](#licensing).

## Product scope

Two workflows keep the product intentionally smaller than a nonlinear editor:

- **Quick Convert:** drop/import media, inspect streams, choose a
  capability-checked preset/streams, convert, preview when the browser can, and
  download.
- **Edit:** a media bin and sequential non-overlapping clip timeline with
  non-destructive settings, one export at a time.

The planned 0.1 line covers typed/capability-gated conversion and remux,
fast/accurate trim, split, compatible/normalized concatenate, reorder, crop,
resize, frame-rate change, audio extraction/stream removal, waveform,
peak/two-pass loudness normalization, gain, audio/video fades, metadata,
chapters, SRT/WebVTT/ASS text subtitles, thumbnails, contact sheets, presets,
versioned project save/load, reattachment, progress, cancellation/recovery,
diagnostics and explicit browser export.

The Edit workspace now exposes structural exports directly: fast stream-copy
or preset-backed accurate trim uses the selected clip's timeline range, while
fast concatenate requires explicit compatibility diagnostics and normalized
concatenate requires a preset plus an explicit missing-audio policy.
Operations enter a bounded, in-memory application queue with at most eight
scheduled operations and one FFmpeg execution at a time.

When a source cannot be played directly, the preview panel can create a
disposable H.264/AAC MP4 proxy. The user can bound it to 1–300 seconds and
choose a maximum width from 480–1920 pixels. A proxy is reused from the local
derivative cache only when the source identity and settings match.

A control is not considered complete merely because its UI or command builder
exists. The exact milestone status is the tested matrix in the final release
report; [CHANGELOG.md](CHANGELOG.md) lists current blockers.

## Engine and format model

Exact runtime packages:

- `@ffmpeg/ffmpeg` 0.12.15;
- `@ffmpeg/util` 0.12.2;
- `@ffmpeg/core` 0.12.10 (single-thread);
- `@ffmpeg/core-mt` 0.12.10 (multithread).

The published cores contain **FFmpeg n5.1.4** (libavcodec 59.37.100), built
with x264, x265, libvpx, LAME, Ogg/Theora/Opus/Vorbis, zlib, libwebp,
FreeType/FriBidi/HarfBuzz/libass and zimg. Both ~32 MiB WASM cores and their
JS/worker files are copied from exact npm packages and served from the app's
own origin. `npm run build` does not download them.

Every engine load queries actual demuxers, muxers, decoders, encoders and
filters. Presets remain disabled when requirements are absent; an extension or
FFmpeg build flag is not a compatibility promise.

To bound native heap retained by heterogeneous long-running sessions, the app
refreshes an idle core before its seventeenth media execution and before a
fourth contact sheet uses the pinned core's retained `tile`-filter state.
Queue state, sources and completed results stay in application memory; only
the worker is recreated. A bare core abort also triggers recovery for a retry.

Generated fixtures cover MP4/H.264/AAC, WebM/VP8/Vorbis, WAV/PCM, MP3, FLAC,
metadata, chapters, attached cover art, SRT, WebVTT, ASS and malformed input.
All 15 fixtures were generated/re-probed with native FFmpeg 7.1.3 for test
preparation and pass the repository checksum/probe verifier. Chromium
automation uses the pinned ST core for a broad operation matrix and separately
proves a real MT H.264/AAC conversion. That does not establish cross-browser
support or exhaustive MT parity, and fixture generation is not evidence that
“all formats” work. See
[format support](docs/FORMAT_SUPPORT.md) and
[fixture provenance](tests/fixtures/README.md).

The recorded pinned-core capability inventory contains 341 demuxers, 173
muxers, 515 decoders, 204 encoders and 465 filters (1,698 entries total).
These are discovered capabilities, not product or browser-playback guarantees.

Browser playback is independent of FFmpeg readability. MP4/H.264/AAC,
WebM/VP8/Vorbis, MP3 and WAV are common preview candidates, but actual codec,
browser and platform support is checked at runtime. Matroska and many other
valid outputs may be download-only. A preview proxy may require re-encoding.

The pinned core has an unresolved Opus encoding regression risk (zero-byte or
out-of-bounds behavior reported around upstream stack-size fixes). Opus
presets must stay unavailable unless a real non-empty, re-probeable ST/MT smoke
passes. Details are in [FFmpeg build](docs/FFMPEG_BUILD.md).

## Execution modes

- **Multithread** is selected only in secure, cross-origin-isolated contexts
  with `SharedArrayBuffer`.
- **Single-thread compatibility mode** is the automatic fallback when
  isolation is unavailable or MT loading fails. It remains fully worker-based
  but may be slower.

MT requires every response in the page to support:

```text
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Resource-Policy: same-origin
```

Serve `.wasm` as `application/wasm`, keep all core files same-origin and allow
`'wasm-unsafe-eval'` plus workers in CSP. Blob media preview additionally needs
`media-src 'self' blob:`. The manifest intentionally does **not** require
cross-origin isolation because ST is supported.

## Limits, storage and privacy

ffmpeg.wasm documents a hard **2 GiB input ceiling**. Real browser limits are
often lower because sources, decoded frames, filters and outputs coexist. The
initial application policy warns at 256 MiB input, blocks a single or total
input above 1.5 GiB, warns at a 512 MiB output estimate and blocks an estimated
output above 1 GiB. The in-memory queue accepts at most eight scheduled
operations and runs only one FFmpeg operation at a time. Other defaults bound
clips, streams, dimensions/pixels, thumbnails, contact-sheet cells and runtime.
Refreshing or closing the page discards queued work.

Estimates are not guarantees. Lower resolution, duration, frame rate, pass
count and output count can reduce failure risk. WebAssembly does not offer
desktop FFmpeg performance or hardware encoding.

Source `File` objects and preview/result Blob URLs are temporary and object URLs
are revoked on replacement/cleanup. Per-job FFmpeg files/mounts are deleted;
cancellation discards the engine filesystem. The mounted application currently
uses IndexedDB for validated user presets; project JSON is explicitly
saved/loaded by the user and the operation queue remains in memory. Optional
OPFS caching is limited to regenerable derivatives and is clearable; source
bytes are **not persisted by default**. OPFS belongs to the current origin, so
changing scheme/host/port does not move cached data. See
[storage](docs/STORAGE.md).

Imported media, names, tags and subtitles are untrusted. They are rendered as
text, paths are generated, operations build argument arrays rather than shell
commands, logs are bounded/redacted, and extracted URLs are inert. See
[security](docs/SECURITY.md).

## Development

Requirements: Node.js 22+, npm matching the lockfile, and a modern browser.
Native FFmpeg is a development/test dependency for fixture regeneration and
the conditional native command-plan integration suite; it is not a production
runtime dependency.

```sh
git clone git@git.add-ideas.de:zemion/av-tools.git
cd av-tools
npm ci
npm run dev
```

The token-free `.npmrc` selects the published Toolbox SDK scope. Do not add
credentials to repository configuration.

Core assets are generated local build inputs:

```sh
npm run vendor:ffmpeg
npm run vendor:verify
```

Regenerate or verify the tiny synthetic fixtures:

```sh
npm run fixtures:generate
npm run fixtures:verify
```

## Checks

```sh
npm run manifest:check
npm run vendor:verify
npm run typecheck
npm run lint
npm run format:check
npm test
npm run build
npm run toolbox:check
npm run test:browser
```

`npm run check` combines the non-browser production checks. Browser tests cover
the real pinned cores, MT-to-ST fallback, cancellation/recovery, generated
fixtures, nested deployment, Toolbox context and the absence of runtime CDN
traffic.

The latest recorded local Vitest run on 2026-07-24 completed **303 passing
tests and one conditional skip**. It includes broad structured-command
execution against native FFmpeg 7.1.3; the skipped native subtitle burn-in case
requires a `subtitles` filter unavailable in that host build.

The Chromium suite exercises the pinned ST core across conversion,
trim/concat/split, analysis, waveform, image, metadata, chapter, subtitle,
queue and recovery workflows. The MT core is directly initialized and queried,
and completes a real non-empty, re-probeable H.264/AAC conversion; the same
path is benchmarked with 2-second 160×90 and 6-second 640×360 inputs. Nested
static/Toolbox-context loading is also covered. This evidence is
Chromium-specific: Firefox, Safari, mobile and exhaustive MT operation parity
remain unrecorded. The exact one-run performance record and its limitations are
in [performance observations](docs/PERFORMANCE.md).

## Deployment

Deploy the contents of `dist/` beneath any static path; URLs are base-relative
and no server route fallback is required. Production examples:

- [Nginx](deployment/nginx.example.conf)
- [Caddy](deployment/caddy.example)
- [generic static headers](deployment/headers.md)

The policy permits same-origin WebAssembly and Blob workers/media without
broad `'unsafe-eval'`. Keep stable entry/manifest/notices revalidated. Exact
versioned core files and Vite content-hashed assets may be cached immutable.
When isolation headers cannot be configured, ST remains the supported
fallback.

## Toolbox integration

The one typed `de.add-ideas.av-tools` manifest generates
`public/toolbox-app.json`. `AppShell` receives the relative
`./toolbox-app.json` URL and falls back to standalone behavior when Toolbox
context is absent or invalid. av-tools does not duplicate suite
personalization or app switching.

The manifest uses `entry: "./"`, relative icon/assets, top-level navigation,
local processing/no upload/no telemetry, and advertises workers, secure
context and IndexedDB. Its `source` object identifies the canonical av-tools
repository and the `GPL-3.0-or-later` SPDX licence.

Portal must consume an independently built ZIP; it must not build or import
av-tools source. Immutable Portal revision
`bda9da044d61c109afd9969d357cc27f76648938` has isolation and WASM CSP
permissions but lacks `media-src blob:`, which blocks Blob previews. The
developer's active Toolbox Portal worktree currently contains the narrow local
`media-src 'self' blob:` patch; that uncommitted worktree state does not change
the immutable reference or release evidence. The exact distinction, smallest
change and fallback are documented in
[Portal requirements](docs/PORTAL_REQUIREMENTS.md).

## Release

After all checks, normal packaging would produce deterministic:

```text
release/av-tools-<version>.zip
release/av-tools-<version>.zip.sha256
```

The ZIP root contains `index.html`, `toolbox-app.json`, `CHANGELOG.md`,
`LICENSES/`, `SOURCE.md`, `THIRD_PARTY_NOTICES.md`, `assets/` and all
`vendor/` core files. It excludes `node_modules`, sources, source maps and
fixtures. Paths are sorted, timestamps/modes normalized and the SHA-256
sidecar is verified.

Normal packaging requires the root GPL licence and matching package/manifest
metadata:

```sh
npm run package:release
npm run portal:smoke -- \
  --allow-known-header-gap
```

The header-gap override remains development-only until the Portal CSP change is
committed. Full commands and safety behavior are in
[release documentation](docs/RELEASE.md).

## Licensing

av-tools-owned code and documentation are Copyright © 2026 Albrecht Degering
and licensed **GPL-3.0-or-later**. See [LICENSE](LICENSE). The application
provides a visible licence, source and no-warranty notice in its Help dialog.
Third-party components retain their copyright and licence terms.

The wrapper/util are MIT. Core/core-mt 0.12.10 declare
GPL-2.0-or-later and statically include FFmpeg/external codec code with
additional terms. The branch-based x264/LAME inputs have been resolved to
immutable commits in the source record. A binary publication must include
equivalent access to the reviewed Corresponding Source and notices. See:

- [third-party notices](THIRD_PARTY_NOTICES.md)
- [source record](SOURCE.md)
- [licence texts/status](LICENSES/README.md)

The complete distributed application must not be called MIT-only. Publishing a
binary or hosted JS/WASM build requires equivalent access to its corresponding
source; linking only to a component homepage is not asserted to satisfy that
obligation. Codec patent requirements are separate from copyright licensing.

## Known limitations

The auditable list is in
[known limitations](docs/KNOWN_LIMITATIONS.md). In summary:

- No server transcoder, remote URL input, live capture, DRM or encrypted media.
- No overlapping video tracks, arbitrary compositing, collaboration or
  professional nonlinear editing.
- No “all formats,” unlimited file size, desktop performance or hardware
  encoding guarantee.
- Stream-copy trim is fast but not promised frame-accurate.
- Re-encoding is not lossless; lossless codec output cannot restore discarded
  source information.
- Metadata removal is best effort, not an anonymity guarantee.
- Browser preview supports fewer combinations than FFmpeg.
- MT depends on isolation; ST can still hit memory/time limits.
- Opus output is release-gated for the pinned core.
- Projects do not contain source bytes by default and require reattachment.
- Cross-browser/mobile and exhaustive MT performance matrices remain
  unrecorded; published support claims must stay within the tested evidence.

## Roadmap

1. Complete Quick Convert and structural-export browser verification with the
   real pinned ST/MT cores, including fallback and queued execution.
2. Verify waveform, loudness normalization and combined filters in browsers.
3. Verify metadata/chapter/subtitle round trips in browsers.
4. Verify bounded thumbnails/contact sheets, projects, derivative-cache reuse
   and export in browsers.
5. Complete accessibility, mobile, performance and nested Portal matrices.
6. Commit/tag exact source, publish its reviewed Corresponding Source, assemble
   the exact production ZIP through Portal and verify deployed headers.