lotobo/av-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity
Files
fcc537b02403bb77247f69f1b2fbe31cc5f65951
av-tools/README.md

15 KiB
Raw Blame History

Audio & Video Tools

av-tools is a standalone, local-first browser application for converting and lightly editing audio/video. Media is processed by self-hosted FFmpeg WebAssembly in a browser worker: there is no upload API, server transcoder, telemetry, runtime CDN, remote-media fetch or third-party script.

It is also a Toolbox SDK application. The same static build runs directly or under Toolbox Portal with shared shell, theme, help and app switching.

Release status: 0.1.0 is licensed GPL-3.0-or-later. Its canonical release consists of the exact reviewed v0.1.0 source, production ZIP/checksum and Corresponding Source bundle published together. See Licensing.

Product scope

Two workflows keep the product intentionally smaller than a nonlinear editor:

  • Quick Convert: drop/import media, inspect streams, choose a capability-checked preset/streams, convert, preview when the browser can, and download.
  • Edit: a media bin and sequential non-overlapping clip timeline with non-destructive settings, one export at a time.

The planned 0.1 line covers typed/capability-gated conversion and remux, fast/accurate trim, split, compatible/normalized concatenate, reorder, crop, resize, frame-rate change, audio extraction/stream removal, waveform, peak/two-pass loudness normalization, gain, audio/video fades, metadata, chapters, SRT/WebVTT/ASS text subtitles, thumbnails, contact sheets, presets, versioned project save/load, reattachment, progress, cancellation/recovery, diagnostics and explicit browser export.

The Edit workspace now exposes structural exports directly: fast stream-copy or preset-backed accurate trim uses the selected clip's timeline range, while fast concatenate requires explicit compatibility diagnostics and normalized concatenate requires a preset plus an explicit missing-audio policy. Operations enter a bounded, in-memory application queue with at most eight scheduled operations and one FFmpeg execution at a time.

When a source cannot be played directly, the preview panel can create a disposable H.264/AAC MP4 proxy. The user can bound it to 1–300 seconds and choose a maximum width from 480–1920 pixels. A proxy is reused from the local derivative cache only when the source identity and settings match.

A control is not considered complete merely because its UI or command builder exists. The exact milestone status is the tested matrix in the final release report; CHANGELOG.md lists current blockers.

Engine and format model

Exact runtime packages:

  • @ffmpeg/ffmpeg 0.12.15;
  • @ffmpeg/util 0.12.2;
  • @ffmpeg/core 0.12.10 (single-thread);
  • @ffmpeg/core-mt 0.12.10 (multithread).

The published cores contain FFmpeg n5.1.4 (libavcodec 59.37.100), built with x264, x265, libvpx, LAME, Ogg/Theora/Opus/Vorbis, zlib, libwebp, FreeType/FriBidi/HarfBuzz/libass and zimg. Both ~32 MiB WASM cores and their JS/worker files are copied from exact npm packages and served from the app's own origin. npm run build does not download them.

Every engine load queries actual demuxers, muxers, decoders, encoders and filters. Presets remain disabled when requirements are absent; an extension or FFmpeg build flag is not a compatibility promise.

To bound native heap retained by heterogeneous long-running sessions, the app refreshes an idle core before its seventeenth media execution and before a fourth contact sheet uses the pinned core's retained tile-filter state. Queue state, sources and completed results stay in application memory; only the worker is recreated. A bare core abort also triggers recovery for a retry.

Generated fixtures cover MP4/H.264/AAC, WebM/VP8/Vorbis, WAV/PCM, MP3, FLAC, metadata, chapters, attached cover art, SRT, WebVTT, ASS and malformed input. All 15 fixtures were generated/re-probed with native FFmpeg 7.1.3 for test preparation and pass the repository checksum/probe verifier. Chromium automation uses the pinned ST core for a broad operation matrix and separately proves a real MT H.264/AAC conversion. That does not establish cross-browser support or exhaustive MT parity, and fixture generation is not evidence that “all formats” work. See format support and fixture provenance.

The recorded pinned-core capability inventory contains 341 demuxers, 173 muxers, 515 decoders, 204 encoders and 465 filters (1,698 entries total). These are discovered capabilities, not product or browser-playback guarantees.

Browser playback is independent of FFmpeg readability. MP4/H.264/AAC, WebM/VP8/Vorbis, MP3 and WAV are common preview candidates, but actual codec, browser and platform support is checked at runtime. Matroska and many other valid outputs may be download-only. A preview proxy may require re-encoding.

The pinned core has an unresolved Opus encoding regression risk (zero-byte or out-of-bounds behavior reported around upstream stack-size fixes). Opus presets must stay unavailable unless a real non-empty, re-probeable ST/MT smoke passes. Details are in FFmpeg build.

Execution modes

  • Multithread is selected only in secure, cross-origin-isolated contexts with SharedArrayBuffer.
  • Single-thread compatibility mode is the automatic fallback when isolation is unavailable or MT loading fails. It remains fully worker-based but may be slower.

MT requires every response in the page to support:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Resource-Policy: same-origin

Serve .wasm as application/wasm, keep all core files same-origin and allow 'wasm-unsafe-eval' plus workers in CSP. Blob media preview additionally needs media-src 'self' blob:. The manifest intentionally does not require cross-origin isolation because ST is supported.

Limits, storage and privacy

ffmpeg.wasm documents a hard 2 GiB input ceiling. Real browser limits are often lower because sources, decoded frames, filters and outputs coexist. The initial application policy warns at 256 MiB input, blocks a single or total input above 1.5 GiB, warns at a 512 MiB output estimate and blocks an estimated output above 1 GiB. The in-memory queue accepts at most eight scheduled operations and runs only one FFmpeg operation at a time. Other defaults bound clips, streams, dimensions/pixels, thumbnails, contact-sheet cells and runtime. Refreshing or closing the page discards queued work.

Estimates are not guarantees. Lower resolution, duration, frame rate, pass count and output count can reduce failure risk. WebAssembly does not offer desktop FFmpeg performance or hardware encoding.

Source File objects and preview/result Blob URLs are temporary and object URLs are revoked on replacement/cleanup. Per-job FFmpeg files/mounts are deleted; cancellation discards the engine filesystem. The mounted application currently uses IndexedDB for validated user presets; project JSON is explicitly saved/loaded by the user and the operation queue remains in memory. Optional OPFS caching is limited to regenerable derivatives and is clearable; source bytes are not persisted by default. OPFS belongs to the current origin, so changing scheme/host/port does not move cached data. See storage.

Imported media, names, tags and subtitles are untrusted. They are rendered as text, paths are generated, operations build argument arrays rather than shell commands, logs are bounded/redacted, and extracted URLs are inert. See security.

Development

Requirements: Node.js 22+, npm matching the lockfile, and a modern browser. Native FFmpeg is a development/test dependency for fixture regeneration and the conditional native command-plan integration suite; it is not a production runtime dependency.

git clone git@git.add-ideas.de:zemion/av-tools.git
cd av-tools
npm ci
npm run dev

The token-free .npmrc selects the published Toolbox SDK scope. Do not add credentials to repository configuration.

Core assets are generated local build inputs:

npm run vendor:ffmpeg
npm run vendor:verify

Regenerate or verify the tiny synthetic fixtures:

npm run fixtures:generate
npm run fixtures:verify

Checks

npm run manifest:check
npm run vendor:verify
npm run typecheck
npm run lint
npm run format:check
npm test
npm run build
npm run toolbox:check
npm run test:browser

npm run check combines the non-browser production checks. Browser tests cover the real pinned cores, MT-to-ST fallback, cancellation/recovery, generated fixtures, nested deployment, Toolbox context and the absence of runtime CDN traffic.

The latest recorded local Vitest run on 2026-07-24 completed 303 passing tests and one conditional skip. It includes broad structured-command execution against native FFmpeg 7.1.3; the skipped native subtitle burn-in case requires a subtitles filter unavailable in that host build.

The Chromium suite exercises the pinned ST core across conversion, trim/concat/split, analysis, waveform, image, metadata, chapter, subtitle, queue and recovery workflows. The MT core is directly initialized and queried, and completes a real non-empty, re-probeable H.264/AAC conversion; the same path is benchmarked with 2-second 160×90 and 6-second 640×360 inputs. Nested static/Toolbox-context loading is also covered. This evidence is Chromium-specific: Firefox, Safari, mobile and exhaustive MT operation parity remain unrecorded. The exact one-run performance record and its limitations are in performance observations.

Deployment

Deploy the contents of dist/ beneath any static path; URLs are base-relative and no server route fallback is required. Production examples:

The policy permits same-origin WebAssembly and Blob workers/media without broad 'unsafe-eval'. Keep stable entry/manifest/notices revalidated. Exact versioned core files and Vite content-hashed assets may be cached immutable. When isolation headers cannot be configured, ST remains the supported fallback.

Toolbox integration

The one typed de.add-ideas.av-tools manifest generates public/toolbox-app.json. AppShell receives the relative ./toolbox-app.json URL and falls back to standalone behavior when Toolbox context is absent or invalid. av-tools does not duplicate suite personalization or app switching.

The manifest uses entry: "./", relative icon/assets, top-level navigation, local processing/no upload/no telemetry, and advertises workers, secure context and IndexedDB. Its source object identifies the canonical av-tools repository and the GPL-3.0-or-later SPDX licence.

Portal must consume an independently built ZIP; it must not build or import av-tools source. Immutable Portal revision bda9da044d61c109afd9969d357cc27f76648938 has isolation and WASM CSP permissions but lacks media-src blob:, which blocks Blob previews. The developer's active Toolbox Portal worktree currently contains the narrow local media-src 'self' blob: patch; that uncommitted worktree state does not change the immutable reference or release evidence. The exact distinction, smallest change and fallback are documented in Portal requirements.

Release

After all checks, normal packaging would produce deterministic:

release/av-tools-<version>.zip
release/av-tools-<version>.zip.sha256

The ZIP root contains index.html, toolbox-app.json, CHANGELOG.md, LICENSES/, SOURCE.md, THIRD_PARTY_NOTICES.md, assets/ and all vendor/ core files. It excludes node_modules, sources, source maps and fixtures. Paths are sorted, timestamps/modes normalized and the SHA-256 sidecar is verified.

Normal packaging requires the root GPL licence and matching package/manifest metadata:

npm run package:release
npm run portal:smoke -- \
  --allow-known-header-gap

The header-gap override remains development-only until the Portal CSP change is committed. Full commands and safety behavior are in release documentation.

Licensing

av-tools-owned code and documentation are Copyright © 2026 Albrecht Degering and licensed GPL-3.0-or-later. See LICENSE. The application provides a visible licence, source and no-warranty notice in its Help dialog. Third-party components retain their copyright and licence terms.

The wrapper/util are MIT. Core/core-mt 0.12.10 declare GPL-2.0-or-later and statically include FFmpeg/external codec code with additional terms. The branch-based x264/LAME inputs have been resolved to immutable commits in the source record. A binary publication must include equivalent access to the reviewed Corresponding Source and notices. See:

The complete distributed application must not be called MIT-only. Publishing a binary or hosted JS/WASM build requires equivalent access to its corresponding source; linking only to a component homepage is not asserted to satisfy that obligation. Codec patent requirements are separate from copyright licensing.

Known limitations

The auditable list is in known limitations. In summary:

  • No server transcoder, remote URL input, live capture, DRM or encrypted media.
  • No overlapping video tracks, arbitrary compositing, collaboration or professional nonlinear editing.
  • No “all formats,” unlimited file size, desktop performance or hardware encoding guarantee.
  • Stream-copy trim is fast but not promised frame-accurate.
  • Re-encoding is not lossless; lossless codec output cannot restore discarded source information.
  • Metadata removal is best effort, not an anonymity guarantee.
  • Browser preview supports fewer combinations than FFmpeg.
  • MT depends on isolation; ST can still hit memory/time limits.
  • Opus output is release-gated for the pinned core.
  • Projects do not contain source bytes by default and require reattachment.
  • Cross-browser/mobile and exhaustive MT performance matrices remain unrecorded; published support claims must stay within the tested evidence.

Roadmap

  1. Complete Quick Convert and structural-export browser verification with the real pinned ST/MT cores, including fallback and queued execution.
  2. Verify waveform, loudness normalization and combined filters in browsers.
  3. Verify metadata/chapter/subtitle round trips in browsers.
  4. Verify bounded thumbnails/contact sheets, projects, derivative-cache reuse and export in browsers.
  5. Complete accessibility, mobile, performance and nested Portal matrices.
  6. Commit/tag exact source, publish its reviewed Corresponding Source, assemble the exact production ZIP through Portal and verify deployed headers.