feat: publish av-tools 0.1.0
This commit is contained in:
335
README.md
Normal file
335
README.md
Normal file
@@ -0,0 +1,335 @@
|
||||
# Audio & Video Tools
|
||||
|
||||
av-tools is a standalone, local-first browser application for converting and
|
||||
lightly editing audio/video. Media is processed by self-hosted FFmpeg
|
||||
WebAssembly in a browser worker: there is no upload API, server transcoder,
|
||||
telemetry, runtime CDN, remote-media fetch or third-party script.
|
||||
|
||||
It is also a Toolbox SDK application. The same static build runs directly or
|
||||
under Toolbox Portal with shared shell, theme, help and app switching.
|
||||
|
||||
> **Release status:** 0.1.0 is licensed GPL-3.0-or-later. Its canonical
|
||||
> release consists of the exact reviewed `v0.1.0` source, production
|
||||
> ZIP/checksum and Corresponding Source bundle published together. See
|
||||
> [Licensing](#licensing).
|
||||
|
||||
## Product scope
|
||||
|
||||
Two workflows keep the product intentionally smaller than a nonlinear editor:
|
||||
|
||||
- **Quick Convert:** drop/import media, inspect streams, choose a
|
||||
capability-checked preset/streams, convert, preview when the browser can, and
|
||||
download.
|
||||
- **Edit:** a media bin and sequential non-overlapping clip timeline with
|
||||
non-destructive settings, one export at a time.
|
||||
|
||||
The planned 0.1 line covers typed/capability-gated conversion and remux,
|
||||
fast/accurate trim, split, compatible/normalized concatenate, reorder, crop,
|
||||
resize, frame-rate change, audio extraction/stream removal, waveform,
|
||||
peak/two-pass loudness normalization, gain, audio/video fades, metadata,
|
||||
chapters, SRT/WebVTT/ASS text subtitles, thumbnails, contact sheets, presets,
|
||||
versioned project save/load, reattachment, progress, cancellation/recovery,
|
||||
diagnostics and explicit browser export.
|
||||
|
||||
The Edit workspace now exposes structural exports directly: fast stream-copy
|
||||
or preset-backed accurate trim uses the selected clip's timeline range, while
|
||||
fast concatenate requires explicit compatibility diagnostics and normalized
|
||||
concatenate requires a preset plus an explicit missing-audio policy.
|
||||
Operations enter a bounded, in-memory application queue with at most eight
|
||||
scheduled operations and one FFmpeg execution at a time.
|
||||
|
||||
When a source cannot be played directly, the preview panel can create a
|
||||
disposable H.264/AAC MP4 proxy. The user can bound it to 1–300 seconds and
|
||||
choose a maximum width from 480–1920 pixels. A proxy is reused from the local
|
||||
derivative cache only when the source identity and settings match.
|
||||
|
||||
A control is not considered complete merely because its UI or command builder
|
||||
exists. The exact milestone status is the tested matrix in the final release
|
||||
report; [CHANGELOG.md](CHANGELOG.md) lists current blockers.
|
||||
|
||||
## Engine and format model
|
||||
|
||||
Exact runtime packages:
|
||||
|
||||
- `@ffmpeg/ffmpeg` 0.12.15;
|
||||
- `@ffmpeg/util` 0.12.2;
|
||||
- `@ffmpeg/core` 0.12.10 (single-thread);
|
||||
- `@ffmpeg/core-mt` 0.12.10 (multithread).
|
||||
|
||||
The published cores contain **FFmpeg n5.1.4** (libavcodec 59.37.100), built
|
||||
with x264, x265, libvpx, LAME, Ogg/Theora/Opus/Vorbis, zlib, libwebp,
|
||||
FreeType/FriBidi/HarfBuzz/libass and zimg. Both ~32 MiB WASM cores and their
|
||||
JS/worker files are copied from exact npm packages and served from the app's
|
||||
own origin. `npm run build` does not download them.
|
||||
|
||||
Every engine load queries actual demuxers, muxers, decoders, encoders and
|
||||
filters. Presets remain disabled when requirements are absent; an extension or
|
||||
FFmpeg build flag is not a compatibility promise.
|
||||
|
||||
To bound native heap retained by heterogeneous long-running sessions, the app
|
||||
refreshes an idle core before its seventeenth media execution and before a
|
||||
fourth contact sheet uses the pinned core's retained `tile`-filter state.
|
||||
Queue state, sources and completed results stay in application memory; only
|
||||
the worker is recreated. A bare core abort also triggers recovery for a retry.
|
||||
|
||||
Generated fixtures cover MP4/H.264/AAC, WebM/VP8/Vorbis, WAV/PCM, MP3, FLAC,
|
||||
metadata, chapters, attached cover art, SRT, WebVTT, ASS and malformed input.
|
||||
All 15 fixtures were generated/re-probed with native FFmpeg 7.1.3 for test
|
||||
preparation and pass the repository checksum/probe verifier. Chromium
|
||||
automation uses the pinned ST core for a broad operation matrix and separately
|
||||
proves a real MT H.264/AAC conversion. That does not establish cross-browser
|
||||
support or exhaustive MT parity, and fixture generation is not evidence that
|
||||
“all formats” work. See
|
||||
[format support](docs/FORMAT_SUPPORT.md) and
|
||||
[fixture provenance](tests/fixtures/README.md).
|
||||
|
||||
The recorded pinned-core capability inventory contains 341 demuxers, 173
|
||||
muxers, 515 decoders, 204 encoders and 465 filters (1,698 entries total).
|
||||
These are discovered capabilities, not product or browser-playback guarantees.
|
||||
|
||||
Browser playback is independent of FFmpeg readability. MP4/H.264/AAC,
|
||||
WebM/VP8/Vorbis, MP3 and WAV are common preview candidates, but actual codec,
|
||||
browser and platform support is checked at runtime. Matroska and many other
|
||||
valid outputs may be download-only. A preview proxy may require re-encoding.
|
||||
|
||||
The pinned core has an unresolved Opus encoding regression risk (zero-byte or
|
||||
out-of-bounds behavior reported around upstream stack-size fixes). Opus
|
||||
presets must stay unavailable unless a real non-empty, re-probeable ST/MT smoke
|
||||
passes. Details are in [FFmpeg build](docs/FFMPEG_BUILD.md).
|
||||
|
||||
## Execution modes
|
||||
|
||||
- **Multithread** is selected only in secure, cross-origin-isolated contexts
|
||||
with `SharedArrayBuffer`.
|
||||
- **Single-thread compatibility mode** is the automatic fallback when
|
||||
isolation is unavailable or MT loading fails. It remains fully worker-based
|
||||
but may be slower.
|
||||
|
||||
MT requires every response in the page to support:
|
||||
|
||||
```text
|
||||
Cross-Origin-Opener-Policy: same-origin
|
||||
Cross-Origin-Embedder-Policy: require-corp
|
||||
Cross-Origin-Resource-Policy: same-origin
|
||||
```
|
||||
|
||||
Serve `.wasm` as `application/wasm`, keep all core files same-origin and allow
|
||||
`'wasm-unsafe-eval'` plus workers in CSP. Blob media preview additionally needs
|
||||
`media-src 'self' blob:`. The manifest intentionally does **not** require
|
||||
cross-origin isolation because ST is supported.
|
||||
|
||||
## Limits, storage and privacy
|
||||
|
||||
ffmpeg.wasm documents a hard **2 GiB input ceiling**. Real browser limits are
|
||||
often lower because sources, decoded frames, filters and outputs coexist. The
|
||||
initial application policy warns at 256 MiB input, blocks a single or total
|
||||
input above 1.5 GiB, warns at a 512 MiB output estimate and blocks an estimated
|
||||
output above 1 GiB. The in-memory queue accepts at most eight scheduled
|
||||
operations and runs only one FFmpeg operation at a time. Other defaults bound
|
||||
clips, streams, dimensions/pixels, thumbnails, contact-sheet cells and runtime.
|
||||
Refreshing or closing the page discards queued work.
|
||||
|
||||
Estimates are not guarantees. Lower resolution, duration, frame rate, pass
|
||||
count and output count can reduce failure risk. WebAssembly does not offer
|
||||
desktop FFmpeg performance or hardware encoding.
|
||||
|
||||
Source `File` objects and preview/result Blob URLs are temporary and object URLs
|
||||
are revoked on replacement/cleanup. Per-job FFmpeg files/mounts are deleted;
|
||||
cancellation discards the engine filesystem. The mounted application currently
|
||||
uses IndexedDB for validated user presets; project JSON is explicitly
|
||||
saved/loaded by the user and the operation queue remains in memory. Optional
|
||||
OPFS caching is limited to regenerable derivatives and is clearable; source
|
||||
bytes are **not persisted by default**. OPFS belongs to the current origin, so
|
||||
changing scheme/host/port does not move cached data. See
|
||||
[storage](docs/STORAGE.md).
|
||||
|
||||
Imported media, names, tags and subtitles are untrusted. They are rendered as
|
||||
text, paths are generated, operations build argument arrays rather than shell
|
||||
commands, logs are bounded/redacted, and extracted URLs are inert. See
|
||||
[security](docs/SECURITY.md).
|
||||
|
||||
## Development
|
||||
|
||||
Requirements: Node.js 22+, npm matching the lockfile, and a modern browser.
|
||||
Native FFmpeg is a development/test dependency for fixture regeneration and
|
||||
the conditional native command-plan integration suite; it is not a production
|
||||
runtime dependency.
|
||||
|
||||
```sh
|
||||
git clone git@git.add-ideas.de:zemion/av-tools.git
|
||||
cd av-tools
|
||||
npm ci
|
||||
npm run dev
|
||||
```
|
||||
|
||||
The token-free `.npmrc` selects the published Toolbox SDK scope. Do not add
|
||||
credentials to repository configuration.
|
||||
|
||||
Core assets are generated local build inputs:
|
||||
|
||||
```sh
|
||||
npm run vendor:ffmpeg
|
||||
npm run vendor:verify
|
||||
```
|
||||
|
||||
Regenerate or verify the tiny synthetic fixtures:
|
||||
|
||||
```sh
|
||||
npm run fixtures:generate
|
||||
npm run fixtures:verify
|
||||
```
|
||||
|
||||
## Checks
|
||||
|
||||
```sh
|
||||
npm run manifest:check
|
||||
npm run vendor:verify
|
||||
npm run typecheck
|
||||
npm run lint
|
||||
npm run format:check
|
||||
npm test
|
||||
npm run build
|
||||
npm run toolbox:check
|
||||
npm run test:browser
|
||||
```
|
||||
|
||||
`npm run check` combines the non-browser production checks. Browser tests cover
|
||||
the real pinned cores, MT-to-ST fallback, cancellation/recovery, generated
|
||||
fixtures, nested deployment, Toolbox context and the absence of runtime CDN
|
||||
traffic.
|
||||
|
||||
The latest recorded local Vitest run on 2026-07-24 completed **303 passing
|
||||
tests and one conditional skip**. It includes broad structured-command
|
||||
execution against native FFmpeg 7.1.3; the skipped native subtitle burn-in case
|
||||
requires a `subtitles` filter unavailable in that host build.
|
||||
|
||||
The Chromium suite exercises the pinned ST core across conversion,
|
||||
trim/concat/split, analysis, waveform, image, metadata, chapter, subtitle,
|
||||
queue and recovery workflows. The MT core is directly initialized and queried,
|
||||
and completes a real non-empty, re-probeable H.264/AAC conversion; the same
|
||||
path is benchmarked with 2-second 160×90 and 6-second 640×360 inputs. Nested
|
||||
static/Toolbox-context loading is also covered. This evidence is
|
||||
Chromium-specific: Firefox, Safari, mobile and exhaustive MT operation parity
|
||||
remain unrecorded. The exact one-run performance record and its limitations are
|
||||
in [performance observations](docs/PERFORMANCE.md).
|
||||
|
||||
## Deployment
|
||||
|
||||
Deploy the contents of `dist/` beneath any static path; URLs are base-relative
|
||||
and no server route fallback is required. Production examples:
|
||||
|
||||
- [Nginx](deployment/nginx.example.conf)
|
||||
- [Caddy](deployment/caddy.example)
|
||||
- [generic static headers](deployment/headers.md)
|
||||
|
||||
The policy permits same-origin WebAssembly and Blob workers/media without
|
||||
broad `'unsafe-eval'`. Keep stable entry/manifest/notices revalidated. Exact
|
||||
versioned core files and Vite content-hashed assets may be cached immutable.
|
||||
When isolation headers cannot be configured, ST remains the supported
|
||||
fallback.
|
||||
|
||||
## Toolbox integration
|
||||
|
||||
The one typed `de.add-ideas.av-tools` manifest generates
|
||||
`public/toolbox-app.json`. `AppShell` receives the relative
|
||||
`./toolbox-app.json` URL and falls back to standalone behavior when Toolbox
|
||||
context is absent or invalid. av-tools does not duplicate suite
|
||||
personalization or app switching.
|
||||
|
||||
The manifest uses `entry: "./"`, relative icon/assets, top-level navigation,
|
||||
local processing/no upload/no telemetry, and advertises workers, secure
|
||||
context and IndexedDB. Its `source` object identifies the canonical av-tools
|
||||
repository and the `GPL-3.0-or-later` SPDX licence.
|
||||
|
||||
Portal must consume an independently built ZIP; it must not build or import
|
||||
av-tools source. Immutable Portal revision
|
||||
`bda9da044d61c109afd9969d357cc27f76648938` has isolation and WASM CSP
|
||||
permissions but lacks `media-src blob:`, which blocks Blob previews. The
|
||||
developer's active Toolbox Portal worktree currently contains the narrow local
|
||||
`media-src 'self' blob:` patch; that uncommitted worktree state does not change
|
||||
the immutable reference or release evidence. The exact distinction, smallest
|
||||
change and fallback are documented in
|
||||
[Portal requirements](docs/PORTAL_REQUIREMENTS.md).
|
||||
|
||||
## Release
|
||||
|
||||
After all checks, normal packaging would produce deterministic:
|
||||
|
||||
```text
|
||||
release/av-tools-<version>.zip
|
||||
release/av-tools-<version>.zip.sha256
|
||||
```
|
||||
|
||||
The ZIP root contains `index.html`, `toolbox-app.json`, `CHANGELOG.md`,
|
||||
`LICENSES/`, `SOURCE.md`, `THIRD_PARTY_NOTICES.md`, `assets/` and all
|
||||
`vendor/` core files. It excludes `node_modules`, sources, source maps and
|
||||
fixtures. Paths are sorted, timestamps/modes normalized and the SHA-256
|
||||
sidecar is verified.
|
||||
|
||||
Normal packaging requires the root GPL licence and matching package/manifest
|
||||
metadata:
|
||||
|
||||
```sh
|
||||
npm run package:release
|
||||
npm run portal:smoke -- \
|
||||
--allow-known-header-gap
|
||||
```
|
||||
|
||||
The header-gap override remains development-only until the Portal CSP change is
|
||||
committed. Full commands and safety behavior are in
|
||||
[release documentation](docs/RELEASE.md).
|
||||
|
||||
## Licensing
|
||||
|
||||
av-tools-owned code and documentation are Copyright © 2026 Albrecht Degering
|
||||
and licensed **GPL-3.0-or-later**. See [LICENSE](LICENSE). The application
|
||||
provides a visible licence, source and no-warranty notice in its Help dialog.
|
||||
Third-party components retain their copyright and licence terms.
|
||||
|
||||
The wrapper/util are MIT. Core/core-mt 0.12.10 declare
|
||||
GPL-2.0-or-later and statically include FFmpeg/external codec code with
|
||||
additional terms. The branch-based x264/LAME inputs have been resolved to
|
||||
immutable commits in the source record. A binary publication must include
|
||||
equivalent access to the reviewed Corresponding Source and notices. See:
|
||||
|
||||
- [third-party notices](THIRD_PARTY_NOTICES.md)
|
||||
- [source record](SOURCE.md)
|
||||
- [licence texts/status](LICENSES/README.md)
|
||||
|
||||
The complete distributed application must not be called MIT-only. Publishing a
|
||||
binary or hosted JS/WASM build requires equivalent access to its corresponding
|
||||
source; linking only to a component homepage is not asserted to satisfy that
|
||||
obligation. Codec patent requirements are separate from copyright licensing.
|
||||
|
||||
## Known limitations
|
||||
|
||||
The auditable list is in
|
||||
[known limitations](docs/KNOWN_LIMITATIONS.md). In summary:
|
||||
|
||||
- No server transcoder, remote URL input, live capture, DRM or encrypted media.
|
||||
- No overlapping video tracks, arbitrary compositing, collaboration or
|
||||
professional nonlinear editing.
|
||||
- No “all formats,” unlimited file size, desktop performance or hardware
|
||||
encoding guarantee.
|
||||
- Stream-copy trim is fast but not promised frame-accurate.
|
||||
- Re-encoding is not lossless; lossless codec output cannot restore discarded
|
||||
source information.
|
||||
- Metadata removal is best effort, not an anonymity guarantee.
|
||||
- Browser preview supports fewer combinations than FFmpeg.
|
||||
- MT depends on isolation; ST can still hit memory/time limits.
|
||||
- Opus output is release-gated for the pinned core.
|
||||
- Projects do not contain source bytes by default and require reattachment.
|
||||
- Cross-browser/mobile and exhaustive MT performance matrices remain
|
||||
unrecorded; published support claims must stay within the tested evidence.
|
||||
|
||||
## Roadmap
|
||||
|
||||
1. Complete Quick Convert and structural-export browser verification with the
|
||||
real pinned ST/MT cores, including fallback and queued execution.
|
||||
2. Verify waveform, loudness normalization and combined filters in browsers.
|
||||
3. Verify metadata/chapter/subtitle round trips in browsers.
|
||||
4. Verify bounded thumbnails/contact sheets, projects, derivative-cache reuse
|
||||
and export in browsers.
|
||||
5. Complete accessibility, mobile, performance and nested Portal matrices.
|
||||
6. Commit/tag exact source, publish its reviewed Corresponding Source, assemble
|
||||
the exact production ZIP through Portal and verify deployed headers.
|
||||
Reference in New Issue
Block a user