159 lines
8.1 KiB
Markdown
159 lines
8.1 KiB
Markdown
# Architecture
|
|
|
|
av-tools is a static, local-first React application. The browser receives code
|
|
and the two self-hosted FFmpeg cores; media bytes do not leave the origin
|
|
through an application backend. Toolbox Portal assembles the already-built
|
|
artifact and never compiles its source.
|
|
|
|
## Layers and boundaries
|
|
|
|
```text
|
|
Toolbox AppShell / standalone context
|
|
└─ React application
|
|
├─ Quick Convert and sequential Edit workflows
|
|
├─ project + typed preset state
|
|
├─ media inspection / preview / result views
|
|
├─ bounded application queue + resource policy
|
|
└─ typed command plans
|
|
└─ EngineManager
|
|
├─ @ffmpeg/ffmpeg wrapper worker
|
|
├─ ST or MT same-origin core
|
|
└─ per-job virtual filesystem
|
|
|
|
Browser storage boundary
|
|
├─ object URLs (ephemeral previews)
|
|
├─ IndexedDB (validated user presets)
|
|
└─ OPFS cache (optional derivatives; never source in the current app)
|
|
```
|
|
|
|
- `src/toolbox/` owns the one typed manifest and suite integration.
|
|
- `src/media/` reads lightweight browser-native metadata, normalizes
|
|
authoritative ffprobe JSON, and owns duration/timecode, stream-selection,
|
|
playback-MIME and safe-name decisions. Browser metadata and probe data are
|
|
deliberately separate models.
|
|
- `src/project/` owns the versioned, non-destructive project document and
|
|
migrations. A source is represented by identity/metadata and must be
|
|
reattached after load; the current application does not persist source bytes.
|
|
- `src/presets/` defines typed built-ins and validates user presets against
|
|
detected capabilities.
|
|
- `src/commands/` converts validated operations to a `CommandPlan`. A plan has
|
|
generated input/output paths, argument arrays, expected outputs,
|
|
capabilities and cleanup intent; it is not a shell string.
|
|
- `src/ffmpeg/` selects/loads the engine, discovers capabilities, mounts
|
|
inputs, executes one job, reads outputs, sanitizes diagnostics and cleans.
|
|
- `src/jobs/` provides typed queue, progress, cancellation, recovery and
|
|
durable-metadata infrastructure. The mounted application scheduler is a
|
|
bounded in-memory promise chain around `EngineManager`.
|
|
- `src/waveform/`, `src/storage/` and `src/export/` keep bounded analysis,
|
|
persistence and browser-download responsibilities outside the UI.
|
|
|
|
The DOM receives normalized values as text/attributes. It never renders media
|
|
metadata as HTML. Command builders never accept arbitrary virtual paths or a
|
|
raw command field.
|
|
|
|
## FFmpeg lifecycle
|
|
|
|
1. Import creates an object URL immediately. A detached native media element
|
|
reads duration/dimensions where supported; playback and file identity do not
|
|
initialize FFmpeg.
|
|
2. Detailed inspection remains idle until the user requests it or an operation
|
|
requires codecs, complete streams, chapters, tags or dispositions.
|
|
3. Inspect secure-context, worker, `crossOriginIsolated` and
|
|
`SharedArrayBuffer` availability.
|
|
4. Select MT only when all MT prerequisites are true (or ST when explicitly
|
|
selected). A failed MT load is terminated and retried once with ST.
|
|
5. Load wrapper/core assets via base-relative, same-origin URLs.
|
|
6. Query version/build configuration/formats/codecs/encoders/filters
|
|
sequentially and publish immutable capability state.
|
|
7. For one job, validate resource estimates and structured settings, allocate
|
|
a generated directory, mount/write inputs, run a generated argument array,
|
|
read only declared outputs, and re-probe applicable results.
|
|
8. Return result bytes and an export report, then unmount/delete virtual files.
|
|
9. Retain the idle engine for the next sequential job; terminate it on mode
|
|
changes, unrecoverable errors or cancellation.
|
|
|
|
A detailed-probe failure changes only the inspection state. The File, object
|
|
URL, browser metadata and provisional timeline clip remain available, and the
|
|
user can retry. An authoritative probe supersedes browser duration/dimensions
|
|
and reconciles an untouched provisional full-source clip.
|
|
|
|
The reviewed Emscripten runtime can stall when Firefox probes on a core that
|
|
has retained prior application state. Before an explicit Firefox probe, the
|
|
manager recreates the identical ST/MT core and reuses only that mode's already
|
|
validated capability inventory. Chromium retains the normal warm-core path.
|
|
|
|
The FFmpeg instance is a single process. `maxConcurrentFFmpegJobs` is one and
|
|
no probe/export operations overlap on it. The application-facing scheduler is
|
|
also bounded: it retains at most eight scheduled operations in memory,
|
|
including the active operation, and advances its promise chain after either
|
|
success or failure. Queue state is deliberately not persisted across a page
|
|
reload.
|
|
|
|
## Project and edit model
|
|
|
|
A project is versioned JSON containing source descriptors, a sequential clip
|
|
timeline, non-destructive clip in/out and transform/audio settings, stream
|
|
selection, metadata/chapter/subtitle edits and export preset references. Clips
|
|
do not overlap. There is no arbitrary compositing graph, collaborative state or
|
|
server-side document.
|
|
|
|
The structural-export panel maps the selected clip's current range to either a
|
|
stream-copy or preset-backed trim plan. It maps the visible timeline order to
|
|
either a compatibility-checked stream-copy concat plan or a normalized
|
|
re-encode plan. Normalized concat requires the user to choose what happens when
|
|
some clips lack audio; unchecked fast compatibility fails closed.
|
|
|
|
Migrations are explicit version-to-version functions. Unknown future versions,
|
|
invalid shapes and unsafe source references fail with actionable validation
|
|
errors. Source bytes are not embedded in normal project JSON. Reattachment
|
|
matches stable descriptors and asks the user before associating a local file.
|
|
|
|
## Command plans and filesystem
|
|
|
|
Plans use arrays such as `['-i', generatedInput, ...]`; no shell evaluates
|
|
them. Filtergraph builders escape values for their exact FFmpeg grammar.
|
|
Generated paths contain a job id plus bounded safe basename and never reuse a
|
|
user-controlled directory. Expected output paths are declared before
|
|
execution, so the result reader cannot enumerate arbitrary output.
|
|
|
|
Small inputs may be copied into FFmpeg's in-memory filesystem. Where supported,
|
|
WORKERFS mounts `File`/`Blob` inputs to avoid an initial JavaScript copy; it
|
|
does not remove WASM address-space/output constraints. OPFS is optional for
|
|
regenerable derivatives and uses asynchronous origin-private file handles with
|
|
an in-memory fallback. See `docs/STORAGE.md`.
|
|
|
|
## Result flow
|
|
|
|
Output bytes become a Blob with an operation-derived filename and MIME type.
|
|
The UI creates a short-lived object URL only when the browser can preview that
|
|
container/codec, exposes an explicit save/download action, and includes probe
|
|
data plus warnings in the export report. Revoking a preview does not invalidate
|
|
the retained Blob until the result is cleared. Batch derivatives may be
|
|
assembled into a bounded ZIP.
|
|
|
|
For unsupported direct playback, a preview proxy is a disposable H.264/AAC MP4
|
|
derivative rather than a source mutation. Duration and maximum width are
|
|
bounded by validated user controls. Its cache identity includes the source
|
|
fingerprint and proxy settings, so only an exact matching derivative is reused;
|
|
cache failure falls back to normal generation.
|
|
|
|
## Cancellation and recovery
|
|
|
|
ffmpeg.wasm cancellation terminates the active wrapper/core worker rather than
|
|
pretending a native FFmpeg process can receive a signal. The job becomes
|
|
cancelled, outstanding promises are rejected, object URLs/mounts are released
|
|
where possible, and that engine generation is discarded. The next job creates
|
|
a fresh engine and re-runs capability discovery. A cancellation is complete
|
|
only after the stale worker cannot publish a result.
|
|
|
|
## Integration boundaries
|
|
|
|
`AppShell` owns suite chrome, theme/personalization, help and app switching.
|
|
av-tools passes `manifestUrl="./toolbox-app.json"` and works when context is
|
|
missing, malformed or unavailable. It does not embed another switcher.
|
|
|
|
The manifest advertises secure context, workers, IndexedDB and top-level
|
|
navigation. It does not require cross-origin isolation because ST is a valid
|
|
fallback. Portal receives only the deterministic ZIP/checksum and places it
|
|
under a locked target such as `apps/av/`.
|