lotobo/av-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

feat: make media inspection progressive

Browse Source
This commit is contained in:
Albrecht Degering
2026-07-27 18:25:47 +02:00
parent 6c68f9bfbd
commit f2c0fc12d2
32 changed files with 2169 additions and 462 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
README.md
View File

@@ -17,9 +17,9 @@ under Toolbox Portal with shared shell, theme, help and app switching.
Two workflows keep the product intentionally smaller than a nonlinear editor:
- **Quick Convert:** drop/import media, inspect streams, choose a
capability-checked preset/streams, convert, preview when the browser can, and
download.
- **Quick Convert:** drop/import media, play it and read browser-native file
information immediately, then inspect streams on demand when an export needs
authoritative details.
- **Edit:** a media bin and sequential non-overlapping clip timeline with
non-destructive settings, one export at a time.
@@ -225,10 +225,12 @@ profile.
`AV_TOOLS_LARGE_MEDIA=/absolute/video.mp4 npm run evidence:large-preview`
performs the opt-in large-file regression: the native video element must mount
and advance before bounded inspection finishes, remain the same element after
the authoritative probe completes and produce no browser error.
and advance while detailed inspection remains idle, browser-native
duration/dimensions must appear without loading FFmpeg, and the same element
must survive a subsequently requested authoritative probe. The command runs
that evidence in both Chromium and Firefox.
The latest recorded local Vitest run on 2026-07-26 completed **313 passing
The latest recorded local Vitest run on 2026-07-27 completed **332 passing
tests and two conditional skips**. It includes broad structured-command
execution against native FFmpeg 7.1.3; the skipped native subtitle burn-in and
labeled contact-sheet cases require `subtitles` and `drawtext` filters that are
@@ -241,9 +243,11 @@ queue and recovery workflows. The MT core is directly initialized and queried,
and completes a real non-empty, re-probeable H.264/AAC conversion; the same
path is benchmarked with 2-second 160×90 and 6-second 640×360 inputs. Nested
static/Toolbox-context loading is also covered. This evidence is
Chromium-specific: Firefox, Safari, mobile and exhaustive MT operation parity
remain unrecorded. The exact one-run performance record and its limitations are
in [performance observations](docs/PERFORMANCE.md).
Chromium-specific. Firefox now covers the opt-in large-file
play-before-inspection regression and its explicit authoritative-probe
transition; broad Firefox operation parity, Safari, mobile and exhaustive MT
operation parity remain unrecorded. The exact one-run performance record and
its limitations are in [performance observations](docs/PERFORMANCE.md).
## Deployment

35
docs/ARCHITECTURE.md
View File

@@ -27,8 +27,10 @@ Browser storage boundary
```
- `src/toolbox/` owns the one typed manifest and suite integration.
- `src/media/` normalizes ffprobe JSON, duration/timecode, stream selection,
playback MIME decisions and safe names.
- `src/media/` reads lightweight browser-native metadata, normalizes
authoritative ffprobe JSON, and owns duration/timecode, stream-selection,
playback-MIME and safe-name decisions. Browser metadata and probe data are
deliberately separate models.
- `src/project/` owns the versioned, non-destructive project document and
migrations. A source is represented by identity/metadata and must be
reattached after load; the current application does not persist source bytes.
@@ -51,20 +53,35 @@ raw command field.
## FFmpeg lifecycle
1. Inspect secure-context, worker, `crossOriginIsolated` and
1. Import creates an object URL immediately. A detached native media element
reads duration/dimensions where supported; playback and file identity do not
initialize FFmpeg.
2. Detailed inspection remains idle until the user requests it or an operation
requires codecs, complete streams, chapters, tags or dispositions.
3. Inspect secure-context, worker, `crossOriginIsolated` and
`SharedArrayBuffer` availability.
2. Select MT only when all MT prerequisites are true (or ST when explicitly
4. Select MT only when all MT prerequisites are true (or ST when explicitly
selected). A failed MT load is terminated and retried once with ST.
3. Load wrapper/core assets via base-relative, same-origin URLs.
4. Query version/build configuration/formats/codecs/encoders/filters
5. Load wrapper/core assets via base-relative, same-origin URLs.
6. Query version/build configuration/formats/codecs/encoders/filters
sequentially and publish immutable capability state.
5. For one job, validate resource estimates and structured settings, allocate
7. For one job, validate resource estimates and structured settings, allocate
a generated directory, mount/write inputs, run a generated argument array,
read only declared outputs, and re-probe applicable results.
6. Return result bytes and an export report, then unmount/delete virtual files.
7. Retain the idle engine for the next sequential job; terminate it on mode
8. Return result bytes and an export report, then unmount/delete virtual files.
9. Retain the idle engine for the next sequential job; terminate it on mode
changes, unrecoverable errors or cancellation.
A detailed-probe failure changes only the inspection state. The File, object
URL, browser metadata and provisional timeline clip remain available, and the
user can retry. An authoritative probe supersedes browser duration/dimensions
and reconciles an untouched provisional full-source clip.
The reviewed Emscripten runtime can stall when Firefox probes on a core that
has retained prior application state. Before an explicit Firefox probe, the
manager recreates the identical ST/MT core and reuses only that mode's already
validated capability inventory. Chromium retains the normal warm-core path.
The FFmpeg instance is a single process. `maxConcurrentFFmpegJobs` is one and
no probe/export operations overlap on it. The application-facing scheduler is
also bounded: it retains at most eight scheduled operations in memory,

11
docs/KNOWN_LIMITATIONS.md
View File

@@ -45,6 +45,9 @@ FFmpeg output does not remove these product/browser/resource limits.
presets from runtime formats/codecs/encoders/filters.
- FFmpeg-readable output may be download-only because browser media elements
support fewer container/codec combinations.
- Browser-native basic information intentionally omits codecs, complete stream
lists, chapters, tags, subtitles, attachments and dispositions. Features
that need those details request an authoritative local FFprobe pass.
- Browser support, codec policy and memory differ across Chromium, Firefox and
Safari and across desktop/mobile versions.
- The reviewed 0.12.10 core profile passes the non-empty Opus encode/re-probe
@@ -83,9 +86,11 @@ FFmpeg output does not remove these product/browser/resource limits.
together; development-only smoke artifacts must not be distributed.
- Chromium covers a broad real pinned-core ST operation matrix, a real MT
H.264/AAC conversion, MT-to-ST fallback, nested static/Toolbox-context
loading and one two-size ST/MT performance baseline. Firefox, Safari, mobile,
exhaustive MT feature parity and repeated performance measurements remain
unrecorded. Live Portal container/header verification is a deployment gate.
loading and one two-size ST/MT performance baseline. Firefox covers the
opt-in large-file browser-metadata/playback-to-authoritative-probe
regression. Broad Firefox operation parity, Safari, mobile, exhaustive MT
feature parity and repeated performance measurements remain unrecorded. Live
Portal container/header verification is a deployment gate.
See `docs/FORMAT_SUPPORT.md`, `docs/PERFORMANCE.md`,
`docs/PORTAL_REQUIREMENTS.md` and `docs/RELEASE.md` for testable detail.

14
docs/PERFORMANCE.md
View File

@@ -80,11 +80,15 @@ frames meet the threshold. The documented ffmpeg.wasm input ceiling is 2 GiB,
but real limits can be substantially lower on mobile or memory-constrained
browsers.
Source inspection has a bounded, size-aware watchdog: 30 seconds for small
files, a two-MiB-per-second scan budget for larger files, and a five-minute
ceiling. Recognized local audio/video can begin browser playback from its
temporary object URL while that worker inspection continues; completed probe
metadata remains authoritative for editing and export.
Import does not start FFmpeg. Recognized local audio/video can begin browser
playback from its temporary object URL while a native media element reads
lightweight duration/dimensions. Detailed FFprobe inspection starts only when
requested or required for an export and has a bounded, size-aware watchdog:
30 seconds for small files, a two-MiB-per-second budget for larger files, and a
five-minute ceiling. A detailed-inspection timeout is nonfatal; browser
playback and basic file information remain available. Completed probe metadata
remains authoritative for stream mapping, remux safety, chapters, tags,
subtitles/attachments and export validation.
Lowering resolution, duration, frame rate, passes and output count reduces
work. MT can be faster but consumes more memory/CPU and requires isolation; ST

2
package.json
View File

@@ -36,7 +36,7 @@
"test": "vitest run",
"test:watch": "vitest",
"test:browser": "playwright test",
"evidence:large-preview": "playwright test tests/browser/large-media-preview.spec.ts --project=chromium --workers=1",
"evidence:large-preview": "playwright test tests/browser/large-media-preview.spec.ts --project=chromium --project=firefox-large-media --workers=1",
"evidence:opus": "npm run vendor:ffmpeg && npm run vendor:verify-runtime && node scripts/opus-regression.mjs",
"test:opus-regression": "npm run vendor:ffmpeg && npm run vendor:verify-runtime && node scripts/opus-regression.mjs --require-pass",
"benchmark:browser": "AV_BENCHMARK=1 playwright test tests/browser/performance.spec.ts --project=chromium --workers=1",

5
playwright.config.ts
View File

@@ -21,5 +21,10 @@ export default defineConfig({
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox-large-media',
testMatch: 'large-media-preview.spec.ts',
use: { ...devices['Desktop Firefox'] },
},
],
});

640
src/App.tsx
View File

@@ -9,10 +9,7 @@ import {
type WorkspaceMode,
} from './app/application-state';
import { AppErrorBoundary } from './app/AppErrorBoundary';
import {
SourceInspectionRegistry,
type SourceInspectionToken,
} from './app/source-inspection-registry';
import { SourceInspectionRegistry } from './app/source-inspection-registry';
import { FileDropZone } from './components/FileDropZone';
import { MediaBin } from './components/MediaBin';
import {
@@ -23,6 +20,7 @@ import {
QuickConvert,
type QuickExportConfiguration,
} from './components/QuickConvert';
import { defaultQuickStreamSelection } from './components/quick-convert-selection';
import { CapabilityPanel } from './components/CapabilityPanel';
import { JobPanel } from './components/JobPanel';
import { StoragePanel } from './components/StoragePanel';
@@ -58,7 +56,11 @@ import type {
EngineState,
} from './ffmpeg/ffmpeg.types';
import { EngineError } from './ffmpeg/ffmpeg-error';
import { normalizeFfprobeReport } from './media';
import {
BrowserMetadataError,
inspectBrowserMediaMetadata,
normalizeFfprobeReport,
} from './media';
import {
createEmptyProject,
importProjectDocument,
@@ -304,6 +306,74 @@ function resultAsFile(result: ExportResult): File {
});
}
function projectWithAssetInspection(
current: AvProjectV1,
asset: ImportedMediaAsset,
probe: MediaProbe,
provisionalDuration: number | undefined
): AvProjectV1 {
if (!current.assets.some((entry) => entry.id === asset.id)) return current;
let next = projectReducer(current, {
type: 'asset/update',
assetId: asset.id,
changes: { probe, mimeType: asset.file.type },
updatedAt: timestamp(),
});
const existingClip = next.timeline.find((clip) => clip.assetId === asset.id);
if (
existingClip &&
provisionalDuration !== undefined &&
probe.durationSeconds !== undefined &&
existingClip.sourceInSeconds === 0 &&
Math.abs(existingClip.sourceOutSeconds - provisionalDuration) < 0.05 &&
Math.abs(probe.durationSeconds - provisionalDuration) >= 0.001
) {
next = projectReducer(next, {
type: 'clip/update',
clipId: existingClip.id,
changes: { sourceOutSeconds: probe.durationSeconds },
updatedAt: timestamp(),
});
} else if (
!existingClip &&
probe.durationSeconds !== undefined &&
Number.isFinite(probe.durationSeconds) &&
probe.durationSeconds >= 0.001
) {
next = projectReducer(next, {
type: 'clip/add',
clip: {
id: crypto.randomUUID(),
assetId: asset.id,
sourceInSeconds: 0,
sourceOutSeconds: probe.durationSeconds,
audio: { enabled: true },
video: { enabled: true },
},
updatedAt: timestamp(),
});
}
if (next.chapters.length === 0 && probe.chapters.length > 0) {
next = projectReducer(next, {
type: 'chapters/set',
chapters: probe.chapters.map((chapter) => ({
id: `${asset.id}-chapter-${chapter.id}`,
startSeconds: chapter.startSeconds,
endSeconds: chapter.endSeconds,
title: chapter.title ?? `Chapter ${chapter.id + 1}`,
...(chapter.timeBase ? { timeBase: chapter.timeBase } : {}),
...(Object.keys(chapter.tags).length > 0
? { metadata: chapter.tags }
: {}),
})),
updatedAt: timestamp(),
});
}
return next;
}
function concatSourceForClip(
clip: TimelineClip,
asset: ImportedMediaAsset,
@@ -567,6 +637,14 @@ export function AvToolsApplication() {
const sourceProbeTail = useRef<Promise<void>>(Promise.resolve());
const sourceProbePendingRef = useRef(0);
const sourceInspectionRegistry = useRef(new SourceInspectionRegistry());
const sourceInspectionPromises = useRef(
new Map<
string,
{ readonly objectUrl: string; readonly promise: Promise<MediaProbe> }
>()
);
const browserMetadataControllers = useRef(new Map<string, AbortController>());
const browserMetadataClipTail = useRef<Promise<void>>(Promise.resolve());
const planExecutionTail = useRef<Promise<void>>(Promise.resolve());
const scheduledPlanCountRef = useRef(0);
const scheduledWorkRef = useRef(new Map<string, ScheduledWorkDefinition>());
@@ -717,9 +795,14 @@ export function AvToolsApplication() {
useEffect(() => {
const inspectionRegistry = sourceInspectionRegistry.current;
const metadataControllers = browserMetadataControllers.current;
return () => {
const cancelActiveInspection = inspectionRegistry.clear();
if (cancelActiveInspection) void engineManager.cancelActive();
for (const controller of metadataControllers.values()) {
controller.abort();
}
metadataControllers.clear();
for (const asset of assetsRef.current) {
URL.revokeObjectURL(asset.objectUrl);
}
@@ -734,11 +817,13 @@ export function AvToolsApplication() {
const updateAssetState = useCallback(
(id: string, changes: Partial<ImportedMediaAsset>) => {
setAssets((current) =>
current.map((asset) =>
setAssets((current) => {
const next = current.map((asset) =>
asset.id === id ? { ...asset, ...changes } : asset
)
);
assetsRef.current = next;
return next;
});
},
[]
);
@@ -766,25 +851,123 @@ export function AvToolsApplication() {
[]
);
const inspectAsset = useCallback(
async (
asset: ImportedMediaAsset,
existingAsset: boolean,
token: SourceInspectionToken
) => {
if (!sourceInspectionRegistry.current.isCurrent(token)) return;
const addDefaultClip = useCallback(
(
current: AvProjectV1,
assetId: string,
durationSeconds: number | undefined
): AvProjectV1 => {
if (
durationSeconds === undefined ||
!Number.isFinite(durationSeconds) ||
durationSeconds < 0.001 ||
current.timeline.some((clip) => clip.assetId === assetId)
) {
return current;
}
return projectReducer(current, {
type: 'clip/add',
clip: {
id: crypto.randomUUID(),
assetId,
sourceInSeconds: 0,
sourceOutSeconds: durationSeconds,
audio: { enabled: true },
video: { enabled: true },
},
updatedAt: timestamp(),
});
},
[]
);
const readBrowserMetadata = useCallback(
(asset: ImportedMediaAsset) => {
browserMetadataControllers.current.get(asset.id)?.abort();
const controller = new AbortController();
browserMetadataControllers.current.set(asset.id, controller);
return inspectBrowserMediaMetadata(asset.file, asset.objectUrl, {
signal: controller.signal,
})
.then((browserMetadata) => {
const currentAsset = assetsRef.current.find(
(entry) =>
entry.id === asset.id && entry.objectUrl === asset.objectUrl
);
if (!currentAsset || controller.signal.aborted) return;
updateAssetState(asset.id, {
browserMetadata,
browserMetadataError: undefined,
});
return browserMetadata;
})
.catch((error: unknown) => {
if (
controller.signal.aborted ||
(error instanceof BrowserMetadataError && error.code === 'aborted')
) {
return;
}
const currentAsset = assetsRef.current.find(
(entry) =>
entry.id === asset.id && entry.objectUrl === asset.objectUrl
);
if (!currentAsset) return;
updateAssetState(asset.id, {
browserMetadataError:
error instanceof Error
? error.message
: 'The browser could not read basic media metadata.',
});
return undefined;
})
.finally(() => {
if (browserMetadataControllers.current.get(asset.id) === controller) {
browserMetadataControllers.current.delete(asset.id);
}
});
},
[updateAssetState]
);
const inspectAsset = useCallback(
(requestedAsset: ImportedMediaAsset): Promise<MediaProbe> => {
const currentAsset = assetsRef.current.find(
(entry) =>
entry.id === requestedAsset.id &&
entry.objectUrl === requestedAsset.objectUrl
);
if (!currentAsset) {
return Promise.reject(
new EngineError(
'cancelled',
'The media source is no longer attached.'
)
);
}
if (currentAsset.probe) return Promise.resolve(currentAsset.probe);
const active = sourceInspectionPromises.current.get(currentAsset.id);
if (active?.objectUrl === currentAsset.objectUrl) return active.promise;
const token = sourceInspectionRegistry.current.register(currentAsset.id);
updateAssetState(currentAsset.id, {
phase: 'probing',
error: undefined,
});
const promise = (async (): Promise<MediaProbe> => {
try {
const probeResult = await runSourceProbe(async () => {
if (!sourceInspectionRegistry.current.activate(token)) {
return undefined;
throw new EngineError(
'cancelled',
'The detailed media inspection was superseded.'
);
}
try {
return await engineManager.probe(
asset.file,
currentAsset.file,
undefined,
token.signal
);
@@ -792,91 +975,65 @@ export function AvToolsApplication() {
sourceInspectionRegistry.current.finish(token);
}
});
if (
!probeResult ||
!sourceInspectionRegistry.current.isCurrent(token)
) {
return;
if (!sourceInspectionRegistry.current.isCurrent(token)) {
throw new EngineError(
'cancelled',
'The detailed media inspection was superseded.'
);
}
const probe = normalizeFfprobeReport(probeResult.json);
const rawProbe = `${JSON.stringify(probeResult.json, null, 2)}\n`;
const importedChapters = probe.chapters.map((chapter) => ({
id: `${asset.id}-chapter-${chapter.id}`,
startSeconds: chapter.startSeconds,
endSeconds: chapter.endSeconds,
title: chapter.title ?? `Chapter ${chapter.id + 1}`,
...(chapter.timeBase ? { timeBase: chapter.timeBase } : {}),
...(Object.keys(chapter.tags).length > 0
? { metadata: chapter.tags }
: {}),
}));
if (!sourceInspectionRegistry.current.isCurrent(token)) return;
updateAssetState(asset.id, { phase: 'ready', probe, rawProbe });
const provisionalDuration = assetsRef.current.find(
(entry) =>
entry.id === currentAsset.id &&
entry.objectUrl === currentAsset.objectUrl
)?.browserMetadata?.durationSeconds;
updateAssetState(currentAsset.id, {
phase: 'ready',
probe,
rawProbe,
error: undefined,
});
setProject((current) => {
if (
!sourceInspectionRegistry.current.isCurrent(token) ||
!current.assets.some((entry) => entry.id === asset.id)
) {
if (!sourceInspectionRegistry.current.isCurrent(token)) {
return current;
}
let next = projectReducer(current, {
type: 'asset/update',
assetId: asset.id,
changes: { probe, mimeType: asset.file.type },
updatedAt: timestamp(),
});
if (
existingAsset &&
next.assets.find((entry) => entry.id === asset.id)?.sourceStatus ===
'missing'
) {
next = projectReducer(next, {
type: 'asset/set-source-status',
assetId: asset.id,
sourceStatus: 'attached',
updatedAt: timestamp(),
});
}
const hasClip = next.timeline.some(
(clip) => clip.assetId === asset.id
const next = projectWithAssetInspection(
current,
currentAsset,
probe,
provisionalDuration
);
if (
!hasClip &&
probe.durationSeconds !== undefined &&
probe.durationSeconds >= 0.001
) {
next = projectReducer(next, {
type: 'clip/add',
clip: {
id: crypto.randomUUID(),
assetId: asset.id,
sourceInSeconds: 0,
sourceOutSeconds: probe.durationSeconds,
audio: { enabled: true },
video: { enabled: true },
},
updatedAt: timestamp(),
});
}
if (next.chapters.length === 0 && importedChapters.length > 0) {
next = projectReducer(next, {
type: 'chapters/set',
chapters: importedChapters,
updatedAt: timestamp(),
});
}
projectRef.current = next;
return next;
});
return probe;
} catch (error) {
if (!sourceInspectionRegistry.current.isCurrent(token)) return;
updateAssetState(asset.id, {
if (sourceInspectionRegistry.current.isCurrent(token)) {
updateAssetState(currentAsset.id, {
phase: 'error',
error:
error instanceof Error
? error.message
: 'The file could not be inspected.',
: 'Detailed media inspection failed.',
});
}
throw error;
} finally {
const activeInspection = sourceInspectionPromises.current.get(
currentAsset.id
);
if (activeInspection?.objectUrl === currentAsset.objectUrl) {
sourceInspectionPromises.current.delete(currentAsset.id);
}
}
})();
sourceInspectionPromises.current.set(currentAsset.id, {
objectUrl: currentAsset.objectUrl,
promise,
});
return promise;
},
[runSourceProbe, updateAssetState]
);
@@ -919,16 +1076,12 @@ export function AvToolsApplication() {
.map((issue) => issue.message)
.join(
'\n'
)}\n\n${importDecision.estimateDisclaimer}\n\nImport and inspect anyway?`
)}\n\n${importDecision.estimateDisclaimer}\n\nImport anyway?`
)
) {
return;
}
const pending: Array<{
asset: ImportedMediaAsset;
existingAsset: boolean;
token: SourceInspectionToken;
}> = [];
const pending: ImportedMediaAsset[] = [];
const claimedIds = new Set(assetsRef.current.map((asset) => asset.id));
for (const file of files) {
@@ -963,11 +1116,15 @@ export function AvToolsApplication() {
if (reattach && match) {
asset.id = match.entry.id;
claimedIds.add(match.entry.id);
pending.push({
asset,
existingAsset: true,
token: sourceInspectionRegistry.current.register(asset.id),
});
setProject((current) =>
projectReducer(current, {
type: 'asset/set-source-status',
assetId: asset.id,
sourceStatus: 'attached',
updatedAt: timestamp(),
})
);
pending.push(asset);
} else {
setProject((current) =>
projectReducer(current, {
@@ -983,31 +1140,53 @@ export function AvToolsApplication() {
updatedAt: timestamp(),
})
);
pending.push({
asset,
existingAsset: false,
token: sourceInspectionRegistry.current.register(asset.id),
});
pending.push(asset);
}
}
setAssets((current) => {
const next = [...current, ...pending.map(({ asset }) => asset)];
const next = [...current, ...pending];
assetsRef.current = next;
return next;
});
setSelectedAssetId((current) => current ?? pending[0]?.asset.id);
for (const item of pending) {
await inspectAsset(item.asset, item.existingAsset, item.token);
setSelectedAssetId((current) => current ?? pending[0]?.id);
for (const asset of pending) {
const metadata = readBrowserMetadata(asset);
browserMetadataClipTail.current = browserMetadataClipTail.current
.then(async () => {
const resolved = await metadata;
if (!resolved) return;
setProject((current) => {
if (!current.assets.some((entry) => entry.id === asset.id)) {
return current;
}
const next = addDefaultClip(
current,
asset.id,
resolved.durationSeconds
);
projectRef.current = next;
return next;
});
})
.catch(() => undefined);
}
},
[inspectAsset, project.assets, resourceContext.policy]
[
addDefaultClip,
project.assets,
readBrowserMetadata,
resourceContext.policy,
]
);
const removeAsset = useCallback(
(id: string) => {
const asset = assets.find((entry) => entry.id === id);
if (asset) URL.revokeObjectURL(asset.objectUrl);
browserMetadataControllers.current.get(id)?.abort();
browserMetadataControllers.current.delete(id);
sourceInspectionPromises.current.delete(id);
const cancelActiveInspection =
sourceInspectionRegistry.current.remove(id);
setAssets((current) => {
@@ -1048,6 +1227,33 @@ export function AvToolsApplication() {
[applyProjectAction, assets, selectedAssetId]
);
const requestDetailedInspection = useCallback(
(assetId: string) => {
const asset = assetsRef.current.find((entry) => entry.id === assetId);
if (!asset) return;
setOperationError(undefined);
if (
scheduledPlanCountRef.current > 0 ||
(engineManager.state.status === 'running' &&
sourceProbePendingRef.current === 0)
) {
setOperationError(
'Wait for the active export or analysis job before inspecting another source.'
);
return;
}
void inspectAsset(asset).catch((error: unknown) => {
if (error instanceof EngineError && error.code === 'cancelled') return;
setOperationError(
error instanceof Error
? `Detailed inspection failed: ${error.message}`
: 'Detailed inspection failed.'
);
});
},
[inspectAsset]
);
const selectedAsset = assets.find((asset) => asset.id === selectedAssetId);
const selectedClip = project.timeline.find(
(clip) => clip.id === selectedClipId
@@ -1625,9 +1831,18 @@ export function AvToolsApplication() {
const exportQuick = useCallback(
async (configuration: QuickExportConfiguration) => {
if (!selectedAsset?.probe) return;
if (!selectedAsset) return;
const jobId = crypto.randomUUID();
try {
const probe =
selectedAsset.probe ?? (await inspectAsset(selectedAsset));
const streamSelection = configuration.selectDetectedStreams
? defaultQuickStreamSelection(
probe,
configuration.operation,
configuration.preset
)
: configuration.streamSelection;
let plan: FFmpegCommandPlan;
if (configuration.operation === 'remux') {
const container = configuration.remuxContainer ?? 'matroska';
@@ -1646,7 +1861,7 @@ export function AvToolsApplication() {
? 'ogg'
: container,
description: {
streams: selectedAsset.probe.streams.flatMap((stream) =>
streams: probe.streams.flatMap((stream) =>
stream.type === 'video' ||
stream.type === 'audio' ||
stream.type === 'subtitle' ||
@@ -1661,13 +1876,13 @@ export function AvToolsApplication() {
]
: []
),
hasChapters: selectedAsset.probe.chapters.length > 0,
hasMetadata: Object.keys(selectedAsset.probe.tags).length > 0,
hasChapters: probe.chapters.length > 0,
hasMetadata: Object.keys(probe.tags).length > 0,
},
streamSelection: configuration.streamSelection,
streamSelection,
metadataPolicy: configuration.removeMetadata ? 'remove' : 'copy',
chapterPolicy: configuration.removeChapters ? 'remove' : 'keep',
expectedDurationSeconds: selectedAsset.probe.durationSeconds,
expectedDurationSeconds: probe.durationSeconds,
});
} else {
if (!configuration.preset) return;
@@ -1683,8 +1898,8 @@ export function AvToolsApplication() {
metadataPolicy: configuration.removeMetadata ? 'remove' : 'copy',
chapterPolicy: configuration.removeChapters ? 'remove' : 'keep',
},
streamSelection: configuration.streamSelection,
expectedDurationSeconds: selectedAsset.probe.durationSeconds,
streamSelection,
expectedDurationSeconds: probe.durationSeconds,
});
}
await runPlan(plan, [selectedAsset.file], plan.operation);
@@ -1698,7 +1913,7 @@ export function AvToolsApplication() {
);
}
},
[runPlan, selectedAsset]
[inspectAsset, runPlan, selectedAsset]
);
const extractThumbnail = useCallback(
@@ -1763,14 +1978,19 @@ export function AvToolsApplication() {
const createPreviewProxy = useCallback(
async (asset: ImportedMediaAsset | undefined) => {
const duration = asset?.probe?.durationSeconds;
if (!asset || duration === undefined || duration <= 0) return;
if (!asset) return;
try {
const probe = asset.probe ?? (await inspectAsset(asset));
const duration = probe.durationSeconds;
if (duration === undefined || duration <= 0) {
throw new Error(
'The source duration is unavailable after detailed inspection.'
);
}
const includeAudio =
asset.probe?.streams.some((stream) => stream.type === 'audio') ===
true;
probe.streams.some((stream) => stream.type === 'audio') === true;
const hasVideo =
asset.probe?.streams.some(
probe.streams.some(
(stream) =>
stream.type === 'video' &&
stream.disposition.attached_pic !== true
@@ -1861,6 +2081,7 @@ export function AvToolsApplication() {
}
},
[
inspectAsset,
previewProxySettings,
project.id,
refreshResults,
@@ -1888,7 +2109,39 @@ export function AvToolsApplication() {
}
const timelineContainer = preset.container as 'mp4' | 'webm' | 'matroska';
try {
const enabledSubtitleTracks = project.subtitles.filter(
const timelineAssetIds = [
...new Set(project.timeline.map((clip) => clip.assetId)),
];
const timelineInspections = new Map<
string,
{ readonly asset: ImportedMediaAsset; readonly probe: MediaProbe }
>();
for (const assetId of timelineAssetIds) {
const asset = assetsRef.current.find((entry) => entry.id === assetId);
if (!asset) {
throw new Error(
'Reattach every timeline source before rendering the project.'
);
}
const probe = asset.probe ?? (await inspectAsset(asset));
timelineInspections.set(assetId, { asset, probe });
}
let inspectedProject = projectRef.current;
for (const { asset, probe } of timelineInspections.values()) {
const latestAsset =
assetsRef.current.find(
(entry) =>
entry.id === asset.id && entry.objectUrl === asset.objectUrl
) ?? asset;
inspectedProject = projectWithAssetInspection(
inspectedProject,
latestAsset,
probe,
latestAsset.browserMetadata?.durationSeconds
);
}
projectRef.current = inspectedProject;
const enabledSubtitleTracks = inspectedProject.subtitles.filter(
(entry) => entry.mode !== 'exclude'
);
const timelineStepCount =
@@ -1897,13 +2150,15 @@ export function AvToolsApplication() {
enabledSubtitleTracks.filter(
(entry) => (entry.offsetSeconds ?? 0) !== 0
).length +
(project.chapters.length > 0 ? 1 : 0);
(inspectedProject.chapters.length > 0 ? 1 : 0);
await runWorkflow(
'Render timeline project',
timelineStepCount,
async (executeStep) => {
const resolved = project.timeline.map((clip, index) => {
const asset = assets.find((entry) => entry.id === clip.assetId);
const resolved = inspectedProject.timeline.map((clip, index) => {
const asset = assetsRef.current.find(
(entry) => entry.id === clip.assetId
);
const video = asset?.probe?.streams.find(
(stream) =>
stream.type === 'video' &&
@@ -2001,7 +2256,7 @@ export function AvToolsApplication() {
sourceDurationSeconds: asset.probe?.durationSeconds,
hasVideo: true,
hasAudio:
project.output.audioEnabled &&
inspectedProject.output.audioEnabled &&
clip.audio?.enabled !== false &&
asset.probe?.streams.some(
(stream) => stream.type === 'audio'
@@ -2035,9 +2290,13 @@ export function AvToolsApplication() {
const even = (value: number) =>
Math.max(2, Math.round(value / 2) * 2);
const requestedWidth =
project.output.width ?? timelineVideoPreset.width ?? undefined;
inspectedProject.output.width ??
timelineVideoPreset.width ??
undefined;
const requestedHeight =
project.output.height ?? timelineVideoPreset.height ?? undefined;
inspectedProject.output.height ??
timelineVideoPreset.height ??
undefined;
let outputWidth: number;
let outputHeight: number;
if (requestedWidth && requestedHeight) {
@@ -2064,7 +2323,7 @@ export function AvToolsApplication() {
),
];
if (
project.output.frameRate === undefined &&
inspectedProject.output.frameRate === undefined &&
timelineVideoPreset.frameRate === undefined &&
requestedClipFrameRates.length > 1
) {
@@ -2073,28 +2332,34 @@ export function AvToolsApplication() {
);
}
const frameRate =
project.output.frameRate ??
inspectedProject.output.frameRate ??
timelineVideoPreset.frameRate ??
requestedClipFrameRates[0] ??
first.sourceFrameRate ??
30;
const sampleRate =
project.output.sampleRate ?? preset.audio?.sampleRate ?? 48_000;
inspectedProject.output.sampleRate ??
preset.audio?.sampleRate ??
48_000;
const channelLayout =
project.output.channelLayout ??
inspectedProject.output.channelLayout ??
(preset.audio?.channels === 1
? 'mono'
: preset.audio?.channels === 6
? '5.1'
: 'stereo');
const missingAudioPolicy =
!project.output.audioEnabled || !preset.audio
!inspectedProject.output.audioEnabled || !preset.audio
? ('drop-all' as const)
: (project.output.missingAudioPolicy ?? 'insert-silence');
const outputFileName = safeExportFileName(project.output.fileName, {
: (inspectedProject.output.missingAudioPolicy ??
'insert-silence');
const outputFileName = safeExportFileName(
inspectedProject.output.fileName,
{
extension: preset.fileExtension,
fallback: 'timeline',
});
}
);
const plan = buildTimelineExportPlan({
jobId: crypto.randomUUID(),
clips: resolved.map(({ clip }) => clip),
@@ -2124,7 +2389,7 @@ export function AvToolsApplication() {
0
);
const activeSubtitleTracks = project.subtitles
const activeSubtitleTracks = inspectedProject.subtitles
.filter((entry) => entry.mode !== 'exclude')
.sort((left, right) => {
if (left.mode === right.mode) return 0;
@@ -2165,7 +2430,7 @@ export function AvToolsApplication() {
}
const mediaSource = {
id: `timeline-${project.id}`,
id: `timeline-${inspectedProject.id}`,
sourceIndex: 0,
fileName: currentFile.name,
};
@@ -2225,16 +2490,16 @@ export function AvToolsApplication() {
currentFile = resultAsFile(subtitledResult);
}
if (project.chapters.length > 0) {
if (inspectedProject.chapters.length > 0) {
const chaptered = await executeStep(
buildChapterMuxPlan({
jobId: crypto.randomUUID(),
source: {
id: `timeline-${project.id}`,
id: `timeline-${inspectedProject.id}`,
sourceIndex: 0,
fileName: currentFile.name,
},
chapters: project.chapters.map((chapter) => ({
chapters: inspectedProject.chapters.map((chapter) => ({
id: chapter.id,
title: chapter.title,
startSeconds: chapter.startSeconds,
@@ -2259,12 +2524,14 @@ export function AvToolsApplication() {
currentFile = resultAsFile(chapteredResult);
}
const metadataEdits = metadataEditsFromProject(project.metadata);
const metadataEdits = metadataEditsFromProject(
inspectedProject.metadata
);
const finalized = await executeStep(
buildMetadataEditPlan({
jobId: crypto.randomUUID(),
source: {
id: `timeline-${project.id}`,
id: `timeline-${inspectedProject.id}`,
sourceIndex: 0,
fileName: currentFile.name,
},
@@ -2295,7 +2562,7 @@ export function AvToolsApplication() {
);
}
}, [
assets,
inspectAsset,
loudnessByAsset,
project,
refreshResults,
@@ -2307,6 +2574,11 @@ export function AvToolsApplication() {
const resetProject = useCallback(() => {
for (const asset of assets) URL.revokeObjectURL(asset.objectUrl);
for (const controller of browserMetadataControllers.current.values()) {
controller.abort();
}
browserMetadataControllers.current.clear();
sourceInspectionPromises.current.clear();
const cancelActiveInspection = sourceInspectionRegistry.current.clear();
if (cancelActiveInspection) void engineManager.cancelActive();
assetsRef.current = [];
@@ -2345,6 +2617,11 @@ export function AvToolsApplication() {
try {
const imported = importProjectDocument(await file.text());
for (const asset of assets) URL.revokeObjectURL(asset.objectUrl);
for (const controller of browserMetadataControllers.current.values()) {
controller.abort();
}
browserMetadataControllers.current.clear();
sourceInspectionPromises.current.clear();
const cancelActiveInspection = sourceInspectionRegistry.current.clear();
if (cancelActiveInspection) void engineManager.cancelActive();
assetsRef.current = [];
@@ -3578,10 +3855,7 @@ export function AvToolsApplication() {
: requirementsForUserPreset(preset);
const availability = activeCapabilities
? presetAvailability({ requirements }, activeCapabilities)
: {
status: 'unavailable' as const,
reasons: ['Initialize the engine to verify this preset.'],
};
: undefined;
const opusBlocked =
preset.audio?.codec === 'libopus' && !FFMPEG_OPUS_REGRESSION_VERIFIED;
return {
@@ -3590,11 +3864,12 @@ export function AvToolsApplication() {
fileExtension: preset.fileExtension,
supportsAudio: preset.audio !== undefined,
supportsVideo: preset.video !== undefined,
...(availability.status === 'unavailable' || opusBlocked
...(availability?.status === 'unavailable' || opusBlocked
? {
disabledReason: opusBlocked
? 'Opus output is disabled until the reviewed core passes the non-empty ST/MT regression encode and re-probe gate.'
: availability.reasons.join(' '),
: (availability?.reasons.join(' ') ??
'The selected preset is unavailable.'),
}
: {}),
};
@@ -3617,6 +3892,10 @@ export function AvToolsApplication() {
: !selectedTimelinePreset?.video
? 'Choose a video export preset.'
: selectedTimelinePresetOption?.disabledReason;
const timelineNeedsInspection = project.timeline.some((clip) => {
const asset = assets.find((entry) => entry.id === clip.assetId);
return asset !== undefined && asset.probe === undefined;
});
const anyAccuratePreset = advancedPresetOptions.some(
(preset) => !preset.disabledReason
@@ -3931,6 +4210,8 @@ export function AvToolsApplication() {
scheduledPlanCount >= 8;
const sourceManagementBusy =
sourceProbeBusy || scheduledPlanCount > 0 || waveformProgress !== undefined;
const sourceRemovalBusy =
scheduledPlanCount > 0 || waveformProgress !== undefined;
const editWorkspace = (
<>
@@ -3993,6 +4274,9 @@ export function AvToolsApplication() {
assets={assets}
selectedId={selectedAssetId}
onSelect={setSelectedAssetId}
onInspect={requestDetailedInspection}
inspectDisabled={sourceManagementBusy}
removeDisabled={sourceRemovalBusy}
onRemove={removeAsset}
/>
</section>
@@ -4059,6 +4343,7 @@ export function AvToolsApplication() {
}
onSubtitlesChange={changeSubtitles}
onSubtitleImport={importSubtitleFile}
onInspectDetails={requestDetailedInspection}
onThumbnail={extractThumbnail}
currentPreviewTimeSeconds={
inspectedClipAsset
@@ -4114,6 +4399,42 @@ export function AvToolsApplication() {
if (selectedClipId === clipId) setSelectedClipId(undefined);
}}
/>
{inspectedClipAsset && !inspectedClipAsset.probe ? (
<div
className={`notice ${
inspectedClipAsset.phase === 'error' ? 'notice--warning' : ''
}`}
role="status"
>
<Icon
name={inspectedClipAsset.phase === 'error' ? 'warning' : 'info'}
/>
<div>
<strong>
{inspectedClipAsset.phase === 'probing'
? 'Inspecting source details…'
: 'Advanced tools need source details.'}
</strong>
<small>
Playback and basic metadata stay available while codecs, streams
and chapters are inspected locally.
</small>
</div>
{inspectedClipAsset.phase !== 'probing' ? (
<button
className="button button--secondary"
type="button"
disabled={sourceManagementBusy}
onClick={() => requestDetailedInspection(inspectedClipAsset.id)}
>
<Icon name="info" />
{inspectedClipAsset.phase === 'error'
? 'Retry inspection'
: 'Enable advanced tools'}
</button>
) : null}
</div>
) : null}
<StructuralOperations
{...(structuralSelectedClip
? { selectedClip: structuralSelectedClip }
@@ -4504,6 +4825,8 @@ export function AvToolsApplication() {
<Icon name="sparkles" />
{engineState.status === 'running' || scheduledPlanCount > 0
? 'Add timeline to queue'
: timelineNeedsInspection
? 'Inspect & render timeline'
: 'Render timeline'}
</button>
</div>
@@ -4632,6 +4955,9 @@ export function AvToolsApplication() {
assets={assets}
selectedId={selectedAssetId}
onSelect={setSelectedAssetId}
onInspect={requestDetailedInspection}
inspectDisabled={sourceManagementBusy}
removeDisabled={sourceRemovalBusy}
onRemove={removeAsset}
/>
</section>
@@ -4645,6 +4971,9 @@ export function AvToolsApplication() {
engineState.status === 'running' || scheduledPlanCount > 0
}
presets={[...BUILT_IN_PRESETS, ...userPresets]}
onInspect={() => {
if (selectedAsset) requestDetailedInspection(selectedAsset.id);
}}
onExport={exportQuick}
/>
<section className="panel quick-preview">
@@ -4722,9 +5051,18 @@ export function AvToolsApplication() {
preference={enginePreference}
resourcePolicy={resourceContext.policy}
deviceMemoryGiB={resourceContext.deviceMemoryGiB}
disabled={sourceManagementBusy}
onPreferenceChange={(preference) => {
setEnginePreference(preference);
try {
engineManager.setPreference(preference);
setEnginePreference(preference);
} catch (error) {
setOperationError(
error instanceof Error
? error.message
: 'The engine mode could not be changed.'
);
}
}}
onInitialize={() => {
if (engineManager.state.status === 'error') {

52
src/app/application-state.ts
View File

@@ -1,12 +1,21 @@
import type { MediaProbe } from '../media';
import type { BrowserMediaMetadata, MediaProbe } from '../media';
export type AssetPhase = 'queued' | 'probing' | 'ready' | 'error';
/**
* State of the optional authoritative FFprobe inspection.
*
* The File and its object URL are usable in every phase. In particular,
* `error` means only that detailed metadata is unavailable; it never marks the
* local source or browser playback as failed.
*/
export type AssetPhase = 'idle' | 'probing' | 'ready' | 'error';
export interface ImportedMediaAsset {
id: string;
file: File;
objectUrl: string;
phase: AssetPhase;
browserMetadata?: BrowserMediaMetadata;
browserMetadataError?: string;
probe?: MediaProbe;
rawProbe?: string;
error?: string;
@@ -29,10 +38,47 @@ export function createAsset(file: File): ImportedMediaAsset {
id: crypto.randomUUID(),
file,
objectUrl: URL.createObjectURL(file),
phase: 'queued',
phase: 'idle',
};
}
export function assetDurationSeconds(
asset: ImportedMediaAsset | undefined
): number | undefined {
return (
asset?.probe?.durationSeconds ?? asset?.browserMetadata?.durationSeconds
);
}
export function assetHasVideo(
asset: ImportedMediaAsset | undefined
): boolean | undefined {
if (asset?.probe) {
return asset.probe.streams.some(
(stream) =>
stream.type === 'video' && stream.disposition.attached_pic !== true
);
}
if (asset?.browserMetadata?.hasVideo !== undefined) {
return asset.browserMetadata.hasVideo;
}
if (asset?.browserMetadata?.kind === 'video') return true;
if (asset?.browserMetadata?.kind === 'audio') return false;
return undefined;
}
export function assetHasAudio(
asset: ImportedMediaAsset | undefined
): boolean | undefined {
if (asset?.probe) {
return asset.probe.streams.some((stream) => stream.type === 'audio');
}
if (asset?.browserMetadata?.hasAudio !== undefined) {
return asset.browserMetadata.hasAudio;
}
return asset?.browserMetadata?.kind === 'audio' ? true : undefined;
}
export function formatBytes(bytes: number): string {
if (!Number.isFinite(bytes) || bytes < 0) return 'Unknown size';
if (bytes < 1_000) return `${bytes} B`;

15
src/components/CapabilityPanel.tsx
View File

@@ -12,6 +12,7 @@ export interface CapabilityPanelProps {
preference: EnginePreference;
resourcePolicy: ResourcePolicy;
deviceMemoryGiB?: number;
disabled?: boolean;
onPreferenceChange: (preference: EnginePreference) => void;
onInitialize: () => void;
}
@@ -21,6 +22,7 @@ export function CapabilityPanel({
preference,
resourcePolicy,
deviceMemoryGiB,
disabled = false,
onPreferenceChange,
onInitialize,
}: CapabilityPanelProps) {
@@ -58,7 +60,12 @@ export function CapabilityPanel({
Engine mode
<select
value={preference}
disabled={state.status === 'running'}
disabled={
disabled ||
state.status === 'loading' ||
state.status === 'recovering' ||
state.status === 'running'
}
onChange={(event) =>
onPreferenceChange(event.currentTarget.value as EnginePreference)
}
@@ -188,7 +195,11 @@ export function CapabilityPanel({
</p>
</section>
{state.status === 'idle' ? (
<button className="button button--secondary" onClick={onInitialize}>
<button
className="button button--secondary"
disabled={disabled}
onClick={onInitialize}
>
Initialize engine
</button>
) : null}

10
src/components/HelpDialog.tsx
View File

@@ -60,10 +60,9 @@ export function HelpDialog({ open, onClose }: HelpDialogProps) {
<span className="step-number">1</span>
<h3>Add media</h3>
<p>
Pick or drop local files. The browser gives FFmpeg direct,
read-only access when WORKERFS is available. If it must use the
bounded memory-copy fallback, the job log reports the exact copied
byte count.
Pick or drop local files. Playback, file identity, duration and
dimensions use the browser first; FFmpeg stays unloaded until
detailed streams or an export are requested.
</p>
</section>
<section>
@@ -71,7 +70,8 @@ export function HelpDialog({ open, onClose }: HelpDialogProps) {
<h3>Choose an operation</h3>
<p>
Quick Convert produces one output. Edit arranges non-destructive
clips on a sequential timeline.
clips on a sequential timeline. Detailed inspection adds codecs,
complete streams, chapters and tags only when needed.
</p>
</section>
<section>

120
src/components/Inspector.tsx
View File

@@ -1,6 +1,11 @@
import { useState } from 'react';
import type { ImportedMediaAsset } from '../app/application-state';
import { formatBytes, formatDuration } from '../app/application-state';
import {
assetDurationSeconds,
assetHasVideo,
formatBytes,
formatDuration,
} from '../app/application-state';
import type {
AvProjectV1,
FadeCurve,
@@ -39,6 +44,7 @@ export interface InspectorProps {
onChaptersChange: (chapters: ProjectChapter[]) => void;
onSubtitlesChange: (tracks: ProjectSubtitleTrack[]) => void;
onSubtitleImport?: (file: File) => void;
onInspectDetails?: (assetId: string) => void;
onThumbnail: (request: ThumbnailRequest) => void;
currentPreviewTimeSeconds?: number;
disabled?: boolean;
@@ -74,6 +80,7 @@ export function Inspector({
onChaptersChange,
onSubtitlesChange,
onSubtitleImport,
onInspectDetails,
onThumbnail,
currentPreviewTimeSeconds,
disabled = false,
@@ -119,7 +126,13 @@ export function Inspector({
))}
</div>
<div className="inspector__body">
{tab === 'media' ? <MediaInspector asset={asset} /> : null}
{tab === 'media' ? (
<MediaInspector
asset={asset}
disabled={disabled}
onInspectDetails={onInspectDetails}
/>
) : null}
{tab === 'transform' ? (
<TransformInspector
clip={clip}
@@ -159,7 +172,7 @@ export function Inspector({
) : null}
{tab === 'thumbnails' ? (
<ThumbnailInspector
duration={asset?.probe?.durationSeconds}
duration={assetDurationSeconds(asset)}
currentPreviewTimeSeconds={currentPreviewTimeSeconds}
disabled={disabled || !asset}
onCreate={onThumbnail}
@@ -170,20 +183,30 @@ export function Inspector({
);
}
function MediaInspector({ asset }: { asset?: ImportedMediaAsset }) {
function MediaInspector({
asset,
disabled,
onInspectDetails,
}: {
asset?: ImportedMediaAsset;
disabled: boolean;
onInspectDetails?: (assetId: string) => void;
}) {
if (!asset) return <p className="panel-empty">Select a media source.</p>;
const probe = asset.probe;
const durationSeconds = assetDurationSeconds(asset);
const hasVideo = assetHasVideo(asset) === true;
const width =
probe?.streams.find((stream) => stream.type === 'video')?.width ??
asset.browserMetadata?.width;
const height =
probe?.streams.find((stream) => stream.type === 'video')?.height ??
asset.browserMetadata?.height;
return (
<div className="inspector-section">
<div className="inspector-file">
<span>
<Icon
name={
probe?.streams.some((stream) => stream.type === 'video')
? 'video'
: 'audio'
}
/>
<Icon name={hasVideo ? 'video' : 'audio'} />
</span>
<div>
<strong>{asset.file.name}</strong>
@@ -191,6 +214,18 @@ function MediaInspector({ asset }: { asset?: ImportedMediaAsset }) {
</div>
</div>
<dl className="detail-list">
<div>
<dt>File type</dt>
<dd>
{asset.browserMetadata?.file.declaredMimeType &&
asset.browserMetadata.file.declaredMimeType !==
'application/octet-stream'
? asset.browserMetadata.file.declaredMimeType
: (asset.browserMetadata?.file.inferredMimeType ??
asset.browserMetadata?.file.declaredMimeType ??
(asset.file.type || '—'))}
</dd>
</div>
<div>
<dt>Container</dt>
<dd>
@@ -199,8 +234,21 @@ function MediaInspector({ asset }: { asset?: ImportedMediaAsset }) {
</div>
<div>
<dt>Duration</dt>
<dd>{formatDuration(probe?.durationSeconds)}</dd>
<dd>
{formatDuration(durationSeconds)}
{!probe && durationSeconds !== undefined ? ' · browser' : ''}
</dd>
</div>
{hasVideo ? (
<div>
<dt>Dimensions</dt>
<dd>
{width !== undefined && height !== undefined
? `${width}×${height}${probe ? '' : ' · browser'}`
: '—'}
</dd>
</div>
) : null}
<div>
<dt>Bit rate</dt>
<dd>
@@ -210,6 +258,41 @@ function MediaInspector({ asset }: { asset?: ImportedMediaAsset }) {
</dd>
</div>
</dl>
{!probe ? (
<div
className={`notice ${
asset.phase === 'error' ? 'notice--warning' : ''
}`}
role="status"
>
<Icon name={asset.phase === 'error' ? 'warning' : 'info'} />
<div>
<strong>
{asset.phase === 'probing'
? 'Inspecting detailed media information…'
: asset.phase === 'error'
? 'Detailed inspection is unavailable.'
: 'Detailed information has not been inspected.'}
</strong>
<small>
{asset.phase === 'error'
? asset.error
: 'Codecs, complete streams, chapters and tags are loaded only when needed.'}
</small>
</div>
{asset.phase !== 'probing' && onInspectDetails ? (
<button
type="button"
className="button button--secondary"
disabled={disabled}
onClick={() => onInspectDetails(asset.id)}
>
<Icon name="info" />
{asset.phase === 'error' ? 'Retry inspection' : 'Inspect details'}
</button>
) : null}
</div>
) : null}
{probe && Object.keys(probe.tags).length > 0 ? (
<>
<h3>Container tags</h3>
@@ -223,7 +306,8 @@ function MediaInspector({ asset }: { asset?: ImportedMediaAsset }) {
</dl>
</>
) : null}
<h3>Streams</h3>
{probe ? <h3>Streams</h3> : null}
{probe ? (
<ol className="inspector-streams">
{probe?.streams.map((stream) => (
<li key={stream.index}>
@@ -235,7 +319,8 @@ function MediaInspector({ asset }: { asset?: ImportedMediaAsset }) {
{stream.type === 'video' ? (
<>
<small>
Encoded pixels {stream.width ?? '?'}×{stream.height ?? '?'}
Encoded pixels {stream.width ?? '?'}×
{stream.height ?? '?'}
{stream.codedWidth !== undefined &&
stream.codedHeight !== undefined
? ` · coded ${stream.codedWidth}×${stream.codedHeight}`
@@ -281,8 +366,9 @@ function MediaInspector({ asset }: { asset?: ImportedMediaAsset }) {
) : null}
</div>
</li>
)) ?? <li>Waiting for inspection.</li>}
))}
</ol>
) : null}
{probe && probe.chapters.length > 0 ? (
<>
<h3>Source chapters</h3>
@@ -333,8 +419,8 @@ function TransformInspector({
}) {
if (!clip) return <p className="panel-empty">Select a timeline clip.</p>;
const video = asset?.probe?.streams.find((stream) => stream.type === 'video');
const sourceWidth = video?.width;
const sourceHeight = video?.height;
const sourceWidth = video?.width ?? asset?.browserMetadata?.width;
const sourceHeight = video?.height ?? asset?.browserMetadata?.height;
const canShowVisualCrop =
asset !== undefined &&
isPositiveSafeInteger(sourceWidth) &&

64
src/components/MediaBin.tsx
View File

@@ -1,11 +1,19 @@
import type { ImportedMediaAsset } from '../app/application-state';
import { formatBytes, formatDuration } from '../app/application-state';
import {
assetDurationSeconds,
assetHasVideo,
formatBytes,
formatDuration,
} from '../app/application-state';
import { Icon } from './Icon';
export interface MediaBinProps {
assets: readonly ImportedMediaAsset[];
selectedId?: string;
onSelect: (id: string) => void;
onInspect: (id: string) => void;
inspectDisabled?: boolean;
removeDisabled?: boolean;
onRemove: (id: string) => void;
}
@@ -13,6 +21,9 @@ export function MediaBin({
assets,
selectedId,
onSelect,
onInspect,
inspectDisabled = false,
removeDisabled = false,
onRemove,
}: MediaBinProps) {
if (assets.length === 0) {
@@ -27,9 +38,13 @@ export function MediaBin({
return (
<ul className="media-list" aria-label="Imported media">
{assets.map((asset) => {
const hasVideo = assetHasVideo(asset) === true;
const video = asset.probe?.streams.find(
(stream) => stream.type === 'video'
);
const durationSeconds = assetDurationSeconds(asset);
const width = video?.width ?? asset.browserMetadata?.width;
const height = video?.height ?? asset.browserMetadata?.height;
const audioCount =
asset.probe?.streams.filter((stream) => stream.type === 'audio')
.length ?? 0;
@@ -43,34 +58,67 @@ export function MediaBin({
]
.filter(Boolean)
.join(' ')}
aria-pressed={selectedId === asset.id}
onClick={() => onSelect(asset.id)}
>
<span className="media-card__type">
<Icon name={video ? 'video' : 'audio'} />
<Icon name={hasVideo ? 'video' : 'audio'} />
</span>
<span className="media-card__content">
<strong title={asset.file.name}>{asset.file.name}</strong>
<small>
{formatBytes(asset.file.size)}
{asset.probe?.durationSeconds !== undefined
? ` · ${formatDuration(asset.probe.durationSeconds)}`
{durationSeconds !== undefined
? ` · ${formatDuration(durationSeconds)}`
: ''}
</small>
<small className={`phase phase--${asset.phase}`}>
{asset.phase === 'probing'
? 'Inspecting locally…'
? `${asset.browserMetadata ? 'Ready to play' : 'File ready'} · Inspecting details…`
: asset.phase === 'ready'
? `${video ? `${video.width ?? '?'}×${video.height ?? '?'}` : 'Audio'} · ${audioCount} audio`
? `${hasVideo ? `${width ?? '?'}×${height ?? '?'}` : 'Audio'} · ${audioCount} audio`
: asset.phase === 'error'
? asset.error
: 'Queued'}
? `${asset.browserMetadata ? 'Ready to play' : 'File ready'} · Details unavailable`
: asset.browserMetadataError
? 'File ready · Basic metadata unavailable'
: asset.browserMetadata
? 'Ready to play · Details on demand'
: 'File ready · Reading basic metadata…'}
</small>
</span>
</button>
{asset.phase !== 'ready' ? (
<button
className="icon-action"
type="button"
aria-label={`${asset.phase === 'error' ? 'Retry detailed inspection for' : 'Inspect details for'} ${asset.file.name}`}
title={
asset.phase === 'error'
? asset.error
: 'Read codecs, streams, chapters and tags'
}
disabled={inspectDisabled || asset.phase === 'probing'}
{...(asset.error
? { 'aria-describedby': `inspection-error-${asset.id}` }
: {})}
onClick={() => onInspect(asset.id)}
>
<Icon name={asset.phase === 'probing' ? 'queue' : 'info'} />
</button>
) : null}
{asset.error ? (
<span
className="visually-hidden"
id={`inspection-error-${asset.id}`}
>
{asset.error}
</span>
) : null}
<button
className="icon-action"
type="button"
aria-label={`Remove ${asset.file.name}`}
disabled={removeDisabled}
onClick={() => onRemove(asset.id)}
>
<Icon name="trash" />

18
src/components/MediaPreview.tsx
View File

@@ -1,5 +1,6 @@
import { useEffect, useMemo, useRef, useState } from 'react';
import type { ImportedMediaAsset } from '../app/application-state';
import { assetHasAudio, assetHasVideo } from '../app/application-state';
import {
assessBrowserPlayback,
assessGeneratedResultPreview,
@@ -60,18 +61,15 @@ export function MediaPreview({
: undefined;
const hasVideo = result
? result.mimeType.startsWith('video/')
: asset?.probe
? asset.probe.streams.some(
(stream) =>
stream.type === 'video' && stream.disposition.attached_pic !== true
)
: assetHasVideo(asset) !== undefined
? assetHasVideo(asset) === true
: declaredSourceMimeType
? isVideoMimeType(declaredSourceMimeType)
: false;
const hasAudio = result
? result.mimeType.startsWith('audio/')
: asset?.probe
? asset.probe.streams.some((stream) => stream.type === 'audio')
: assetHasAudio(asset) !== undefined
? assetHasAudio(asset) === true
: declaredSourceMimeType
? isAudioMimeType(declaredSourceMimeType)
: false;
@@ -207,8 +205,10 @@ export function MediaPreview({
: `Generated result · ${result.name}`
: (playback?.reason ??
(asset?.phase === 'probing'
? 'Local source preview · Media inspection continues in the background.'
: 'Local source preview'))}
? 'Local source preview · Detailed inspection runs separately.'
: asset?.phase === 'error'
? 'Local source preview · Detailed inspection unavailable.'
: 'Local source preview · Basic browser metadata'))}
</span>
{needsProxy && !proxySettings ? (
<button type="button" className="text-button" onClick={onCreateProxy}>

130
src/components/QuickConvert.tsx
View File

@@ -1,7 +1,7 @@
import { useMemo, useState } from 'react';
import type { ImportedMediaAsset } from '../app/application-state';
import { assetDurationSeconds, assetHasVideo } from '../app/application-state';
import type { EngineState } from '../ffmpeg/ffmpeg.types';
import type { MediaStreamProbe } from '../media';
import type { StreamSelection } from '../commands/command-utils';
import {
validateRemuxCompatibility,
@@ -18,8 +18,12 @@ import {
import { formatBytes, formatDuration } from '../app/application-state';
import { FFMPEG_OPUS_REGRESSION_VERIFIED } from '../version';
import { Icon } from './Icon';
import {
quickStreamUnavailableReason,
type QuickOperation,
} from './quick-convert-selection';
export type QuickOperation = 'convert' | 'remux';
export type { QuickOperation } from './quick-convert-selection';
export interface QuickExportConfiguration {
operation: QuickOperation;
@@ -28,6 +32,11 @@ export interface QuickExportConfiguration {
streamSelection: StreamSelection;
removeMetadata: boolean;
removeChapters: boolean;
/**
* The source was not authoritatively inspected when the user submitted the
* form. Resolve a safe default selection from the on-demand probe.
*/
selectDetectedStreams?: true;
}
export interface QuickConvertProps {
@@ -36,6 +45,7 @@ export interface QuickConvertProps {
busy: boolean;
queueing?: boolean;
presets?: readonly ExportPreset[];
onInspect?: () => void;
onExport: (configuration: QuickExportConfiguration) => void;
}
@@ -45,6 +55,7 @@ export function QuickConvert({
busy,
queueing = false,
presets = BUILT_IN_PRESETS,
onInspect,
onExport,
}: QuickConvertProps) {
const [operation, setOperation] = useState<QuickOperation>('convert');
@@ -59,6 +70,7 @@ export function QuickConvert({
}>();
const preset = presets.find((entry) => entry.id === presetId);
const hasDetailedInformation = asset?.probe !== undefined;
const streams = useMemo(() => asset?.probe?.streams ?? [], [asset?.probe]);
const requestedStreams = useMemo(
() =>
@@ -143,7 +155,9 @@ export function QuickConvert({
const opusBlockedByPinnedCore =
preset?.audio?.codec === 'libopus' && !FFMPEG_OPUS_REGRESSION_VERIFIED;
const presetNeedsMissingAudio =
preset?.kind === 'audio' && effectiveSelection.audio.length === 0;
hasDetailedInformation &&
preset?.kind === 'audio' &&
effectiveSelection.audio.length === 0;
const remuxStreams: readonly RemuxStream[] = streams.flatMap((stream) =>
stream.type === 'video' ||
stream.type === 'audio' ||
@@ -159,7 +173,8 @@ export function QuickConvert({
]
: []
);
const remuxDiagnostics = remuxContainer
const remuxDiagnostics =
remuxContainer && hasDetailedInformation
? validateRemuxCompatibility(
{
streams: remuxStreams,
@@ -179,9 +194,9 @@ export function QuickConvert({
const remuxUnavailableReason = remuxMuxerMissing
? `The loaded core is missing muxer ${remuxContainer}.`
: remuxErrors.map((diagnostic) => diagnostic.message).join(' ');
const hasSelectedStream = selectedStreams.size > 0;
const hasSelectedStream = !hasDetailedInformation || selectedStreams.size > 0;
const canExport =
asset?.phase === 'ready' &&
asset !== undefined &&
!busy &&
hasSelectedStream &&
(operation === 'remux'
@@ -228,27 +243,23 @@ export function QuickConvert({
{asset ? (
<div className="source-summary">
<span className="source-summary__icon">
<Icon
name={
asset.probe?.streams.some((stream) => stream.type === 'video')
? 'video'
: 'audio'
}
/>
<Icon name={assetHasVideo(asset) === true ? 'video' : 'audio'} />
</span>
<div>
<strong>{asset.file.name}</strong>
<span>
{formatBytes(asset.file.size)} ·{' '}
{formatDuration(asset.probe?.durationSeconds)}
{formatDuration(assetDurationSeconds(asset))}
</span>
</div>
<span className={`phase phase--${asset.phase}`}>
{asset.phase === 'ready'
? 'Ready'
? 'Details ready'
: asset.phase === 'probing'
? 'Inspecting details'
: asset.phase === 'error'
? 'Could not inspect'
: 'Preparing'}
? 'Ready to play'
: 'Ready to play'}
</span>
</div>
) : (
@@ -340,14 +351,16 @@ export function QuickConvert({
['ogg', 'Ogg (.ogg)'],
] as const
).map(([container, label]) => {
const diagnostics = validateRemuxCompatibility(
const diagnostics = hasDetailedInformation
? validateRemuxCompatibility(
{
streams: remuxStreams,
hasChapters: Boolean(asset?.probe?.chapters.length),
},
container,
effectiveSelection
);
)
: [];
const missingMuxer =
engineState.status === 'ready' &&
!engineState.capabilities.muxers.has(container);
@@ -410,7 +423,25 @@ export function QuickConvert({
);
})
) : (
<p>Streams appear after local inspection.</p>
<div className="stream-picker__empty">
<p>
Streams will be identified only when you inspect details or
start this export.
</p>
{asset && asset.phase !== 'probing' && onInspect ? (
<button
type="button"
className="button button--secondary"
disabled={busy}
onClick={onInspect}
>
<Icon name="info" />
{asset.phase === 'error'
? 'Retry inspection'
: 'Inspect details now'}
</button>
) : null}
</div>
)}
</fieldset>
@@ -451,6 +482,13 @@ export function QuickConvert({
Select at least one audio stream for an audio-only preset.
</p>
) : null}
{asset?.phase === 'error' && !asset.probe ? (
<p className="notice notice--warning">
<Icon name="warning" />
Playback and basic file information remain available. Detailed
inspection can be retried when an export needs it.
</p>
) : null}
{!hasSelectedStream && streams.length > 0 ? (
<p className="notice notice--error">
<Icon name="warning" />
@@ -474,7 +512,11 @@ export function QuickConvert({
? 'Validated user preset'
: 'Stream copy'}
</strong>
<span>Resource estimates are checked before allocation.</span>
<span>
{hasDetailedInformation
? 'Resource estimates are checked before allocation.'
: 'Detailed stream inspection will run before this export.'}
</span>
</div>
<button
className="button button--primary button--large"
@@ -488,6 +530,9 @@ export function QuickConvert({
streamSelection: effectiveSelection,
removeMetadata,
removeChapters,
...(!hasDetailedInformation
? { selectDetectedStreams: true as const }
: {}),
})
}
>
@@ -497,47 +542,14 @@ export function QuickConvert({
: queueing
? 'Add to queue'
: operation === 'remux'
? hasDetailedInformation
? 'Remux'
: 'Convert'}
: 'Inspect & remux'
: hasDetailedInformation
? 'Convert'
: 'Inspect & convert'}
</button>
</footer>
</section>
);
}
function quickStreamUnavailableReason(
stream: MediaStreamProbe,
operation: QuickOperation,
preset: ExportPreset | undefined
): string | undefined {
if (stream.type === 'unknown') {
return 'unknown stream kinds cannot be mapped safely.';
}
if (operation === 'remux') {
return undefined;
}
if (!preset) {
return 'choose an export preset first.';
}
if (stream.type === 'video' && !preset.video) {
return 'the selected preset creates audio only.';
}
if (stream.type === 'audio' && !preset.audio) {
return 'the selected preset creates video or images without audio.';
}
if (stream.type === 'subtitle') {
if (preset.subtitlePolicy === 'none') {
return 'the selected preset explicitly omits subtitles.';
}
if (preset.subtitlePolicy === 'burn-in') {
return 'burn-in requires an explicitly attached subtitle source.';
}
}
if (
(stream.type === 'attachment' || stream.type === 'data') &&
preset.container !== 'matroska'
) {
return `${stream.type} streams are only preserved by the reviewed Matroska conversion path.`;
}
return undefined;
}

70
src/components/quick-convert-selection.ts Normal file
View File

@@ -0,0 +1,70 @@
import type { StreamSelection } from '../commands/command-utils';
import type { MediaProbe, MediaStreamProbe } from '../media';
import type { ExportPreset } from '../commands/convert';
export type QuickOperation = 'convert' | 'remux';
export function defaultQuickStreamSelection(
probe: MediaProbe,
operation: QuickOperation,
preset: ExportPreset | undefined
): StreamSelection {
const selected = probe.streams.filter(
(stream) =>
quickStreamUnavailableReason(stream, operation, preset) === undefined
);
return {
video: selected
.filter((stream) => stream.type === 'video')
.map((stream) => stream.index),
audio: selected
.filter((stream) => stream.type === 'audio')
.map((stream) => stream.index),
subtitles: selected
.filter((stream) => stream.type === 'subtitle')
.map((stream) => stream.index),
attachments: selected
.filter((stream) => stream.type === 'attachment')
.map((stream) => stream.index),
data: selected
.filter((stream) => stream.type === 'data')
.map((stream) => stream.index),
};
}
export function quickStreamUnavailableReason(
stream: MediaStreamProbe,
operation: QuickOperation,
preset: ExportPreset | undefined
): string | undefined {
if (stream.type === 'unknown') {
return 'unknown stream kinds cannot be mapped safely.';
}
if (operation === 'remux') {
return undefined;
}
if (!preset) {
return 'choose an export preset first.';
}
if (stream.type === 'video' && !preset.video) {
return 'the selected preset creates audio only.';
}
if (stream.type === 'audio' && !preset.audio) {
return 'the selected preset creates video or images without audio.';
}
if (stream.type === 'subtitle') {
if (preset.subtitlePolicy === 'none') {
return 'the selected preset explicitly omits subtitles.';
}
if (preset.subtitlePolicy === 'burn-in') {
return 'burn-in requires an explicitly attached subtitle source.';
}
}
if (
(stream.type === 'attachment' || stream.type === 'data') &&
preset.container !== 'matroska'
) {
return `${stream.type} streams are only preserved by the reviewed Matroska conversion path.`;
}
return undefined;
}

258
src/ffmpeg/EngineManager.ts
View File

@@ -23,7 +23,11 @@ import type {
import { createLoadConfig } from './load-config';
import { LogBuffer } from './log-buffer';
import { normalizeProgress } from './progress-parser';
import { describeMountedInputAccess, mountJobFiles } from './virtual-fs';
import {
describeMountedInputAccess,
mountJobFiles,
type MountedJobFiles,
} from './virtual-fs';
import {
parseFFmpegProgress,
progressUpdateFromFFmpeg,
@@ -71,6 +75,25 @@ export function probeTimeoutMilliseconds(inputBytes: number): number {
);
}
/**
* Firefox can leave this reviewed Emscripten core in a state where a later
* ffprobe call stalls after otherwise successful FFmpeg executions. A fresh
* worker/core completes the same probe promptly. Keep the workaround narrow
* so other browsers retain their warm-core startup path.
*/
export function shouldRefreshCoreBeforeProbe(userAgent?: string): boolean {
const resolvedUserAgent =
userAgent ?? (typeof navigator === 'undefined' ? '' : navigator.userAgent);
return /(?:^|\s)(?:Firefox|FxiOS)\//u.test(resolvedUserAgent);
}
export function canReuseProbeCapabilities(
loadedMode: EngineMode,
cachedMode: EngineMode
): boolean {
return loadedMode === cachedMode;
}
async function defaultFFmpegFactory(): Promise<FFmpegAdapter> {
const { FFmpeg } = await import('@ffmpeg/ffmpeg');
return new FFmpeg();
@@ -98,6 +121,7 @@ export class EngineManager {
#warning: Extract<EngineState, { status: 'ready' }>['warning'];
#loadPromise: Promise<void> | null = null;
#activeJobId: string | null = null;
#preparingOperation = false;
#activeExpectedDurationSeconds: number | undefined;
#activeStartedAtMilliseconds = 0;
#machineProgressLines: string[] = [];
@@ -108,8 +132,14 @@ export class EngineManager {
#generation = 0;
readonly #handleLog = (event: LogEvent): void => {
// Capability discovery emits thousands of individual lines. Forwarding
// each one into React while the engine is still loading can starve
// Firefox's worker message queue long enough for the following probe to
// hit its watchdog. queryCapabilities installs its own short-lived
// listener, so application job logs only need to be retained/published
// while an operation is running.
if (this.#state.status !== 'running') return;
this.#logs.append(event);
if (this.#state.status === 'running') {
for (const line of event.message.split(/\r?\n/gu)) {
if (/^(?:frame|fps|out_time_us|out_time|speed|progress)=/u.test(line)) {
this.#machineProgressLines.push(line);
@@ -138,7 +168,6 @@ export class EngineManager {
});
}
}
}
for (const listener of this.#logListeners) listener(event);
};
@@ -174,10 +203,16 @@ export class EngineManager {
}
setPreference(preference: EnginePreference): void {
if (this.#state.status === 'running') {
if (
this.#activeJobId ||
this.#preparingOperation ||
this.#state.status === 'loading' ||
this.#state.status === 'recovering' ||
this.#state.status === 'running'
) {
throw new EngineError(
'busy',
'Engine mode cannot be changed while a job is running.'
'Engine mode cannot be changed while another operation is active.'
);
}
if (preference !== this.#preference) {
@@ -203,7 +238,11 @@ export class EngineManager {
}
async initialize(): Promise<void> {
if (this.#activeJobId || this.#state.status === 'running') {
if (
this.#activeJobId ||
this.#preparingOperation ||
this.#state.status === 'running'
) {
throw new EngineError(
'busy',
'The FFmpeg engine cannot be initialized while another operation is active.'
@@ -290,7 +329,11 @@ export class EngineManager {
async #loadMode(
mode: EngineMode,
warning?: Extract<EngineState, { status: 'ready' }>['warning']
warning?: Extract<EngineState, { status: 'ready' }>['warning'],
cached?: {
readonly mode: EngineMode;
readonly capabilities: FFmpegCapabilities;
}
): Promise<void> {
const generation = ++this.#generation;
this.#setState({
@@ -317,9 +360,15 @@ export class EngineManager {
status: 'loading',
mode,
progress: 0.75,
stage: 'Inspecting codec capabilities',
stage:
cached && canReuseProbeCapabilities(mode, cached.mode)
? 'Restoring codec capabilities'
: 'Inspecting codec capabilities',
});
const capabilities = await queryCapabilities(engine);
const capabilities =
cached && canReuseProbeCapabilities(mode, cached.mode)
? cached.capabilities
: await queryCapabilities(engine);
if (generation !== this.#generation) return;
this.#capabilities = capabilities;
this.#executionsSinceLoad = 0;
@@ -329,27 +378,54 @@ export class EngineManager {
}
async runJob(request: EngineJobRequest): Promise<EngineJobResult> {
if (this.#activeJobId) {
if (this.#activeJobId || this.#preparingOperation) {
throw new EngineError(
'busy',
'Another FFmpeg operation is already running.'
);
}
await this.initialize();
if (this.#activeJobId || this.#preparingOperation) {
throw new EngineError(
'busy',
'Another FFmpeg operation is already running.'
);
}
let prepared:
| {
readonly engine: FFmpegAdapter;
readonly mode: EngineMode;
readonly mounted: MountedJobFiles;
}
| undefined;
const coreWorkload = request.coreWorkload ?? request.operation;
await this.#recycleExecutionCoreIfNeeded(coreWorkload);
this.#preparingOperation = true;
try {
await this.#recycleExecutionCoreIfNeeded(coreWorkload, true);
const engine = this.#engine;
const mode = this.#mode;
if (!engine || !mode || this.#state.status !== 'ready') {
throw new EngineError('load-failed', 'The FFmpeg engine is not ready.');
}
const mounted = await mountJobFiles(
engine,
request.id,
request.inputs,
request.outputs.map(({ name }) => name)
);
this.#activeJobId = request.id;
prepared = { engine, mode, mounted };
} finally {
this.#preparingOperation = false;
}
if (!prepared) {
throw new EngineError(
'filesystem-failed',
'Could not prepare the FFmpeg job.'
);
}
const { engine, mode, mounted } = prepared;
const generation = this.#generation;
const temporaryPaths = (request.temporaryFiles ?? []).map(
(temporary, index) => {
@@ -358,7 +434,6 @@ export class EngineManager {
return `${mounted.workDirectory}/temporary-${index}.${extension}`;
}
);
this.#activeJobId = request.id;
this.#activeExpectedDurationSeconds = request.expectedDurationSeconds;
this.#activeStartedAtMilliseconds = performance.now();
this.#machineProgressLines = [];
@@ -561,22 +636,79 @@ export class EngineManager {
);
}
if (this.#activeJobId || this.#preparingOperation) {
throw new EngineError(
'busy',
'Another FFmpeg operation is already running.'
);
}
let prepared:
| {
readonly engine: FFmpegAdapter;
readonly mode: EngineMode;
readonly mounted: MountedJobFiles;
}
| undefined;
this.#preparingOperation = true;
try {
await this.#refreshCoreBeforeProbe(signal);
if (signal?.aborted) {
throw new EngineError('cancelled', 'The media probe was cancelled.');
}
const refreshedEngine = this.#engine;
const refreshedMode = this.#mode;
if (
!refreshedEngine ||
!refreshedMode ||
this.#state.status !== 'ready'
) {
throw new EngineError('load-failed', 'The FFmpeg engine is not ready.');
}
const id = crypto.randomUUID();
const mounted = await mountJobFiles(engine, id, [file], ['probe.json']);
const mounted = await mountJobFiles(
refreshedEngine,
id,
[file],
['probe.json']
);
if (signal?.aborted) {
await mounted.cleanup();
throw new EngineError('cancelled', 'The media probe was cancelled.');
}
this.#activeJobId = `probe-${id}`;
prepared = {
engine: refreshedEngine,
mode: refreshedMode,
mounted,
};
} finally {
this.#preparingOperation = false;
}
if (!prepared) {
throw new EngineError(
'filesystem-failed',
'Could not prepare the media probe.'
);
}
const { engine: refreshedEngine, mode: refreshedMode, mounted } = prepared;
const probeOutput = mounted.outputPaths[0];
const inputPath = mounted.inputPaths[0];
if (!probeOutput || !inputPath) {
this.#activeJobId = null;
await mounted.cleanup();
throw new EngineError(
'filesystem-failed',
'Could not allocate probe paths.'
);
}
const activeProbeId = `probe-${id}`;
this.#activeJobId = activeProbeId;
const activeProbeId = this.#activeJobId;
if (!activeProbeId) {
throw new EngineError(
'filesystem-failed',
'Could not reserve the media probe.'
);
}
this.#activeExpectedDurationSeconds = undefined;
this.#activeStartedAtMilliseconds = performance.now();
this.#machineProgressLines = [];
@@ -584,7 +716,7 @@ export class EngineManager {
this.#logs.clear();
this.#setState({
status: 'running',
mode,
mode: refreshedMode,
jobId: this.#activeJobId,
operation: 'Inspect media',
inputMode: mounted.inputMode,
@@ -605,7 +737,7 @@ export class EngineManager {
};
signal?.addEventListener('abort', cancelOnAbort, { once: true });
try {
const exitCode = await engine.ffprobe([
const exitCode = await refreshedEngine.ffprobe([
'-v',
'error',
'-show_format',
@@ -620,7 +752,7 @@ export class EngineManager {
let parsed: unknown;
try {
parsed = JSON.parse(
textFromFileData(await engine.readFile(probeOutput))
textFromFileData(await refreshedEngine.readFile(probeOutput))
) as unknown;
} catch (readError) {
throw new EngineError(
@@ -691,16 +823,19 @@ export class EngineManager {
this.#activeJobId = null;
this.#machineProgressLines = [];
this.#machineProgressSeen = false;
if (this.#engine === engine) {
if (this.#engine === refreshedEngine) {
const cleanupSucceeded = await mounted.cleanup();
if (!cleanupSucceeded && this.#engine === engine) {
if (!cleanupSucceeded && this.#engine === refreshedEngine) {
this.#logs.append({
type: 'stderr',
message:
'Probe filesystem cleanup could not be verified; recreating the FFmpeg engine.',
});
this.#terminateEngine();
this.#setState({ status: 'recovering', previousMode: mode });
this.#setState({
status: 'recovering',
previousMode: refreshedMode,
});
try {
await this.initialize();
} catch {
@@ -711,7 +846,7 @@ export class EngineManager {
if (capabilities) {
this.#setState({
status: 'ready',
mode,
mode: refreshedMode,
capabilities,
warning: this.#warning,
});
@@ -721,6 +856,71 @@ export class EngineManager {
}
}
async #refreshCoreBeforeProbe(signal?: AbortSignal): Promise<void> {
if (!shouldRefreshCoreBeforeProbe()) return;
if (signal?.aborted) {
throw new EngineError('cancelled', 'The media probe was cancelled.');
}
const mode = this.#mode;
const capabilities = this.#capabilities;
const warning = this.#warning;
if (!mode || !capabilities || this.#state.status !== 'ready') {
throw new EngineError('load-failed', 'The FFmpeg engine is not ready.');
}
this.#terminateEngine();
this.#setState({ status: 'recovering', previousMode: mode, warning });
try {
await this.#loadMode(mode, warning, { mode, capabilities });
} catch (firstError) {
if (mode !== 'multithread') {
this.#terminateEngine();
const loadError = new EngineError(
'load-failed',
'The FFmpeg core could not be refreshed for media inspection.',
{
cause: firstError,
details: describeInitializationFailure(firstError),
}
);
this.#setState({ status: 'error', error: loadError });
throw loadError;
}
this.#terminateEngine();
const fallbackWarning = {
code: 'multithread-load-failed' as const,
message:
'The multithread core failed to reload. The app recovered with the single-thread compatibility core.',
};
this.#setState({
status: 'recovering',
previousMode: 'multithread',
warning: fallbackWarning,
});
try {
// Capabilities are mode-specific; discover a fresh single-thread set.
await this.#loadMode('single-thread', fallbackWarning);
} catch (fallbackError) {
const details = [
`Multithread core: ${describeInitializationFailure(firstError)}`,
`Single-thread core: ${describeInitializationFailure(fallbackError)}`,
].join('\n');
this.#terminateEngine();
const loadError = new EngineError(
'load-failed',
'Neither FFmpeg execution core could be refreshed for media inspection.',
{ cause: fallbackError, details }
);
this.#setState({ status: 'error', error: loadError });
throw loadError;
}
}
if (signal?.aborted) {
throw new EngineError('cancelled', 'The media probe was cancelled.');
}
}
async cancelActive(): Promise<void> {
const activeJobId = this.#activeJobId;
const previousMode = this.#mode;
@@ -760,7 +960,10 @@ export class EngineManager {
this.#contactSheetsSinceLoad = 0;
}
async #recycleExecutionCoreIfNeeded(nextOperation: string): Promise<void> {
async #recycleExecutionCoreIfNeeded(
nextOperation: string,
operationReserved = false
): Promise<void> {
if (
!shouldRecycleExecutionCore(
this.#executionsSinceLoad,
@@ -789,8 +992,15 @@ export class EngineManager {
});
this.#terminateEngine();
this.#setState({ status: 'recovering', previousMode });
if (operationReserved) {
this.#loadPromise = this.#initializeInternal().finally(() => {
this.#loadPromise = null;
});
await this.#loadPromise;
} else {
await this.initialize();
}
}
#setState(state: EngineState): void {
this.#state = state;

27
src/index.css
View File

@@ -561,12 +561,16 @@ select:disabled {
color: #2c7d63 !important;
}
.phase--idle {
color: #2c7d63 !important;
}
.phase--probing {
color: #8b6a20 !important;
}
.phase--error {
color: var(--toolbox-danger) !important;
color: #8b6a20 !important;
}
.media-bin__empty {
@@ -779,6 +783,18 @@ select:disabled {
font-size: 0.68rem;
}
.stream-picker__empty {
display: flex;
flex-wrap: wrap;
align-items: center;
justify-content: space-between;
gap: 0.75rem;
}
.stream-picker__empty .button {
flex: none;
}
.stream-kind {
display: inline-flex;
align-items: center;
@@ -1759,6 +1775,15 @@ progress::-moz-progress-bar {
padding: 0.8rem;
}
.inspector-section > .notice {
align-items: center;
margin: 0;
}
.inspector-section > .notice > div {
flex: 1;
}
.inspector-section h3 {
margin: 0.45rem 0 0;
}

282
src/media/browser-metadata.ts Normal file
View File

@@ -0,0 +1,282 @@
import {
isAudioMimeType,
isVideoMimeType,
mimeTypeFromFileName,
} from './mime-types';
export type BrowserMediaKind = 'audio' | 'video' | 'unknown';
export type BrowserMetadataErrorCode = 'aborted' | 'load-error' | 'timeout';
export interface BrowserMediaFileInfo {
readonly name: string;
readonly sizeBytes: number;
readonly declaredMimeType?: string;
readonly inferredMimeType?: string;
readonly lastModified: number;
}
/**
* Lightweight metadata exposed by the browser's native media element.
*
* This intentionally does not pretend to replace ffprobe: codec names,
* complete stream lists, chapters, tags and dispositions remain unknown.
*/
export interface BrowserMediaMetadata {
readonly source: 'browser';
readonly file: BrowserMediaFileInfo;
readonly kind: BrowserMediaKind;
readonly durationSeconds?: number;
readonly width?: number;
readonly height?: number;
readonly hasAudio?: boolean;
readonly hasVideo?: boolean;
}
export interface BrowserMetadataOptions {
readonly signal?: AbortSignal;
readonly timeoutMilliseconds?: number;
/**
* Test seam for the DOM-owned media element. Production callers normally
* leave this unset.
*/
readonly createMediaElement?: (
kind: Exclude<BrowserMediaKind, 'unknown'>
) => HTMLMediaElement;
}
export class BrowserMetadataError extends Error {
readonly code: BrowserMetadataErrorCode;
constructor(code: BrowserMetadataErrorCode, message: string) {
super(message);
this.name = 'BrowserMetadataError';
this.code = code;
}
}
const DEFAULT_BROWSER_METADATA_TIMEOUT_MILLISECONDS = 15_000;
interface MediaElementWithAudioHints extends HTMLMediaElement {
readonly audioTracks?: { readonly length: number };
readonly mozHasAudio?: boolean;
}
export function browserMediaFileInfo(file: File): BrowserMediaFileInfo {
return {
name: file.name,
sizeBytes: file.size,
declaredMimeType: normalizeMimeType(file.type),
inferredMimeType: mimeTypeFromFileName(file.name),
lastModified: file.lastModified,
};
}
export function inferBrowserMediaKind(file: File): BrowserMediaKind {
const declaredMimeType = normalizeMimeType(file.type);
const inferredMimeType = mimeTypeFromFileName(file.name);
const mimeType =
declaredMimeType !== undefined &&
(isAudioMimeType(declaredMimeType) || isVideoMimeType(declaredMimeType))
? declaredMimeType
: inferredMimeType;
if (mimeType === undefined) return 'unknown';
if (isVideoMimeType(mimeType)) return 'video';
if (isAudioMimeType(mimeType)) return 'audio';
return 'unknown';
}
/**
* Reads only metadata the browser can expose without FFmpeg. The supplied
* object URL remains owned by the caller and is never revoked here.
*/
export function inspectBrowserMediaMetadata(
file: File,
objectUrl: string,
options: BrowserMetadataOptions = {}
): Promise<BrowserMediaMetadata> {
const declaredKind = inferBrowserMediaKind(file);
const elementKind = declaredKind === 'audio' ? 'audio' : 'video';
const createMediaElement =
options.createMediaElement ??
((kind: 'audio' | 'video') => document.createElement(kind));
const element = createMediaElement(elementKind);
const timeoutMilliseconds =
options.timeoutMilliseconds ??
DEFAULT_BROWSER_METADATA_TIMEOUT_MILLISECONDS;
if (options.signal?.aborted === true) {
return Promise.reject(abortedError());
}
return new Promise((resolve, reject) => {
let settled = false;
const cleanup = () => {
clearTimeout(timeoutId);
element.removeEventListener('loadedmetadata', handleLoadedMetadata);
element.removeEventListener('error', handleError);
element.removeEventListener('abort', handleElementAbort);
options.signal?.removeEventListener('abort', handleSignalAbort);
element.removeAttribute('src');
try {
element.load();
} catch {
// The caller owns the object URL; releasing this detached element is
// best-effort and must not turn successful metadata into a failure.
}
};
const finish = (
outcome:
| { readonly ok: true; readonly value: BrowserMediaMetadata }
| { readonly ok: false; readonly error: BrowserMetadataError }
) => {
if (settled) return;
settled = true;
cleanup();
if (outcome.ok) resolve(outcome.value);
else reject(outcome.error);
};
const handleLoadedMetadata = () => {
finish({
ok: true,
value: metadataFromElement(file, declaredKind, element),
});
};
const handleError = () => {
finish({
ok: false,
error: new BrowserMetadataError(
'load-error',
`The browser could not read metadata from ${file.name}.`
),
});
};
const handleElementAbort = () => {
finish({
ok: false,
error: new BrowserMetadataError(
'load-error',
`The browser stopped reading metadata from ${file.name}.`
),
});
};
const handleSignalAbort = () => {
finish({ ok: false, error: abortedError() });
};
element.addEventListener('loadedmetadata', handleLoadedMetadata);
element.addEventListener('error', handleError);
element.addEventListener('abort', handleElementAbort);
options.signal?.addEventListener('abort', handleSignalAbort, {
once: true,
});
const timeoutId = setTimeout(
() => {
finish({
ok: false,
error: new BrowserMetadataError(
'timeout',
`Browser metadata loading exceeded ${String(
Math.ceil(timeoutMilliseconds / 1_000)
)} seconds.`
),
});
},
Math.max(0, timeoutMilliseconds)
);
element.preload = 'metadata';
element.src = objectUrl;
try {
element.load();
} catch {
finish({
ok: false,
error: new BrowserMetadataError(
'load-error',
`The browser could not start reading metadata from ${file.name}.`
),
});
}
});
}
function metadataFromElement(
file: File,
declaredKind: BrowserMediaKind,
element: HTMLMediaElement
): BrowserMediaMetadata {
const isVideoElement = element instanceof HTMLVideoElement;
const width = isVideoElement
? positiveFiniteNumber(element.videoWidth)
: undefined;
const height = isVideoElement
? positiveFiniteNumber(element.videoHeight)
: undefined;
// A zero-sized <video> can still be the correct, playable element for an
// audio-only MP4 or another video container. Treat missing dimensions as
// inconclusive instead of claiming that no video-style player is usable.
const hasVideo = isVideoElement
? width !== undefined
? true
: undefined
: false;
const hasAudio = audioPresence(element, isVideoElement);
const kind =
hasVideo === true
? 'video'
: hasAudio === true || element instanceof HTMLAudioElement
? 'audio'
: declaredKind;
return {
source: 'browser',
file: browserMediaFileInfo(file),
kind,
durationSeconds: nonNegativeFiniteNumber(element.duration),
width,
height,
hasAudio,
hasVideo,
};
}
function audioPresence(
element: HTMLMediaElement,
isVideoElement: boolean
): boolean | undefined {
if (!isVideoElement) return true;
const withAudioHints = element as MediaElementWithAudioHints;
if (withAudioHints.audioTracks !== undefined) {
return withAudioHints.audioTracks.length > 0;
}
if (typeof withAudioHints.mozHasAudio === 'boolean') {
return withAudioHints.mozHasAudio;
}
return undefined;
}
function normalizeMimeType(mimeType: string): string | undefined {
const normalized = mimeType.trim().toLowerCase().split(';')[0]?.trim();
return normalized === undefined || normalized.length === 0
? undefined
: normalized;
}
function positiveFiniteNumber(value: number): number | undefined {
return Number.isFinite(value) && value > 0 ? value : undefined;
}
function nonNegativeFiniteNumber(value: number): number | undefined {
return Number.isFinite(value) && value >= 0 ? value : undefined;
}
function abortedError(): BrowserMetadataError {
return new BrowserMetadataError(
'aborted',
'Browser metadata loading was cancelled.'
);
}

1
src/media/index.ts
View File

@@ -1,4 +1,5 @@
export * from './browser-playback';
export * from './browser-metadata';
export * from './chapter-time-base';
export * from './duration';
export * from './media.types';

33
tests/browser/application.spec.ts
View File

@@ -1,6 +1,7 @@
import { expect, test } from '@playwright/test';
import {
observeRuntime,
PRIMARY_FIXTURE,
startNestedStaticServer,
type NestedStaticServer,
} from './browser-helpers';
@@ -119,6 +120,38 @@ test.describe('static application and Toolbox shell', () => {
await expect(inspectorDrawer).toBeHidden();
await expect(inspector).toBeFocused();
});
test('shows native preview metadata without loading the FFmpeg runtime', async ({
page,
}) => {
const runtime = observeRuntime(page);
await page.goto('/');
await page
.locator('input[type="file"][accept*="audio"]')
.first()
.setInputFiles(PRIMARY_FIXTURE);
await expect(page.locator('.quick-preview video')).toBeVisible();
const card = page
.locator('.media-card')
.filter({ hasText: 'pattern-av.mp4' });
await expect(card.locator('.phase--idle')).toContainText(
'Ready to play · Details on demand'
);
await expect(card).toContainText(/\d+:\d{2}\.\d{2}/u);
expect(
runtime.requests.some((url) => url.includes('/vendor/ffmpeg/'))
).toBe(false);
await page.getByRole('tab', { name: 'Edit' }).click();
await expect(
page.getByRole('button', { name: 'Inspect & render timeline' })
).toBeEnabled();
expect(
runtime.requests.some((url) => url.includes('/vendor/ffmpeg/'))
).toBe(false);
});
});
test.describe('nested static deployment and Toolbox context', () => {

15
tests/browser/browser-helpers.ts
View File

@@ -7,6 +7,7 @@ import { tmpdir } from 'node:os';
import { dirname, extname, join, resolve, sep } from 'node:path';
import { fileURLToPath } from 'node:url';
import type { ConsoleMessage, Page, Request, Response } from '@playwright/test';
import { expect } from '@playwright/test';
export const REPOSITORY_ROOT = resolve(
dirname(fileURLToPath(import.meta.url)),
@@ -19,6 +20,20 @@ export const GENERATED_FIXTURES = join(
);
export const PRIMARY_FIXTURE = join(GENERATED_FIXTURES, 'pattern-av.mp4');
export async function inspectMediaDetails(
page: Page,
fileName: string
): Promise<void> {
const card = page.locator('.media-card').filter({ hasText: fileName });
const inspect = page.getByRole('button', {
name: `Inspect details for ${fileName}`,
});
if ((await inspect.count()) > 0) await inspect.click();
await expect(card.locator('.phase--ready')).toBeVisible({
timeout: 120_000,
});
}
const NESTED_PREFIX = '/deep/nested/av/';
const SECURITY_HEADERS = {
'Cross-Origin-Embedder-Policy': 'require-corp',

24
tests/browser/ffmpeg-runtime.spec.ts
View File

@@ -6,6 +6,7 @@ import { expect, test } from '@playwright/test';
import {
createCancellationFixture,
GENERATED_FIXTURES,
inspectMediaDetails,
observeRuntime,
PRIMARY_FIXTURE,
type TemporaryMediaFixture,
@@ -99,8 +100,9 @@ test.describe.serial('real pinned FFmpeg browser runtime', () => {
).toBe(false);
await mediaInput(page).setInputFiles(PRIMARY_FIXTURE);
await inspectMediaDetails(page, 'pattern-av.mp4');
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 120_000 }
);
await page.getByRole('button', { name: 'Convert', exact: true }).click();
@@ -184,8 +186,9 @@ test.describe.serial('real pinned FFmpeg browser runtime', () => {
).toBe(false);
await mediaInput(page).setInputFiles(PRIMARY_FIXTURE);
await inspectMediaDetails(page, 'pattern-av.mp4');
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 120_000 }
);
await expect(page.locator('.source-summary')).toContainText(
@@ -275,6 +278,7 @@ test.describe.serial('real pinned FFmpeg browser runtime', () => {
await mediaInput(page)
.first()
.setInputFiles(join(GENERATED_FIXTURES, 'compatible-a.mp4'));
await inspectMediaDetails(page, 'compatible-a.mp4');
await expect(
page
.locator('.media-card')
@@ -329,8 +333,9 @@ test.describe.serial('real pinned FFmpeg browser runtime', () => {
await page.goto('/');
await selectSingleThread(page);
await mediaInput(page).setInputFiles(cancellationFixture.filePath);
await inspectMediaDetails(page, basename(cancellationFixture.filePath));
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 120_000 }
);
@@ -357,8 +362,9 @@ test.describe.serial('real pinned FFmpeg browser runtime', () => {
})
.click();
await mediaInput(page).setInputFiles(PRIMARY_FIXTURE);
await inspectMediaDetails(page, 'pattern-av.mp4');
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 120_000 }
);
await page.getByRole('button', { name: 'Convert', exact: true }).click();
@@ -388,10 +394,15 @@ test.describe.serial('real pinned FFmpeg browser runtime', () => {
const queuedCard = page.locator('.media-card').filter({
hasText: 'pattern-av.mp4',
});
await page
.getByRole('button', {
name: `Inspect details for ${activeName}`,
})
.click();
await expect(activeCard.locator('.phase--probing')).toBeVisible({
timeout: 30_000,
});
await expect(queuedCard.locator('.phase--queued')).toBeVisible();
await expect(queuedCard.locator('.phase--idle')).toBeVisible();
await page.getByRole('button', { name: 'Remove pattern-av.mp4' }).click();
await page.getByRole('button', { name: `Remove ${activeName}` }).click();
@@ -401,8 +412,9 @@ test.describe.serial('real pinned FFmpeg browser runtime', () => {
).toBeEnabled({ timeout: 120_000 });
await mediaInput(page).setInputFiles(PRIMARY_FIXTURE);
await inspectMediaDetails(page, 'pattern-av.mp4');
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 120_000 }
);
await expect(page.getByRole('alert')).toHaveCount(0);

17
tests/browser/fixture-recovery.spec.ts
View File

@@ -4,6 +4,7 @@ import { join } from 'node:path';
import { expect, test, type Page } from '@playwright/test';
import {
GENERATED_FIXTURES,
inspectMediaDetails,
observeRuntime,
PRIMARY_FIXTURE,
type RuntimeObservation,
@@ -51,8 +52,9 @@ test.describe
await mediaInput(page).setInputFiles(
join(GENERATED_FIXTURES, 'attached-cover.mp3')
);
await inspectMediaDetails(page, 'attached-cover.mp3');
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 60_000 }
);
await expect(page.locator('.source-summary')).toContainText(
@@ -108,13 +110,23 @@ test.describe
await mediaInput(page).setInputFiles(
join(GENERATED_FIXTURES, 'truncated.mp4')
);
await page
.getByRole('button', {
name: 'Inspect details for truncated.mp4',
})
.click();
const malformedCard = page
.locator('.media-card')
.filter({ hasText: 'truncated.mp4' });
await expect(malformedCard.locator('.phase--error')).toContainText(
'ffprobe did not find any media streams.',
'File ready · Details unavailable',
{ timeout: 30_000 }
);
await expect(
page.getByRole('button', {
name: 'Retry detailed inspection for truncated.mp4',
})
).toHaveAttribute('title', 'ffprobe did not find any media streams.');
expect(Date.now() - probeStartedAt).toBeLessThan(30_000);
await expect(
page
@@ -123,6 +135,7 @@ test.describe
).toBeVisible();
await mediaInput(page).setInputFiles(PRIMARY_FIXTURE);
await inspectMediaDetails(page, 'pattern-av.mp4');
const validCard = page
.locator('.media-card')
.filter({ hasText: 'pattern-av.mp4' });

20
tests/browser/large-media-preview.spec.ts
View File

@@ -6,7 +6,7 @@ import { expect, test } from '@playwright/test';
const mediaPath = process.env.AV_TOOLS_LARGE_MEDIA;
test('plays a recognized large local video while inspection continues', async ({
test('plays a recognized large local video before detailed inspection', async ({
page,
}) => {
test.skip(
@@ -27,7 +27,15 @@ test('plays a recognized large local video while inspection continues', async ({
await expect(preview).toBeVisible({ timeout: 15_000 });
const previewVisibleMilliseconds = Date.now() - startedAt;
expect(previewVisibleMilliseconds).toBeLessThan(15_000);
await expect(card.locator('.phase--probing')).toBeVisible();
await expect(card.locator('.phase--idle')).toContainText('Details on demand');
await expect(card).toContainText(/43:42\.\d{2}/u, { timeout: 15_000 });
expect(
await page.evaluate(() =>
performance
.getEntriesByType('resource')
.some((entry) => entry.name.includes('/vendor/ffmpeg/'))
)
).toBe(false);
const elementIdentity = await preview.evaluate((element) => {
element.dataset.evidenceIdentity = crypto.randomUUID();
@@ -50,9 +58,11 @@ test('plays a recognized large local video while inspection continues', async ({
)
.toBeGreaterThan(0);
await expect(card.locator('.phase--ready')).toBeVisible({
timeout: 330_000,
});
await page
.getByRole('button', { name: `Inspect details for ${name}` })
.click();
await expect(card.locator('.phase--probing')).toBeVisible();
await expect(card.locator('.phase--ready')).toBeVisible({ timeout: 120_000 });
await expect(preview).toHaveAttribute(
'data-evidence-identity',
elementIdentity

13
tests/browser/operation-matrix.spec.ts
View File

@@ -31,6 +31,9 @@ async function importMedia(
await mediaInput(page).setInputFiles(names.map((name) => fixture(name)));
for (const name of names) {
const card = page.locator('.media-card').filter({ hasText: name });
await page
.getByRole('button', { name: `Inspect details for ${name}` })
.click();
await expect(card.locator('.phase--ready')).toBeVisible({
timeout: 120_000,
});
@@ -210,6 +213,11 @@ test.describe.serial('real ffmpeg.wasm operation matrix', () => {
await test.step('queues a second browser job, cancels the active job and runs the waiting job after recovery', async () => {
const sourceName = basename(cancellationFixture.filePath);
await mediaInput(page).setInputFiles(cancellationFixture.filePath);
await page
.getByRole('button', {
name: `Inspect details for ${sourceName}`,
})
.click();
await expect(
page
.locator('.media-card')
@@ -1055,6 +1063,11 @@ test.describe.serial('real ffmpeg.wasm operation matrix', () => {
mimeType: 'video/x-matroska',
buffer: readFileSync(muxed.filePath),
});
await page
.getByRole('button', {
name: 'Inspect details for subtitled-source.mkv',
})
.click();
await expect(
page
.locator('.media-card')

3
tests/browser/performance.spec.ts
View File

@@ -103,8 +103,9 @@ async function runMode(
const probeStarted = await monotonicNow(page);
await mediaInput(page).setInputFiles(fixturePath);
await page.getByRole('button', { name: /^Inspect details for /u }).click();
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 120_000 }
);
const probeMs = (await monotonicNow(page)) - probeStarted;

14
tests/browser/ui-workflows.spec.ts
View File

@@ -21,8 +21,11 @@ test('covers drop, editor keyboard/drag controls, cache, and project reattachmen
await page.getByLabel('Engine mode').selectOption('force-single-thread');
await dropFixture(page, PRIMARY_FIXTURE, 'video/mp4');
await page
.getByRole('button', { name: 'Inspect details for pattern-av.mp4' })
.click();
await expect(page.locator('.source-summary .phase--ready')).toHaveText(
'Ready',
'Details ready',
{ timeout: 120_000 }
);
await expect(page.locator('.quick-preview video')).toHaveAttribute(
@@ -47,6 +50,9 @@ test('covers drop, editor keyboard/drag controls, cache, and project reattachmen
await page.getByRole('tab', { name: 'Edit' }).click();
const compatibleB = join(GENERATED_FIXTURES, 'compatible-b.mp4');
await mediaInput(page).setInputFiles(compatibleB);
await page
.getByRole('button', { name: 'Inspect details for compatible-b.mp4' })
.click();
await expect(page.locator('.media-card .phase--ready')).toHaveCount(2, {
timeout: 120_000,
});
@@ -127,6 +133,12 @@ test('covers drop, editor keyboard/drag controls, cache, and project reattachmen
page.on('dialog', (dialog) => void dialog.accept());
await mediaInput(page).setInputFiles([PRIMARY_FIXTURE, compatibleB]);
await page
.getByRole('button', { name: 'Inspect details for pattern-av.mp4' })
.click();
await page
.getByRole('button', { name: 'Inspect details for compatible-b.mp4' })
.click();
await expect(page.locator('.media-card .phase--ready')).toHaveCount(2, {
timeout: 120_000,
});

93
tests/unit/components/media-bin.test.tsx Normal file
View File

@@ -0,0 +1,93 @@
import { cleanup, render, screen } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { afterEach, describe, expect, it, vi } from 'vitest';
import type { ImportedMediaAsset } from '../../../src/app/application-state';
import { MediaBin } from '../../../src/components/MediaBin';
afterEach(cleanup);
describe('MediaBin', () => {
it('shows browser metadata and offers optional detailed inspection', async () => {
const user = userEvent.setup();
const onInspect = vi.fn();
const file = new File(['video'], 'lecture.mp4', { type: 'video/mp4' });
const asset: ImportedMediaAsset = {
id: 'lecture',
file,
objectUrl: 'blob:lecture',
phase: 'idle',
browserMetadata: {
source: 'browser',
file: {
name: file.name,
sizeBytes: file.size,
declaredMimeType: file.type,
inferredMimeType: file.type,
lastModified: file.lastModified,
},
kind: 'video',
durationSeconds: 2_622.48,
width: 640,
height: 480,
hasVideo: true,
},
};
render(
<MediaBin
assets={[asset]}
selectedId={asset.id}
onSelect={vi.fn()}
onInspect={onInspect}
onRemove={vi.fn()}
/>
);
expect(screen.getByText(/43:42\.48/u)).toBeVisible();
expect(screen.getByText('Ready to play · Details on demand')).toBeVisible();
expect(screen.getByText('lecture.mp4').closest('button')).toHaveAttribute(
'aria-pressed',
'true'
);
await user.click(
screen.getByRole('button', {
name: 'Inspect details for lecture.mp4',
})
);
expect(onInspect).toHaveBeenCalledWith(asset.id);
});
it('keeps a source ready to play after detailed inspection fails', () => {
const file = new File(['video'], 'lecture.mp4', { type: 'video/mp4' });
const asset: ImportedMediaAsset = {
id: 'lecture',
file,
objectUrl: 'blob:lecture',
phase: 'error',
error: 'Media inspection exceeded its safety limit.',
};
render(
<MediaBin
assets={[asset]}
onSelect={vi.fn()}
onInspect={vi.fn()}
onRemove={vi.fn()}
/>
);
expect(screen.getByText('File ready · Details unavailable')).toBeVisible();
expect(
screen.getByRole('button', {
name: 'Retry detailed inspection for lecture.mp4',
})
).toBeEnabled();
expect(
screen.getByRole('button', {
name: 'Retry detailed inspection for lecture.mp4',
})
).toHaveAccessibleDescription(
'Media inspection exceeded its safety limit.'
);
});
});

60
tests/unit/components/media-preview.test.tsx
View File

@@ -43,7 +43,7 @@ describe('MediaPreview', () => {
expect(progressiveVideo).toHaveAttribute('src', probingAsset.objectUrl);
expect(
screen.getByText(
'Local source preview · Media inspection continues in the background.'
'Local source preview · Detailed inspection runs separately.'
)
).toBeVisible();
@@ -61,6 +61,64 @@ describe('MediaPreview', () => {
expect(container.querySelector('video')).toBe(progressiveVideo);
});
it('keeps browser playback mounted after detailed inspection fails', () => {
const asset: ImportedMediaAsset = {
id: 'asset-probe-error',
file: new File(['video'], 'lecture.mp4', { type: 'video/mp4' }),
objectUrl: 'blob:probe-error',
phase: 'error',
error: 'Detailed inspection timed out.',
};
const { container } = render(<MediaPreview asset={asset} />);
expect(container.querySelector('video')).toHaveAttribute(
'src',
asset.objectUrl
);
expect(
screen.getByText(
'Local source preview · Detailed inspection unavailable.'
)
).toBeVisible();
});
it('keeps the video-container player when browser metadata has no track hints', () => {
const file = new File(['audio-only'], 'audio-only.mp4', {
type: 'video/mp4',
});
const initial: ImportedMediaAsset = {
id: 'asset-audio-only-container',
file,
objectUrl: 'blob:audio-only-container',
phase: 'idle',
};
const { container, rerender } = render(<MediaPreview asset={initial} />);
const player = requiredElement(container.querySelector('video'));
rerender(
<MediaPreview
asset={{
...initial,
browserMetadata: {
source: 'browser',
file: {
name: file.name,
sizeBytes: file.size,
declaredMimeType: file.type,
inferredMimeType: file.type,
lastModified: file.lastModified,
},
kind: 'video',
durationSeconds: 8,
},
}}
/>
);
expect(container.querySelector('video')).toBe(player);
});
it('offers bounded configurable proxy settings after playback fails', async () => {
const user = userEvent.setup();
const onCreateProxy = vi.fn();

60
tests/unit/components/quick-convert.test.tsx
View File

@@ -16,6 +16,66 @@ import type { UserExportPreset } from '../../../src/presets';
afterEach(cleanup);
describe('QuickConvert', () => {
it('keeps an uninspected source usable and requests stream detection with export', async () => {
const user = userEvent.setup();
const onInspect = vi.fn();
const onExport = vi.fn<(configuration: QuickExportConfiguration) => void>();
const file = new File(['media'], 'lecture.mp4', { type: 'video/mp4' });
const asset: ImportedMediaAsset = {
id: 'asset-basic',
file,
objectUrl: 'blob:asset-basic',
phase: 'idle',
browserMetadata: {
source: 'browser',
file: {
name: file.name,
sizeBytes: file.size,
declaredMimeType: file.type,
inferredMimeType: file.type,
lastModified: file.lastModified,
},
kind: 'video',
durationSeconds: 2_622.48,
width: 640,
height: 480,
hasVideo: true,
},
};
render(
<QuickConvert
asset={asset}
engineState={readyEngine()}
busy={false}
onInspect={onInspect}
onExport={onExport}
/>
);
expect(screen.getByText(/43:42\.48/u)).toBeVisible();
expect(screen.getByText('Ready to play')).toBeVisible();
await user.click(
screen.getByRole('button', { name: 'Inspect details now' })
);
expect(onInspect).toHaveBeenCalledOnce();
await user.click(screen.getByRole('button', { name: 'Inspect & convert' }));
expect(onExport).toHaveBeenCalledWith(
expect.objectContaining({
operation: 'convert',
selectDetectedStreams: true,
streamSelection: {
video: [],
audio: [],
subtitles: [],
attachments: [],
data: [],
},
})
);
});
it('labels a submission honestly when another operation is active', () => {
render(
<QuickConvert

36
tests/unit/ffmpeg-engine-arguments.test.ts
View File

@@ -7,9 +7,11 @@ import {
MAX_PROBE_TIMEOUT_MILLISECONDS,
MIN_PROBE_TIMEOUT_MILLISECONDS,
MULTITHREAD_CODEC_THREAD_LIMIT,
canReuseProbeCapabilities,
engineJobTimeoutMilliseconds,
prepareEngineArguments,
probeTimeoutMilliseconds,
shouldRefreshCoreBeforeProbe,
shouldRecycleExecutionCore,
} from '../../src/ffmpeg/EngineManager';
@@ -127,6 +129,40 @@ describe('probeTimeoutMilliseconds', () => {
});
});
describe('Firefox probe core refresh policy', () => {
it('refreshes only affected Firefox-family runtimes', () => {
expect(
shouldRefreshCoreBeforeProbe(
'Mozilla/5.0 (X11; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0'
)
).toBe(true);
expect(
shouldRefreshCoreBeforeProbe(
'Mozilla/5.0 (iPhone; CPU iPhone OS 18_6 like Mac OS X) FxiOS/142.0'
)
).toBe(true);
expect(
shouldRefreshCoreBeforeProbe(
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chrome/140.0 Safari/537.36'
)
).toBe(false);
expect(shouldRefreshCoreBeforeProbe('')).toBe(false);
});
it('reuses cached capabilities only for the identical engine mode', () => {
expect(canReuseProbeCapabilities('multithread', 'multithread')).toBe(true);
expect(canReuseProbeCapabilities('single-thread', 'single-thread')).toBe(
true
);
expect(canReuseProbeCapabilities('multithread', 'single-thread')).toBe(
false
);
expect(canReuseProbeCapabilities('single-thread', 'multithread')).toBe(
false
);
});
});
describe('execution core recycling policy', () => {
it('refreshes only after the bounded idle-core job budget is exhausted', () => {
expect(shouldRecycleExecutionCore(MAX_EXECUTIONS_PER_ENGINE - 1)).toBe(

187
tests/unit/media/browser-metadata.test.ts Normal file
View File

@@ -0,0 +1,187 @@
import { afterEach, describe, expect, it, vi } from 'vitest';
import {
BrowserMetadataError,
browserMediaFileInfo,
inferBrowserMediaKind,
inspectBrowserMediaMetadata,
} from '../../../src/media/browser-metadata';
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});
describe('browser media metadata', () => {
it('returns file information synchronously and infers useful media kinds', () => {
const file = new File(['media'], 'lecture.MP4', {
type: 'application/octet-stream',
lastModified: 123,
});
expect(browserMediaFileInfo(file)).toEqual({
name: 'lecture.MP4',
sizeBytes: 5,
declaredMimeType: 'application/octet-stream',
inferredMimeType: 'video/mp4',
lastModified: 123,
});
expect(inferBrowserMediaKind(file)).toBe('video');
expect(
inferBrowserMediaKind(
new File([], 'recording.bin', { type: 'audio/ogg; codecs=opus' })
)
).toBe('audio');
expect(inferBrowserMediaKind(new File([], 'notes.txt'))).toBe('unknown');
});
it('loads duration and dimensions through a native video element', async () => {
const file = new File(['video'], 'lecture.mp4', { type: 'video/mp4' });
const video = document.createElement('video');
const load = vi.spyOn(video, 'load').mockImplementation(() => undefined);
defineNumber(video, 'duration', 2_622.48);
defineNumber(video, 'videoWidth', 640);
defineNumber(video, 'videoHeight', 480);
Object.defineProperty(video, 'audioTracks', {
configurable: true,
value: { length: 1 },
});
const resultPromise = inspectBrowserMediaMetadata(file, 'blob:lecture', {
createMediaElement: () => video,
});
video.dispatchEvent(new Event('loadedmetadata'));
await expect(resultPromise).resolves.toEqual({
source: 'browser',
file: {
name: 'lecture.mp4',
sizeBytes: 5,
declaredMimeType: 'video/mp4',
inferredMimeType: 'video/mp4',
lastModified: file.lastModified,
},
kind: 'video',
durationSeconds: 2_622.48,
width: 640,
height: 480,
hasAudio: true,
hasVideo: true,
});
expect(video.preload).toBe('metadata');
expect(video).not.toHaveAttribute('src');
expect(load).toHaveBeenCalledTimes(2);
});
it('uses an audio element and leaves unavailable details unknown', async () => {
const file = new File(['audio'], 'speech.flac', { type: 'audio/flac' });
const audio = document.createElement('audio');
vi.spyOn(audio, 'load').mockImplementation(() => undefined);
defineNumber(audio, 'duration', 12.5);
const resultPromise = inspectBrowserMediaMetadata(file, 'blob:speech', {
createMediaElement: (kind) => {
expect(kind).toBe('audio');
return audio;
},
});
audio.dispatchEvent(new Event('loadedmetadata'));
await expect(resultPromise).resolves.toMatchObject({
source: 'browser',
kind: 'audio',
durationSeconds: 12.5,
hasAudio: true,
hasVideo: false,
});
});
it('keeps a video-container player viable when dimensions and audio hints are absent', async () => {
const file = new File(['audio-only'], 'audio-only.mp4', {
type: 'video/mp4',
});
const video = document.createElement('video');
vi.spyOn(video, 'load').mockImplementation(() => undefined);
defineNumber(video, 'duration', 8);
defineNumber(video, 'videoWidth', 0);
defineNumber(video, 'videoHeight', 0);
Object.defineProperty(video, 'audioTracks', {
configurable: true,
value: undefined,
});
const resultPromise = inspectBrowserMediaMetadata(file, 'blob:audio-only', {
createMediaElement: () => video,
});
video.dispatchEvent(new Event('loadedmetadata'));
await expect(resultPromise).resolves.toMatchObject({
kind: 'video',
durationSeconds: 8,
hasVideo: undefined,
hasAudio: undefined,
});
});
it('reports load errors without revoking the caller-owned object URL', async () => {
const revokeObjectUrl = vi.spyOn(URL, 'revokeObjectURL');
const video = document.createElement('video');
vi.spyOn(video, 'load').mockImplementation(() => undefined);
const resultPromise = inspectBrowserMediaMetadata(
new File([], 'broken.mp4', { type: 'video/mp4' }),
'blob:broken',
{ createMediaElement: () => video }
);
video.dispatchEvent(new Event('error'));
await expect(resultPromise).rejects.toMatchObject({
name: 'BrowserMetadataError',
code: 'load-error',
});
expect(revokeObjectUrl).not.toHaveBeenCalled();
});
it('supports cancellation and a bounded metadata timeout', async () => {
const file = new File([], 'long.mp4', { type: 'video/mp4' });
const controller = new AbortController();
const firstVideo = document.createElement('video');
vi.spyOn(firstVideo, 'load').mockImplementation(() => undefined);
const cancelled = inspectBrowserMediaMetadata(file, 'blob:cancel', {
createMediaElement: () => firstVideo,
signal: controller.signal,
});
controller.abort();
await expect(cancelled).rejects.toMatchObject({
code: 'aborted',
});
vi.useFakeTimers();
const secondVideo = document.createElement('video');
vi.spyOn(secondVideo, 'load').mockImplementation(() => undefined);
const timedOut = inspectBrowserMediaMetadata(file, 'blob:timeout', {
createMediaElement: () => secondVideo,
timeoutMilliseconds: 25,
});
const timeoutExpectation = expect(timedOut).rejects.toEqual(
new BrowserMetadataError(
'timeout',
'Browser metadata loading exceeded 1 seconds.'
)
);
await vi.advanceTimersByTimeAsync(25);
await timeoutExpectation;
});
});
function defineNumber(
target: HTMLMediaElement,
property: 'duration' | 'videoHeight' | 'videoWidth',
value: number
): void {
Object.defineProperty(target, property, {
configurable: true,
value,
});
}