feat: add bounded MSZIP cabinet extraction
This commit is contained in:
11
CHANGELOG.md
11
CHANGELOG.md
@@ -8,17 +8,18 @@
|
||||
- Add a worker-hosted native TypeScript parser for a desktop OneNote
|
||||
revision-store subset: bounded file-node and object-space traversal,
|
||||
property sets, page metadata, titles, timestamps, levels, and plain text.
|
||||
- Add in-memory CAB enumeration and extraction for uncompressed and LZX
|
||||
- Add in-memory CAB enumeration and extraction for uncompressed, MSZIP, and LZX
|
||||
folders, including checksums, strict paths, explicit resource limits, and
|
||||
cooperative cancellation between data blocks.
|
||||
- Open contained `.one` sections independently and expose package entries,
|
||||
section/page navigation, partial failures, and structured diagnostics.
|
||||
- Verify a Joplin `.one` golden with three pages and a Joplin LZX `0x1203`
|
||||
`.onepkg` golden with seven entries, five sections, and seven pages. Also
|
||||
verify rust-cab uncompressed and LZX `0x1303` fixtures.
|
||||
verify rust-cab uncompressed, MSZIP, and LZX `0x1303` fixtures plus a
|
||||
generated two-block MSZIP dictionary-chain fixture.
|
||||
- Add deterministic static release packaging, SHA-256 sidecars, pinned-source
|
||||
provenance, fixture licensing, and security/support documentation.
|
||||
|
||||
Known unsupported areas are FSSHTTP-packaged `.one`, MSZIP and Quantum CAB
|
||||
extraction, multi-cabinet archives, `.onetoc2` hierarchy/order interpretation,
|
||||
visual content, attachments, layout/format fidelity, links, and exports.
|
||||
Known unsupported areas are FSSHTTP-packaged `.one`, Quantum CAB extraction,
|
||||
multi-cabinet archives, `.onetoc2` hierarchy/order interpretation, visual
|
||||
content, attachments, layout/format fidelity, links, and exports.
|
||||
|
||||
21
README.md
21
README.md
@@ -14,25 +14,28 @@ and is delivered to Toolbox Portal as an already-built, checksummed static ZIP.
|
||||
- Desktop revision-store `.one` sections from the tested OneNote family. The
|
||||
reader exposes section/page names, stable page IDs, created and updated
|
||||
timestamps when present, page level, and readable plain text.
|
||||
- Single-cabinet `.onepkg` files whose CAB folders are uncompressed or LZX
|
||||
compressed. Each contained desktop `.one` section is parsed independently,
|
||||
so one unsupported section does not hide sections that did parse.
|
||||
- Single-cabinet `.onepkg` files whose CAB folders are uncompressed, MSZIP, or
|
||||
LZX compressed. Each contained desktop `.one` section is parsed
|
||||
independently, so one unsupported section does not hide sections that did
|
||||
parse.
|
||||
- Package entries, safe normalized paths, uncompressed sizes, compression
|
||||
metadata, section/page navigation, and structured diagnostics.
|
||||
|
||||
The golden tests parse a real Joplin desktop `.one` file with three pages and a
|
||||
real Joplin `.onepkg` with seven entries, five sections, and seven pages. LZX
|
||||
`0x1203` and `0x1303` data and an uncompressed CAB are covered by fixtures.
|
||||
`0x1203` and `0x1303`, uncompressed CAB, and one- and two-block MSZIP data are
|
||||
covered by fixtures. The MSZIP fixtures are CAB-level evidence, not a claim
|
||||
about a particular OneNote `.onepkg` producer.
|
||||
|
||||
This is a structural reader, not a full-fidelity OneNote renderer. Images,
|
||||
tables, ink, embedded files, layout, formatting, links, and exports are not
|
||||
rendered. FSSHTTP-packaged `.one`, MSZIP and Quantum CAB extraction,
|
||||
multi-cabinet archives, and `.onetoc2` hierarchy/order interpretation are not
|
||||
supported. See [format support](docs/FORMAT_SUPPORT.md) for the precise matrix.
|
||||
rendered. FSSHTTP-packaged `.one`, Quantum CAB extraction, multi-cabinet
|
||||
archives, and `.onetoc2` hierarchy/order interpretation are not supported. See
|
||||
[format support](docs/FORMAT_SUPPORT.md) for the precise matrix.
|
||||
|
||||
There is no Rust/Wasm runtime, server, proxy, upload, telemetry, or notebook
|
||||
storage. All parsing and CAB/LZX decompression are ordinary browser JavaScript
|
||||
produced from the TypeScript source.
|
||||
storage. All parsing and CAB/LZX/MSZIP decompression is ordinary browser
|
||||
JavaScript produced from the TypeScript source.
|
||||
|
||||
## Use
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ The production application uses:
|
||||
|
||||
- `@add-ideas/toolbox-contract` 0.1.1, Apache-2.0;
|
||||
- `@add-ideas/toolbox-shell-react` 0.1.1, Apache-2.0;
|
||||
- fflate 0.8.3, MIT;
|
||||
- React, React DOM, and Scheduler, MIT.
|
||||
|
||||
The release packager collects the complete installed runtime licence texts into
|
||||
@@ -22,6 +23,9 @@ adaptation of `lzxd` 0.2.5 (MIT OR Apache-2.0), pinned at
|
||||
option. Adapted modules carry provenance headers and the licence texts are in
|
||||
`LICENSES/rust-cab-MIT.txt` and `LICENSES/lzxd-MIT.txt`.
|
||||
|
||||
MSZIP raw DEFLATE decoding uses fflate 0.8.3 (MIT). Its complete installed
|
||||
licence text is included in the generated runtime licence bundle.
|
||||
|
||||
The Base64-encoded `onenote_desktop.one` test fixture comes from Joplin at
|
||||
commit `28eb53bd9f0cd4ae0dec0c59a08037075a38302b` and is distributed under
|
||||
AGPL-3.0-or-later. Its licence text is in `LICENSES/AGPL-3.0.txt`.
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
| `.onepkg`, single CAB, LZX | Supported subset | Real Joplin package: LZX `0x1203`, three CFDATA blocks, seven entries, five sections, seven pages |
|
||||
| CAB LZX `0x1303` | Supported extraction | Byte-for-byte rust-cab MIT unit fixture, two text entries |
|
||||
| CAB without compression | Supported extraction | Byte-for-byte rust-cab MIT unit fixture |
|
||||
| CAB MSZIP | Unsupported | Metadata is recognized; extraction fails with `cab-compression-unsupported` |
|
||||
| CAB MSZIP | Supported extraction | rust-cab single-block fixture plus a generated two-block dictionary-chain fixture |
|
||||
| CAB Quantum | Unsupported | Metadata is recognized; extraction fails with `cab-compression-unsupported` |
|
||||
| Multi-cabinet CAB / spanning files | Unsupported | Header flags and spanning folder indexes are rejected |
|
||||
| `.onetoc2` package entries | Listed only | Files are identified, but notebook hierarchy and section order are not interpreted |
|
||||
@@ -24,6 +24,12 @@ a controlled failure.
|
||||
|
||||
The default LZX limit accepts CAB window sizes through 8 MiB, but compatibility
|
||||
claims are fixture-backed only for window bits 18 (`0x1203`) and 19 (`0x1303`).
|
||||
MSZIP validates the `CK` prefix, limits each compressed block to 32 KiB + 12,
|
||||
decodes into a fixed-size output buffer, requires the declared output length,
|
||||
and carries at most 32 KiB of history between blocks in one folder. The MSZIP
|
||||
tests establish CAB extraction behavior; no redistributable MSZIP-compressed
|
||||
OneNote export is currently present, so they do not establish producer-specific
|
||||
`.onepkg` compatibility.
|
||||
Trailing CAB data is ignored with a warning. Reserved CAB areas are bounded and
|
||||
skipped. CAB names are decoded as strict UTF-8; unmarked valid UTF-8 receives a
|
||||
warning, while invalid non-ASCII legacy-code-page names are rejected rather
|
||||
|
||||
@@ -12,10 +12,11 @@ The desktop `.one` reader adapts selected structures from the pinned
|
||||
|
||||
The browser-native package path has separate MIT provenance:
|
||||
|
||||
| TypeScript module | Upstream path(s) | Revision and selected licence | Port scope and deviations |
|
||||
| -------------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `src/onenote/onepkg/{cabinet,checksum}.ts` | rust-cab `src/{cabinet,folder,file,ctype,checksum}.rs` | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2`, MIT | CAB reading/checksum adapted for immutable browser bytes, strict normalized paths, checked ranges/checksums, whole-folder extraction, explicit limits, and cancellation/yield hooks; writers and MSZIP are not ported |
|
||||
| `src/onenote/onepkg/lzx/{bit-reader,huffman,decoder}.ts` | lzxd 0.2.5 `src/{bitstream,tree,block,window,lib}.rs` | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd`, MIT option of MIT OR Apache-2.0 | Directly translated core with typed arrays, CAB CFDATA boundaries, persistent folder state, strict malformed-stream errors, and operation/window/cancellation limits; delta reference data is omitted |
|
||||
| TypeScript module | Upstream path(s) | Revision and selected licence | Port scope and deviations |
|
||||
| -------------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `src/onenote/onepkg/{cabinet,checksum}.ts` | rust-cab `src/{cabinet,folder,file,ctype,checksum}.rs` | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2`, MIT | CAB reading/checksum adapted for immutable browser bytes, strict normalized paths, checked ranges/checksums, whole-folder extraction, explicit limits, and cancellation/yield hooks; writers are not ported |
|
||||
| `src/onenote/onepkg/mszip/decoder.ts` | rust-cab `src/mszip.rs`; fflate raw DEFLATE | rust-cab revision above; fflate 0.8.3, MIT | `CK` framing and persistent 32 KiB folder history follow rust-cab; raw DEFLATE uses fflate with a caller-owned bounded buffer and strict declared-size checks |
|
||||
| `src/onenote/onepkg/lzx/{bit-reader,huffman,decoder}.ts` | lzxd 0.2.5 `src/{bitstream,tree,block,window,lib}.rs` | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd`, MIT option of MIT OR Apache-2.0 | Directly translated core with typed arrays, CAB CFDATA boundaries, persistent folder state, strict malformed-stream errors, and operation/window/cancellation limits; delta reference data is omitted |
|
||||
|
||||
The bounded binary readers, format detection, parser limits/error model, path
|
||||
policy, worker protocol/DTOs, package-to-section boundary, and application UI
|
||||
|
||||
@@ -2,13 +2,13 @@
|
||||
|
||||
Version 0.1.0 records these immutable implementation revisions:
|
||||
|
||||
| Repository | Exact revision | Use |
|
||||
| ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------- |
|
||||
| [`toolbox-sdk`](https://git.add-ideas.de/zemion/toolbox-sdk/src/commit/5e206baf42dfae5a00f2743fb2cc501bb12d9b7a) | `5e206baf42dfae5a00f2743fb2cc501bb12d9b7a` | App contract, React shell, manifest schema/testkit, and nested-path behavior |
|
||||
| [`toolbox-portal`](https://git.add-ideas.de/zemion/toolbox-portal/src/commit/5d2e466ad61e36a0e2be2a2d8459b1a127f53f11) | `5d2e466ad61e36a0e2be2a2d8459b1a127f53f11` | Reviewed ZIP shape, exact checksum lock, and independent-app assembly model |
|
||||
| [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop OneStore and MS-ONE structures adapted into TypeScript |
|
||||
| [`rust-cab`](https://github.com/mdsteele/rust-cab/tree/c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2) | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2` | CAB structures, checksum, enumeration/extraction, and two unit fixtures |
|
||||
| [`lzxd` 0.2.5](https://github.com/Lonami/lzxd/tree/4748e43594e3e30cff2ace3a6ad7a376c9816fdd) | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd` | LZX bitstream, Huffman, block/window, match, and E8 logic adapted into TypeScript |
|
||||
| Repository | Exact revision | Use |
|
||||
| ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------- |
|
||||
| [`toolbox-sdk`](https://git.add-ideas.de/zemion/toolbox-sdk/src/commit/5e206baf42dfae5a00f2743fb2cc501bb12d9b7a) | `5e206baf42dfae5a00f2743fb2cc501bb12d9b7a` | App contract, React shell, manifest schema/testkit, and nested-path behavior |
|
||||
| [`toolbox-portal`](https://git.add-ideas.de/zemion/toolbox-portal/src/commit/5d2e466ad61e36a0e2be2a2d8459b1a127f53f11) | `5d2e466ad61e36a0e2be2a2d8459b1a127f53f11` | Reviewed ZIP shape, exact checksum lock, and independent-app assembly model |
|
||||
| [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop OneStore and MS-ONE structures adapted into TypeScript |
|
||||
| [`rust-cab`](https://github.com/mdsteele/rust-cab/tree/c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2) | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2` | CAB structures, checksum, enumeration/extraction, MSZIP history, and unit fixtures |
|
||||
| [`lzxd` 0.2.5](https://github.com/Lonami/lzxd/tree/4748e43594e3e30cff2ace3a6ad7a376c9816fdd) | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd` | LZX bitstream, Huffman, block/window, match, and E8 logic adapted into TypeScript |
|
||||
|
||||
The SDK and Portal revisions were inspected from the local repositories. The
|
||||
three parser references were cloned and reviewed at the hashes above; a branch
|
||||
@@ -32,6 +32,9 @@ truth and the pinned projects as implementation references:
|
||||
- [[MS-CAB]: Cabinet File Format](https://download.microsoft.com/download/4/d/a/4da14f27-b4ef-4170-a6e6-5b1ef85b1baa/%5Bms-cab%5D.pdf),
|
||||
sections 2.1-2.4 (`CFHEADER`, `CFFOLDER`, `CFFILE`, and `CFDATA`) and 3.1
|
||||
(checksum method).
|
||||
- [[MS-MCI]: Compression and Decompression Data Structure](https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-mci/3048c471-f739-4d30-bcb3-a962cfe84a69),
|
||||
MSZIP block framing, raw DEFLATE streams, 32 KiB block/history behavior, and
|
||||
compressed-size bounds.
|
||||
- [[MS-PATCH]: LZX Delta Compression and Decompression](https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-patch/cc78752a-b4af-4eee-88cb-01f4d8a4c2bf),
|
||||
sections 2.1-2.7 (bitstream/window concepts, header/E8 translation, blocks,
|
||||
Huffman trees, pretree encoding, tokens, and decoding). `lzxd` 0.2.5 states
|
||||
|
||||
@@ -57,6 +57,7 @@ The CAB/package path applies these defaults:
|
||||
| Total folder output | 512 MiB |
|
||||
| Total extracted entry bytes | 512 MiB |
|
||||
| Folder compression ratio | 200:1 |
|
||||
| One MSZIP compressed block | 32 KiB + 12 B |
|
||||
| LZX window | 8 MiB |
|
||||
| LZX decode operations | 1,000,000,000 |
|
||||
|
||||
@@ -66,11 +67,13 @@ checksums, rejects overlapping/out-of-range data, rejects absolute/traversal,
|
||||
empty, control-character, and case-insensitive duplicate paths, and refuses
|
||||
multi-cabinet/spanning entries.
|
||||
|
||||
LZX checks its window, Huffman trees, block sizes, match ranges, output size,
|
||||
and operation budget. CAB extraction checks cancellation and yields between
|
||||
data blocks so the worker can process termination. Desktop OneStore traversal
|
||||
checks fragment/reference cycles, nesting, object/property/reference counts,
|
||||
and byte ranges.
|
||||
MSZIP checks its `CK` framing and raw DEFLATE data, writes into a fixed caller
|
||||
buffer with one overflow-detection byte, requires the exact declared output
|
||||
size, and retains no more than 32 KiB of history. LZX checks its window, Huffman
|
||||
trees, block sizes, match ranges, output size, and operation budget. CAB
|
||||
extraction checks cancellation and yields between data blocks so the worker can
|
||||
process termination. Desktop OneStore traversal checks fragment/reference
|
||||
cycles, nesting, object/property/reference counts, and byte ranges.
|
||||
|
||||
## Rendering and links
|
||||
|
||||
|
||||
3
package-lock.json
generated
3
package-lock.json
generated
@@ -11,6 +11,7 @@
|
||||
"dependencies": {
|
||||
"@add-ideas/toolbox-contract": "0.1.1",
|
||||
"@add-ideas/toolbox-shell-react": "0.1.1",
|
||||
"fflate": "^0.8.3",
|
||||
"react": "^19.2.6",
|
||||
"react-dom": "^19.2.6"
|
||||
},
|
||||
@@ -28,7 +29,6 @@
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
"eslint-plugin-react-hooks": "^7.1.1",
|
||||
"eslint-plugin-react-refresh": "^0.5.2",
|
||||
"fflate": "^0.8.3",
|
||||
"globals": "^17.6.0",
|
||||
"jsdom": "^29.1.1",
|
||||
"prettier": "^3.8.3",
|
||||
@@ -2342,7 +2342,6 @@
|
||||
"version": "0.8.3",
|
||||
"resolved": "https://registry.npmjs.org/fflate/-/fflate-0.8.3.tgz",
|
||||
"integrity": "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/file-entry-cache": {
|
||||
|
||||
@@ -29,6 +29,7 @@
|
||||
"dependencies": {
|
||||
"@add-ideas/toolbox-contract": "0.1.1",
|
||||
"@add-ideas/toolbox-shell-react": "0.1.1",
|
||||
"fflate": "^0.8.3",
|
||||
"react": "^19.2.6",
|
||||
"react-dom": "^19.2.6"
|
||||
},
|
||||
@@ -46,7 +47,6 @@
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
"eslint-plugin-react-hooks": "^7.1.1",
|
||||
"eslint-plugin-react-refresh": "^0.5.2",
|
||||
"fflate": "^0.8.3",
|
||||
"globals": "^17.6.0",
|
||||
"jsdom": "^29.1.1",
|
||||
"prettier": "^3.8.3",
|
||||
|
||||
@@ -39,6 +39,7 @@ export async function collectReleaseMetadataFiles(projectRoot) {
|
||||
await createThirdPartyLicenseBundle(projectRoot, [
|
||||
'@add-ideas/toolbox-contract',
|
||||
'@add-ideas/toolbox-shell-react',
|
||||
'fflate',
|
||||
'react',
|
||||
'react-dom',
|
||||
'scheduler',
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import type { OneNoteSectionDto } from '../model/dto.js';
|
||||
import { enumerateCabinet, extractCabinet } from './cabinet.js';
|
||||
import { OnePkgError } from './errors.js';
|
||||
import { LzxDecoder } from './lzx/decoder.js';
|
||||
import { MsZipDecoder } from './mszip/decoder.js';
|
||||
import { openOneNotePackage } from './package.js';
|
||||
|
||||
/*
|
||||
@@ -33,6 +36,37 @@ const TWO_FILE_UNCOMPRESSED_CAB = binary(
|
||||
'\0\0\0\0\x1d\0\x1d\0Hello, world!\nSee you later!\n'
|
||||
);
|
||||
|
||||
const MSZIP_CAB = binary(
|
||||
'MSCF\0\0\0\0\x61\0\0\0\0\0\0\0' +
|
||||
'\x2c\0\0\0\0\0\0\0\x03\x01\x01\0\x01\0\0\0\x34\x12\0\0' +
|
||||
'\x43\0\0\0\x01\0\x01\0' +
|
||||
'\x0e\0\0\0\0\0\0\0\0\0\x6c\x22\xe7\x59\x01\0hi.txt\0' +
|
||||
'\0\0\0\0\x16\0\x0e\0' +
|
||||
'CK\xf3H\xcd\xc9\xc9\xd7Q(\xcf/\xcaIQ\xe4\x02\x00$\xf2\x04\x94'
|
||||
);
|
||||
|
||||
/*
|
||||
* Generated two-block CAB; generation details and hashes are recorded in
|
||||
* tests/fixtures/README.md. Its second raw DEFLATE stream requires the first
|
||||
* block's 32 KiB history.
|
||||
*/
|
||||
const MULTI_BLOCK_MSZIP_CAB = Uint8Array.from(
|
||||
Buffer.from(
|
||||
'TVNDRgAAAABRAgAAAAAAACwAAAAAAAAAAwEBAAEAAADvvgAASAAAAAIAAQAAkAAAAAAAAAAA' +
|
||||
'AAAAACAAaGlzdG9yeS5iaW4Ak4QnWtcBAIBDS+3P1yIQAABAURQtRIpK2Tu7oU1SqRCS7BAa' +
|
||||
'2iVZFdo0aBDaGVGRVLaWEYq21RSiRNmeevIX9/zBERAUGjZcWGTEyFGjx4iKiY+VkBwnNX6C' +
|
||||
'tMzESZNlp0yVk1dQVFJWUVVT19DUmqato6unb2A4fcbMWUaz58ydN3/BQmOTRaaLzZYsXWa+' +
|
||||
'fMVKC0urVdY2tqvt1tivdXB0cnZxdVvn7uG53svbZ8PGTZt9t2zdtn3Hzl279/jt9d8XEBgU' +
|
||||
'HLL/wMHQsPBDh48cPXb8RETkyVOnz0RFnz13/kJM7MW4+IRLl69cvXb9xs3EpOSUW6lpt+/c' +
|
||||
'Tc+4l3k/68HDR9k5uXn5BYVFj588ffa8uKS07EV5ReXLV1XVr9+8fff+w8ea2rr6hk+fv3z9' +
|
||||
'9r3xR1Nzy8/Wtl+/2/90dP7919Xd09vXPzAoQJ06derUqVOnTp06derUqVOnTp06derUqVOn' +
|
||||
'Tp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOn' +
|
||||
'Tp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOn' +
|
||||
'Tp06derUqQ/V/wM/Q7GFIgAAEENL7c+BDAAAAMAgf+t7fGWQurq6urq6urq6urq6urr6vx4=',
|
||||
'base64'
|
||||
)
|
||||
);
|
||||
|
||||
const LZX_CAB = binary(
|
||||
'\x4d\x53\x43\x46\x00\x00\x00\x00\x97\x00\x00\x00\x00\x00\x00' +
|
||||
'\x00\x2c\x00\x00\x00\x00\x00\x00\x00\x03\x01\x01\x00\x02\x00' +
|
||||
@@ -109,6 +143,89 @@ describe('CAB enumeration and extraction', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it('extracts the rust-cab single-block MSZIP fixture', async () => {
|
||||
const listing = enumerateCabinet(MSZIP_CAB);
|
||||
expect(listing.folders[0]!.compression).toEqual({
|
||||
kind: 'mszip',
|
||||
raw: 1,
|
||||
label: 'MSZIP',
|
||||
});
|
||||
expect(listing.diagnostics).not.toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({ code: 'cab-compression-unsupported' }),
|
||||
])
|
||||
);
|
||||
|
||||
const extracted = await extractCabinet(MSZIP_CAB);
|
||||
expect(new TextDecoder().decode(extracted.extractedEntries[0]!.bytes)).toBe(
|
||||
'Hello, world!\n'
|
||||
);
|
||||
});
|
||||
|
||||
it('retains the MSZIP dictionary across two CFDATA blocks', async () => {
|
||||
expect(MULTI_BLOCK_MSZIP_CAB).toHaveLength(593);
|
||||
expect(
|
||||
createHash('sha256').update(MULTI_BLOCK_MSZIP_CAB).digest('hex')
|
||||
).toBe('9f71b97623395492f401ddc152ca648e8416996b14092b0df42ad712c27f9cb2');
|
||||
const listing = enumerateCabinet(MULTI_BLOCK_MSZIP_CAB);
|
||||
expect(listing.folders[0]).toMatchObject({
|
||||
dataBlockCount: 2,
|
||||
compressedSize: 505,
|
||||
uncompressedSize: 36_864,
|
||||
compression: { kind: 'mszip' },
|
||||
});
|
||||
|
||||
const extracted = await extractCabinet(MULTI_BLOCK_MSZIP_CAB);
|
||||
const output = extracted.extractedEntries[0]!.bytes;
|
||||
expect(output).toHaveLength(36_864);
|
||||
expect(createHash('sha256').update(output).digest('hex')).toBe(
|
||||
'cb441d1f6e7d9c5789a306c51a00a654c9b7715c2c3d624e3d2a75bd0af549ff'
|
||||
);
|
||||
expect(output).toEqual(
|
||||
Uint8Array.from({ length: 36_864 }, (_, index) => index % 251)
|
||||
);
|
||||
|
||||
const firstBlock = MULTI_BLOCK_MSZIP_CAB.subarray(80, 551);
|
||||
const secondBlock = MULTI_BLOCK_MSZIP_CAB.subarray(559, 593);
|
||||
const decoder = new MsZipDecoder();
|
||||
expect(decoder.decompressNext(firstBlock, 32_768)).toEqual(
|
||||
output.subarray(0, 32_768)
|
||||
);
|
||||
expect(decoder.decompressNext(secondBlock, 4_096)).toEqual(
|
||||
output.subarray(32_768)
|
||||
);
|
||||
expectCode(
|
||||
() => new MsZipDecoder().decompressNext(secondBlock, 4_096),
|
||||
'mszip-invalid-deflate'
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects malformed MSZIP framing, DEFLATE, and output sizes', async () => {
|
||||
const signature = MSZIP_CAB.slice();
|
||||
const signatureOffset = findAscii(signature, 'CK');
|
||||
signature[signatureOffset] = 0;
|
||||
await expect(extractCabinet(signature)).rejects.toMatchObject({
|
||||
diagnostic: { code: 'mszip-invalid-signature' },
|
||||
});
|
||||
|
||||
const deflate = MSZIP_CAB.slice();
|
||||
deflate[findAscii(deflate, 'CK') + 2] = 0x07;
|
||||
await expect(extractCabinet(deflate)).rejects.toMatchObject({
|
||||
diagnostic: { code: 'mszip-invalid-deflate' },
|
||||
});
|
||||
|
||||
const wrongOutputSize = MSZIP_CAB.slice();
|
||||
const outputSizeOffset = findAscii(wrongOutputSize, 'CK') - 2;
|
||||
new DataView(
|
||||
wrongOutputSize.buffer,
|
||||
wrongOutputSize.byteOffset,
|
||||
wrongOutputSize.byteLength
|
||||
).setUint16(outputSizeOffset, 15, true);
|
||||
await expect(extractCabinet(wrongOutputSize)).rejects.toMatchObject({
|
||||
diagnostic: { code: 'mszip-output-size-mismatch' },
|
||||
});
|
||||
});
|
||||
|
||||
it('retains an uncompressed LZX block across CAB chunks', () => {
|
||||
const decoder = new LzxDecoder(18, { maxOperations: 100 });
|
||||
const first = binary(
|
||||
|
||||
@@ -13,6 +13,7 @@ import { checkCancellation, yieldForCancellation } from './cancellation.js';
|
||||
import { calculateDataBlockChecksum } from './checksum.js';
|
||||
import { fail, warning } from './errors.js';
|
||||
import { LzxDecoder } from './lzx/decoder.js';
|
||||
import { MsZipDecoder } from './mszip/decoder.js';
|
||||
import { decodeCabFileName, normalizeCabPath } from './path.js';
|
||||
import {
|
||||
resolveLimits,
|
||||
@@ -30,6 +31,7 @@ const FLAG_NEXT_CABINET = 0x0002;
|
||||
const FLAG_RESERVE_PRESENT = 0x0004;
|
||||
const ATTRIBUTE_NAME_IS_UTF8 = 0x0080;
|
||||
const MAX_CAB_BLOCK_OUTPUT = 32 * 1024;
|
||||
const MAX_MSZIP_BLOCK_INPUT = 32 * 1024 + 12;
|
||||
|
||||
interface CabDataBlock {
|
||||
headerOffset: number;
|
||||
@@ -458,6 +460,16 @@ function parseCabinet(
|
||||
{ offset: headerOffset + 4, structure: 'CFDATA' }
|
||||
);
|
||||
}
|
||||
if (
|
||||
folder.compression.kind === 'mszip' &&
|
||||
compressedSize > MAX_MSZIP_BLOCK_INPUT
|
||||
) {
|
||||
fail(
|
||||
'cab-invalid-mszip-block-size',
|
||||
`MSZIP CAB data block size ${compressedSize} exceeds ${MAX_MSZIP_BLOCK_INPUT} bytes.`,
|
||||
{ offset: headerOffset + 4, structure: 'CFDATA.cbData' }
|
||||
);
|
||||
}
|
||||
if (
|
||||
folder.compression.kind === 'none' &&
|
||||
compressedSize !== uncompressedSize
|
||||
@@ -589,10 +601,7 @@ function parseCabinet(
|
||||
);
|
||||
}
|
||||
for (const folder of folders) {
|
||||
if (
|
||||
folder.compression.kind === 'mszip' ||
|
||||
folder.compression.kind === 'quantum'
|
||||
) {
|
||||
if (folder.compression.kind === 'quantum') {
|
||||
diagnostics.push(
|
||||
warning(
|
||||
'cab-compression-unsupported',
|
||||
@@ -646,10 +655,7 @@ export async function extractCabinet(
|
||||
const extractedEntries = [];
|
||||
for (const folder of parsed.folders) {
|
||||
checkCancellation(options.cancellation);
|
||||
if (
|
||||
folder.compression.kind === 'mszip' ||
|
||||
folder.compression.kind === 'quantum'
|
||||
) {
|
||||
if (folder.compression.kind === 'quantum') {
|
||||
fail(
|
||||
'cab-compression-unsupported',
|
||||
`${folder.compression.label} extraction is not implemented.`,
|
||||
@@ -665,6 +671,8 @@ export async function extractCabinet(
|
||||
checkCancellation: () => checkCancellation(options.cancellation),
|
||||
})
|
||||
: undefined;
|
||||
const msZipDecoder =
|
||||
folder.compression.kind === 'mszip' ? new MsZipDecoder() : undefined;
|
||||
let outputOffset = 0;
|
||||
for (const block of folder.blocks) {
|
||||
checkCancellation(options.cancellation);
|
||||
@@ -688,9 +696,11 @@ export async function extractCabinet(
|
||||
}
|
||||
}
|
||||
const output =
|
||||
decoder === undefined
|
||||
? compressed
|
||||
: decoder.decompressNext(compressed, block.uncompressedSize);
|
||||
decoder !== undefined
|
||||
? decoder.decompressNext(compressed, block.uncompressedSize)
|
||||
: msZipDecoder !== undefined
|
||||
? msZipDecoder.decompressNext(compressed, block.uncompressedSize)
|
||||
: compressed;
|
||||
if (output.length !== block.uncompressedSize) {
|
||||
fail(
|
||||
'cab-output-size-mismatch',
|
||||
|
||||
95
src/onenote/onepkg/mszip/decoder.ts
Normal file
95
src/onenote/onepkg/mszip/decoder.ts
Normal file
@@ -0,0 +1,95 @@
|
||||
/*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*
|
||||
* MSZIP block framing and 32 KiB history semantics follow rust-cab 0.6.0.
|
||||
* rust-cab commit c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2
|
||||
* Copyright (c) 2017 Matthew D. Steele, MIT License.
|
||||
* Raw DEFLATE decoding is provided by fflate 0.8.3 (MIT).
|
||||
* Modified for fixed browser buffers and strict block/output validation.
|
||||
*/
|
||||
|
||||
import { inflateSync } from 'fflate';
|
||||
|
||||
import { fail } from '../errors.js';
|
||||
|
||||
const MAX_BLOCK_OUTPUT = 32 * 1024;
|
||||
const HISTORY_SIZE = 32 * 1024;
|
||||
const SIGNATURE_FIRST = 0x43;
|
||||
const SIGNATURE_SECOND = 0x4b;
|
||||
|
||||
export class MsZipDecoder {
|
||||
private history = new Uint8Array(0);
|
||||
|
||||
decompressNext(chunk: Uint8Array, outputLength: number): Uint8Array {
|
||||
if (
|
||||
!Number.isInteger(outputLength) ||
|
||||
outputLength <= 0 ||
|
||||
outputLength > MAX_BLOCK_OUTPUT
|
||||
) {
|
||||
fail(
|
||||
'mszip-invalid-chunk-size',
|
||||
`MSZIP CAB chunk output size ${outputLength} is outside 1..${MAX_BLOCK_OUTPUT}.`,
|
||||
{ structure: 'CFDATA.cbUncomp' }
|
||||
);
|
||||
}
|
||||
if (
|
||||
chunk.length < 2 ||
|
||||
chunk[0] !== SIGNATURE_FIRST ||
|
||||
chunk[1] !== SIGNATURE_SECOND
|
||||
) {
|
||||
fail(
|
||||
'mszip-invalid-signature',
|
||||
'MSZIP data block does not begin with the CK signature.',
|
||||
{ structure: 'MSZIP block' }
|
||||
);
|
||||
}
|
||||
|
||||
let output: Uint8Array;
|
||||
try {
|
||||
// One extra byte lets us distinguish an exact result from a stream that
|
||||
// expands beyond the trusted CFDATA declaration. fflate writes into this
|
||||
// caller-owned buffer, so malformed input cannot trigger output growth.
|
||||
output = inflateSync(chunk.subarray(2), {
|
||||
dictionary: this.history.length === 0 ? undefined : this.history,
|
||||
out: new Uint8Array(outputLength + 1),
|
||||
});
|
||||
} catch {
|
||||
fail(
|
||||
'mszip-invalid-deflate',
|
||||
'MSZIP block contains an invalid raw DEFLATE stream.',
|
||||
{ structure: 'MSZIP block' }
|
||||
);
|
||||
}
|
||||
|
||||
if (output.length !== outputLength) {
|
||||
fail(
|
||||
'mszip-output-size-mismatch',
|
||||
`MSZIP block produced ${output.length} bytes; expected ${outputLength}.`,
|
||||
{ structure: 'CFDATA.cbUncomp' }
|
||||
);
|
||||
}
|
||||
|
||||
this.updateHistory(output);
|
||||
return output;
|
||||
}
|
||||
|
||||
private updateHistory(output: Uint8Array): void {
|
||||
const historyLength = Math.min(
|
||||
HISTORY_SIZE,
|
||||
this.history.length + output.length
|
||||
);
|
||||
const nextHistory = new Uint8Array(historyLength);
|
||||
const oldBytes = Math.min(
|
||||
this.history.length,
|
||||
historyLength - output.length
|
||||
);
|
||||
if (oldBytes > 0) {
|
||||
nextHistory.set(this.history.subarray(this.history.length - oldBytes));
|
||||
}
|
||||
nextHistory.set(
|
||||
output.subarray(output.length - (historyLength - oldBytes)),
|
||||
oldBytes
|
||||
);
|
||||
this.history = nextHistory;
|
||||
}
|
||||
}
|
||||
30
tests/fixtures/README.md
vendored
30
tests/fixtures/README.md
vendored
@@ -47,9 +47,33 @@ They are MIT-licensed; the exact upstream text is in
|
||||
`LICENSES/rust-cab-MIT.txt`.
|
||||
|
||||
The arrays cover an uncompressed single-entry CAB, an uncompressed two-entry
|
||||
CAB, and an LZX `0x1303` CAB containing `hi.txt` and `bye.txt`. They are CAB
|
||||
unit fixtures, not OneNote exports, and are not evidence of `.onepkg` producer
|
||||
compatibility.
|
||||
CAB, a single-block MSZIP CAB, and an LZX `0x1303` CAB containing `hi.txt` and
|
||||
`bye.txt`. They are CAB unit fixtures, not OneNote exports, and are not evidence
|
||||
of `.onepkg` producer compatibility.
|
||||
|
||||
## Inline generated two-block MSZIP fixture
|
||||
|
||||
`src/onenote/onepkg/cabinet.test.ts` also contains a generated 593-byte CAB
|
||||
with one 36,864-byte `history.bin` entry. The input byte at index `i` is
|
||||
`i % 251`. It is split into a 32,768-byte block and a 4,096-byte block; the
|
||||
second raw DEFLATE stream uses the first block as its preset 32 KiB dictionary,
|
||||
so decoding it without folder history fails.
|
||||
|
||||
- Generator: CPython 3.13.14 `zlib.compressobj`, zlib 1.3.1, raw DEFLATE
|
||||
(`wbits=-15`), level 9; the second call uses `zdict=first_block`. Each stream
|
||||
is prefixed with `CK` and placed in a spec-shaped single-folder CAB with CAB
|
||||
checksums.
|
||||
- Generator invocation: `python3` with the documented deterministic input and
|
||||
CAB layout above; the frozen Base64 bytes, sizes, checksums, and hashes are
|
||||
asserted by the test. No notebook or third-party content is an input.
|
||||
- CAB SHA-256:
|
||||
`9f71b97623395492f401ddc152ca648e8416996b14092b0df42ad712c27f9cb2`.
|
||||
- Extracted SHA-256:
|
||||
`cb441d1f6e7d9c5789a306c51a00a654c9b7715c2c3d624e3d2a75bd0af549ff`.
|
||||
- Licence: original test data under this project's MPL-2.0 licence.
|
||||
|
||||
This fixture establishes block-history behavior only. It is not a OneNote
|
||||
export and is not evidence of `.onepkg` producer compatibility.
|
||||
|
||||
## Distribution policy
|
||||
|
||||
|
||||
@@ -94,6 +94,9 @@ describe('release archive', () => {
|
||||
expect(
|
||||
createHash('sha256').update(files['LICENSES/lzxd-MIT.txt']!).digest('hex')
|
||||
).toBe('c7a1dcbf839b1dfb72f24e1de12ab55f789db2c61d894ca9aa8b5f595490d9ca');
|
||||
expect(
|
||||
files['LICENSES/THIRD-PARTY-LICENSES.txt']!.toString('utf8')
|
||||
).toContain('fflate@0.8.3');
|
||||
expect(files).not.toHaveProperty(
|
||||
'tests/fixtures/joplin-test.onepkg.base64'
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user