feat: add bounded MSZIP cabinet extraction

This commit is contained in:
2026-07-22 19:01:50 +02:00
parent a0c26d604d
commit d282b832ac
15 changed files with 318 additions and 48 deletions

View File

@@ -8,17 +8,18 @@
- Add a worker-hosted native TypeScript parser for a desktop OneNote - Add a worker-hosted native TypeScript parser for a desktop OneNote
revision-store subset: bounded file-node and object-space traversal, revision-store subset: bounded file-node and object-space traversal,
property sets, page metadata, titles, timestamps, levels, and plain text. property sets, page metadata, titles, timestamps, levels, and plain text.
- Add in-memory CAB enumeration and extraction for uncompressed and LZX - Add in-memory CAB enumeration and extraction for uncompressed, MSZIP, and LZX
folders, including checksums, strict paths, explicit resource limits, and folders, including checksums, strict paths, explicit resource limits, and
cooperative cancellation between data blocks. cooperative cancellation between data blocks.
- Open contained `.one` sections independently and expose package entries, - Open contained `.one` sections independently and expose package entries,
section/page navigation, partial failures, and structured diagnostics. section/page navigation, partial failures, and structured diagnostics.
- Verify a Joplin `.one` golden with three pages and a Joplin LZX `0x1203` - Verify a Joplin `.one` golden with three pages and a Joplin LZX `0x1203`
`.onepkg` golden with seven entries, five sections, and seven pages. Also `.onepkg` golden with seven entries, five sections, and seven pages. Also
verify rust-cab uncompressed and LZX `0x1303` fixtures. verify rust-cab uncompressed, MSZIP, and LZX `0x1303` fixtures plus a
generated two-block MSZIP dictionary-chain fixture.
- Add deterministic static release packaging, SHA-256 sidecars, pinned-source - Add deterministic static release packaging, SHA-256 sidecars, pinned-source
provenance, fixture licensing, and security/support documentation. provenance, fixture licensing, and security/support documentation.
Known unsupported areas are FSSHTTP-packaged `.one`, MSZIP and Quantum CAB Known unsupported areas are FSSHTTP-packaged `.one`, Quantum CAB extraction,
extraction, multi-cabinet archives, `.onetoc2` hierarchy/order interpretation, multi-cabinet archives, `.onetoc2` hierarchy/order interpretation, visual
visual content, attachments, layout/format fidelity, links, and exports. content, attachments, layout/format fidelity, links, and exports.

View File

@@ -14,25 +14,28 @@ and is delivered to Toolbox Portal as an already-built, checksummed static ZIP.
- Desktop revision-store `.one` sections from the tested OneNote family. The - Desktop revision-store `.one` sections from the tested OneNote family. The
reader exposes section/page names, stable page IDs, created and updated reader exposes section/page names, stable page IDs, created and updated
timestamps when present, page level, and readable plain text. timestamps when present, page level, and readable plain text.
- Single-cabinet `.onepkg` files whose CAB folders are uncompressed or LZX - Single-cabinet `.onepkg` files whose CAB folders are uncompressed, MSZIP, or
compressed. Each contained desktop `.one` section is parsed independently, LZX compressed. Each contained desktop `.one` section is parsed
so one unsupported section does not hide sections that did parse. independently, so one unsupported section does not hide sections that did
parse.
- Package entries, safe normalized paths, uncompressed sizes, compression - Package entries, safe normalized paths, uncompressed sizes, compression
metadata, section/page navigation, and structured diagnostics. metadata, section/page navigation, and structured diagnostics.
The golden tests parse a real Joplin desktop `.one` file with three pages and a The golden tests parse a real Joplin desktop `.one` file with three pages and a
real Joplin `.onepkg` with seven entries, five sections, and seven pages. LZX real Joplin `.onepkg` with seven entries, five sections, and seven pages. LZX
`0x1203` and `0x1303` data and an uncompressed CAB are covered by fixtures. `0x1203` and `0x1303`, uncompressed CAB, and one- and two-block MSZIP data are
covered by fixtures. The MSZIP fixtures are CAB-level evidence, not a claim
about a particular OneNote `.onepkg` producer.
This is a structural reader, not a full-fidelity OneNote renderer. Images, This is a structural reader, not a full-fidelity OneNote renderer. Images,
tables, ink, embedded files, layout, formatting, links, and exports are not tables, ink, embedded files, layout, formatting, links, and exports are not
rendered. FSSHTTP-packaged `.one`, MSZIP and Quantum CAB extraction, rendered. FSSHTTP-packaged `.one`, Quantum CAB extraction, multi-cabinet
multi-cabinet archives, and `.onetoc2` hierarchy/order interpretation are not archives, and `.onetoc2` hierarchy/order interpretation are not supported. See
supported. See [format support](docs/FORMAT_SUPPORT.md) for the precise matrix. [format support](docs/FORMAT_SUPPORT.md) for the precise matrix.
There is no Rust/Wasm runtime, server, proxy, upload, telemetry, or notebook There is no Rust/Wasm runtime, server, proxy, upload, telemetry, or notebook
storage. All parsing and CAB/LZX decompression are ordinary browser JavaScript storage. All parsing and CAB/LZX/MSZIP decompression is ordinary browser
produced from the TypeScript source. JavaScript produced from the TypeScript source.
## Use ## Use

View File

@@ -4,6 +4,7 @@ The production application uses:
- `@add-ideas/toolbox-contract` 0.1.1, Apache-2.0; - `@add-ideas/toolbox-contract` 0.1.1, Apache-2.0;
- `@add-ideas/toolbox-shell-react` 0.1.1, Apache-2.0; - `@add-ideas/toolbox-shell-react` 0.1.1, Apache-2.0;
- fflate 0.8.3, MIT;
- React, React DOM, and Scheduler, MIT. - React, React DOM, and Scheduler, MIT.
The release packager collects the complete installed runtime licence texts into The release packager collects the complete installed runtime licence texts into
@@ -22,6 +23,9 @@ adaptation of `lzxd` 0.2.5 (MIT OR Apache-2.0), pinned at
option. Adapted modules carry provenance headers and the licence texts are in option. Adapted modules carry provenance headers and the licence texts are in
`LICENSES/rust-cab-MIT.txt` and `LICENSES/lzxd-MIT.txt`. `LICENSES/rust-cab-MIT.txt` and `LICENSES/lzxd-MIT.txt`.
MSZIP raw DEFLATE decoding uses fflate 0.8.3 (MIT). Its complete installed
licence text is included in the generated runtime licence bundle.
The Base64-encoded `onenote_desktop.one` test fixture comes from Joplin at The Base64-encoded `onenote_desktop.one` test fixture comes from Joplin at
commit `28eb53bd9f0cd4ae0dec0c59a08037075a38302b` and is distributed under commit `28eb53bd9f0cd4ae0dec0c59a08037075a38302b` and is distributed under
AGPL-3.0-or-later. Its licence text is in `LICENSES/AGPL-3.0.txt`. AGPL-3.0-or-later. Its licence text is in `LICENSES/AGPL-3.0.txt`.

View File

@@ -10,7 +10,7 @@
| `.onepkg`, single CAB, LZX | Supported subset | Real Joplin package: LZX `0x1203`, three CFDATA blocks, seven entries, five sections, seven pages | | `.onepkg`, single CAB, LZX | Supported subset | Real Joplin package: LZX `0x1203`, three CFDATA blocks, seven entries, five sections, seven pages |
| CAB LZX `0x1303` | Supported extraction | Byte-for-byte rust-cab MIT unit fixture, two text entries | | CAB LZX `0x1303` | Supported extraction | Byte-for-byte rust-cab MIT unit fixture, two text entries |
| CAB without compression | Supported extraction | Byte-for-byte rust-cab MIT unit fixture | | CAB without compression | Supported extraction | Byte-for-byte rust-cab MIT unit fixture |
| CAB MSZIP | Unsupported | Metadata is recognized; extraction fails with `cab-compression-unsupported` | | CAB MSZIP | Supported extraction | rust-cab single-block fixture plus a generated two-block dictionary-chain fixture |
| CAB Quantum | Unsupported | Metadata is recognized; extraction fails with `cab-compression-unsupported` | | CAB Quantum | Unsupported | Metadata is recognized; extraction fails with `cab-compression-unsupported` |
| Multi-cabinet CAB / spanning files | Unsupported | Header flags and spanning folder indexes are rejected | | Multi-cabinet CAB / spanning files | Unsupported | Header flags and spanning folder indexes are rejected |
| `.onetoc2` package entries | Listed only | Files are identified, but notebook hierarchy and section order are not interpreted | | `.onetoc2` package entries | Listed only | Files are identified, but notebook hierarchy and section order are not interpreted |
@@ -24,6 +24,12 @@ a controlled failure.
The default LZX limit accepts CAB window sizes through 8 MiB, but compatibility The default LZX limit accepts CAB window sizes through 8 MiB, but compatibility
claims are fixture-backed only for window bits 18 (`0x1203`) and 19 (`0x1303`). claims are fixture-backed only for window bits 18 (`0x1203`) and 19 (`0x1303`).
MSZIP validates the `CK` prefix, limits each compressed block to 32 KiB + 12,
decodes into a fixed-size output buffer, requires the declared output length,
and carries at most 32 KiB of history between blocks in one folder. The MSZIP
tests establish CAB extraction behavior; no redistributable MSZIP-compressed
OneNote export is currently present, so they do not establish producer-specific
`.onepkg` compatibility.
Trailing CAB data is ignored with a warning. Reserved CAB areas are bounded and Trailing CAB data is ignored with a warning. Reserved CAB areas are bounded and
skipped. CAB names are decoded as strict UTF-8; unmarked valid UTF-8 receives a skipped. CAB names are decoded as strict UTF-8; unmarked valid UTF-8 receives a
warning, while invalid non-ASCII legacy-code-page names are rejected rather warning, while invalid non-ASCII legacy-code-page names are rejected rather

View File

@@ -13,8 +13,9 @@ The desktop `.one` reader adapts selected structures from the pinned
The browser-native package path has separate MIT provenance: The browser-native package path has separate MIT provenance:
| TypeScript module | Upstream path(s) | Revision and selected licence | Port scope and deviations | | TypeScript module | Upstream path(s) | Revision and selected licence | Port scope and deviations |
| -------------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | -------------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `src/onenote/onepkg/{cabinet,checksum}.ts` | rust-cab `src/{cabinet,folder,file,ctype,checksum}.rs` | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2`, MIT | CAB reading/checksum adapted for immutable browser bytes, strict normalized paths, checked ranges/checksums, whole-folder extraction, explicit limits, and cancellation/yield hooks; writers and MSZIP are not ported | | `src/onenote/onepkg/{cabinet,checksum}.ts` | rust-cab `src/{cabinet,folder,file,ctype,checksum}.rs` | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2`, MIT | CAB reading/checksum adapted for immutable browser bytes, strict normalized paths, checked ranges/checksums, whole-folder extraction, explicit limits, and cancellation/yield hooks; writers are not ported |
| `src/onenote/onepkg/mszip/decoder.ts` | rust-cab `src/mszip.rs`; fflate raw DEFLATE | rust-cab revision above; fflate 0.8.3, MIT | `CK` framing and persistent 32 KiB folder history follow rust-cab; raw DEFLATE uses fflate with a caller-owned bounded buffer and strict declared-size checks |
| `src/onenote/onepkg/lzx/{bit-reader,huffman,decoder}.ts` | lzxd 0.2.5 `src/{bitstream,tree,block,window,lib}.rs` | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd`, MIT option of MIT OR Apache-2.0 | Directly translated core with typed arrays, CAB CFDATA boundaries, persistent folder state, strict malformed-stream errors, and operation/window/cancellation limits; delta reference data is omitted | | `src/onenote/onepkg/lzx/{bit-reader,huffman,decoder}.ts` | lzxd 0.2.5 `src/{bitstream,tree,block,window,lib}.rs` | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd`, MIT option of MIT OR Apache-2.0 | Directly translated core with typed arrays, CAB CFDATA boundaries, persistent folder state, strict malformed-stream errors, and operation/window/cancellation limits; delta reference data is omitted |
The bounded binary readers, format detection, parser limits/error model, path The bounded binary readers, format detection, parser limits/error model, path

View File

@@ -3,11 +3,11 @@
Version 0.1.0 records these immutable implementation revisions: Version 0.1.0 records these immutable implementation revisions:
| Repository | Exact revision | Use | | Repository | Exact revision | Use |
| ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------- | | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------- |
| [`toolbox-sdk`](https://git.add-ideas.de/zemion/toolbox-sdk/src/commit/5e206baf42dfae5a00f2743fb2cc501bb12d9b7a) | `5e206baf42dfae5a00f2743fb2cc501bb12d9b7a` | App contract, React shell, manifest schema/testkit, and nested-path behavior | | [`toolbox-sdk`](https://git.add-ideas.de/zemion/toolbox-sdk/src/commit/5e206baf42dfae5a00f2743fb2cc501bb12d9b7a) | `5e206baf42dfae5a00f2743fb2cc501bb12d9b7a` | App contract, React shell, manifest schema/testkit, and nested-path behavior |
| [`toolbox-portal`](https://git.add-ideas.de/zemion/toolbox-portal/src/commit/5d2e466ad61e36a0e2be2a2d8459b1a127f53f11) | `5d2e466ad61e36a0e2be2a2d8459b1a127f53f11` | Reviewed ZIP shape, exact checksum lock, and independent-app assembly model | | [`toolbox-portal`](https://git.add-ideas.de/zemion/toolbox-portal/src/commit/5d2e466ad61e36a0e2be2a2d8459b1a127f53f11) | `5d2e466ad61e36a0e2be2a2d8459b1a127f53f11` | Reviewed ZIP shape, exact checksum lock, and independent-app assembly model |
| [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop OneStore and MS-ONE structures adapted into TypeScript | | [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop OneStore and MS-ONE structures adapted into TypeScript |
| [`rust-cab`](https://github.com/mdsteele/rust-cab/tree/c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2) | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2` | CAB structures, checksum, enumeration/extraction, and two unit fixtures | | [`rust-cab`](https://github.com/mdsteele/rust-cab/tree/c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2) | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2` | CAB structures, checksum, enumeration/extraction, MSZIP history, and unit fixtures |
| [`lzxd` 0.2.5](https://github.com/Lonami/lzxd/tree/4748e43594e3e30cff2ace3a6ad7a376c9816fdd) | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd` | LZX bitstream, Huffman, block/window, match, and E8 logic adapted into TypeScript | | [`lzxd` 0.2.5](https://github.com/Lonami/lzxd/tree/4748e43594e3e30cff2ace3a6ad7a376c9816fdd) | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd` | LZX bitstream, Huffman, block/window, match, and E8 logic adapted into TypeScript |
The SDK and Portal revisions were inspected from the local repositories. The The SDK and Portal revisions were inspected from the local repositories. The
@@ -32,6 +32,9 @@ truth and the pinned projects as implementation references:
- [[MS-CAB]: Cabinet File Format](https://download.microsoft.com/download/4/d/a/4da14f27-b4ef-4170-a6e6-5b1ef85b1baa/%5Bms-cab%5D.pdf), - [[MS-CAB]: Cabinet File Format](https://download.microsoft.com/download/4/d/a/4da14f27-b4ef-4170-a6e6-5b1ef85b1baa/%5Bms-cab%5D.pdf),
sections 2.1-2.4 (`CFHEADER`, `CFFOLDER`, `CFFILE`, and `CFDATA`) and 3.1 sections 2.1-2.4 (`CFHEADER`, `CFFOLDER`, `CFFILE`, and `CFDATA`) and 3.1
(checksum method). (checksum method).
- [[MS-MCI]: Compression and Decompression Data Structure](https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-mci/3048c471-f739-4d30-bcb3-a962cfe84a69),
MSZIP block framing, raw DEFLATE streams, 32 KiB block/history behavior, and
compressed-size bounds.
- [[MS-PATCH]: LZX Delta Compression and Decompression](https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-patch/cc78752a-b4af-4eee-88cb-01f4d8a4c2bf), - [[MS-PATCH]: LZX Delta Compression and Decompression](https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-patch/cc78752a-b4af-4eee-88cb-01f4d8a4c2bf),
sections 2.1-2.7 (bitstream/window concepts, header/E8 translation, blocks, sections 2.1-2.7 (bitstream/window concepts, header/E8 translation, blocks,
Huffman trees, pretree encoding, tokens, and decoding). `lzxd` 0.2.5 states Huffman trees, pretree encoding, tokens, and decoding). `lzxd` 0.2.5 states

View File

@@ -57,6 +57,7 @@ The CAB/package path applies these defaults:
| Total folder output | 512 MiB | | Total folder output | 512 MiB |
| Total extracted entry bytes | 512 MiB | | Total extracted entry bytes | 512 MiB |
| Folder compression ratio | 200:1 | | Folder compression ratio | 200:1 |
| One MSZIP compressed block | 32 KiB + 12 B |
| LZX window | 8 MiB | | LZX window | 8 MiB |
| LZX decode operations | 1,000,000,000 | | LZX decode operations | 1,000,000,000 |
@@ -66,11 +67,13 @@ checksums, rejects overlapping/out-of-range data, rejects absolute/traversal,
empty, control-character, and case-insensitive duplicate paths, and refuses empty, control-character, and case-insensitive duplicate paths, and refuses
multi-cabinet/spanning entries. multi-cabinet/spanning entries.
LZX checks its window, Huffman trees, block sizes, match ranges, output size, MSZIP checks its `CK` framing and raw DEFLATE data, writes into a fixed caller
and operation budget. CAB extraction checks cancellation and yields between buffer with one overflow-detection byte, requires the exact declared output
data blocks so the worker can process termination. Desktop OneStore traversal size, and retains no more than 32 KiB of history. LZX checks its window, Huffman
checks fragment/reference cycles, nesting, object/property/reference counts, trees, block sizes, match ranges, output size, and operation budget. CAB
and byte ranges. extraction checks cancellation and yields between data blocks so the worker can
process termination. Desktop OneStore traversal checks fragment/reference
cycles, nesting, object/property/reference counts, and byte ranges.
## Rendering and links ## Rendering and links

3
package-lock.json generated
View File

@@ -11,6 +11,7 @@
"dependencies": { "dependencies": {
"@add-ideas/toolbox-contract": "0.1.1", "@add-ideas/toolbox-contract": "0.1.1",
"@add-ideas/toolbox-shell-react": "0.1.1", "@add-ideas/toolbox-shell-react": "0.1.1",
"fflate": "^0.8.3",
"react": "^19.2.6", "react": "^19.2.6",
"react-dom": "^19.2.6" "react-dom": "^19.2.6"
}, },
@@ -28,7 +29,6 @@
"eslint-config-prettier": "^10.1.8", "eslint-config-prettier": "^10.1.8",
"eslint-plugin-react-hooks": "^7.1.1", "eslint-plugin-react-hooks": "^7.1.1",
"eslint-plugin-react-refresh": "^0.5.2", "eslint-plugin-react-refresh": "^0.5.2",
"fflate": "^0.8.3",
"globals": "^17.6.0", "globals": "^17.6.0",
"jsdom": "^29.1.1", "jsdom": "^29.1.1",
"prettier": "^3.8.3", "prettier": "^3.8.3",
@@ -2342,7 +2342,6 @@
"version": "0.8.3", "version": "0.8.3",
"resolved": "https://registry.npmjs.org/fflate/-/fflate-0.8.3.tgz", "resolved": "https://registry.npmjs.org/fflate/-/fflate-0.8.3.tgz",
"integrity": "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==", "integrity": "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==",
"dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/file-entry-cache": { "node_modules/file-entry-cache": {

View File

@@ -29,6 +29,7 @@
"dependencies": { "dependencies": {
"@add-ideas/toolbox-contract": "0.1.1", "@add-ideas/toolbox-contract": "0.1.1",
"@add-ideas/toolbox-shell-react": "0.1.1", "@add-ideas/toolbox-shell-react": "0.1.1",
"fflate": "^0.8.3",
"react": "^19.2.6", "react": "^19.2.6",
"react-dom": "^19.2.6" "react-dom": "^19.2.6"
}, },
@@ -46,7 +47,6 @@
"eslint-config-prettier": "^10.1.8", "eslint-config-prettier": "^10.1.8",
"eslint-plugin-react-hooks": "^7.1.1", "eslint-plugin-react-hooks": "^7.1.1",
"eslint-plugin-react-refresh": "^0.5.2", "eslint-plugin-react-refresh": "^0.5.2",
"fflate": "^0.8.3",
"globals": "^17.6.0", "globals": "^17.6.0",
"jsdom": "^29.1.1", "jsdom": "^29.1.1",
"prettier": "^3.8.3", "prettier": "^3.8.3",

View File

@@ -39,6 +39,7 @@ export async function collectReleaseMetadataFiles(projectRoot) {
await createThirdPartyLicenseBundle(projectRoot, [ await createThirdPartyLicenseBundle(projectRoot, [
'@add-ideas/toolbox-contract', '@add-ideas/toolbox-contract',
'@add-ideas/toolbox-shell-react', '@add-ideas/toolbox-shell-react',
'fflate',
'react', 'react',
'react-dom', 'react-dom',
'scheduler', 'scheduler',

View File

@@ -1,9 +1,12 @@
import { createHash } from 'node:crypto';
import { describe, expect, it } from 'vitest'; import { describe, expect, it } from 'vitest';
import type { OneNoteSectionDto } from '../model/dto.js'; import type { OneNoteSectionDto } from '../model/dto.js';
import { enumerateCabinet, extractCabinet } from './cabinet.js'; import { enumerateCabinet, extractCabinet } from './cabinet.js';
import { OnePkgError } from './errors.js'; import { OnePkgError } from './errors.js';
import { LzxDecoder } from './lzx/decoder.js'; import { LzxDecoder } from './lzx/decoder.js';
import { MsZipDecoder } from './mszip/decoder.js';
import { openOneNotePackage } from './package.js'; import { openOneNotePackage } from './package.js';
/* /*
@@ -33,6 +36,37 @@ const TWO_FILE_UNCOMPRESSED_CAB = binary(
'\0\0\0\0\x1d\0\x1d\0Hello, world!\nSee you later!\n' '\0\0\0\0\x1d\0\x1d\0Hello, world!\nSee you later!\n'
); );
const MSZIP_CAB = binary(
'MSCF\0\0\0\0\x61\0\0\0\0\0\0\0' +
'\x2c\0\0\0\0\0\0\0\x03\x01\x01\0\x01\0\0\0\x34\x12\0\0' +
'\x43\0\0\0\x01\0\x01\0' +
'\x0e\0\0\0\0\0\0\0\0\0\x6c\x22\xe7\x59\x01\0hi.txt\0' +
'\0\0\0\0\x16\0\x0e\0' +
'CK\xf3H\xcd\xc9\xc9\xd7Q(\xcf/\xcaIQ\xe4\x02\x00$\xf2\x04\x94'
);
/*
* Generated two-block CAB; generation details and hashes are recorded in
* tests/fixtures/README.md. Its second raw DEFLATE stream requires the first
* block's 32 KiB history.
*/
const MULTI_BLOCK_MSZIP_CAB = Uint8Array.from(
Buffer.from(
'TVNDRgAAAABRAgAAAAAAACwAAAAAAAAAAwEBAAEAAADvvgAASAAAAAIAAQAAkAAAAAAAAAAA' +
'AAAAACAAaGlzdG9yeS5iaW4Ak4QnWtcBAIBDS+3P1yIQAABAURQtRIpK2Tu7oU1SqRCS7BAa' +
'2iVZFdo0aBDaGVGRVLaWEYq21RSiRNmeevIX9/zBERAUGjZcWGTEyFGjx4iKiY+VkBwnNX6C' +
'tMzESZNlp0yVk1dQVFJWUVVT19DUmqato6unb2A4fcbMWUaz58ydN3/BQmOTRaaLzZYsXWa+' +
'fMVKC0urVdY2tqvt1tivdXB0cnZxdVvn7uG53svbZ8PGTZt9t2zdtn3Hzl279/jt9d8XEBgU' +
'HLL/wMHQsPBDh48cPXb8RETkyVOnz0RFnz13/kJM7MW4+IRLl69cvXb9xs3EpOSUW6lpt+/c' +
'Tc+4l3k/68HDR9k5uXn5BYVFj588ffa8uKS07EV5ReXLV1XVr9+8fff+w8ea2rr6hk+fv3z9' +
'9r3xR1Nzy8/Wtl+/2/90dP7919Xd09vXPzAoQJ06derUqVOnTp06derUqVOnTp06derUqVOn' +
'Tp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOn' +
'Tp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOn' +
'Tp06derUqQ/V/wM/Q7GFIgAAEENL7c+BDAAAAMAgf+t7fGWQurq6urq6urq6urq6urr6vx4=',
'base64'
)
);
const LZX_CAB = binary( const LZX_CAB = binary(
'\x4d\x53\x43\x46\x00\x00\x00\x00\x97\x00\x00\x00\x00\x00\x00' + '\x4d\x53\x43\x46\x00\x00\x00\x00\x97\x00\x00\x00\x00\x00\x00' +
'\x00\x2c\x00\x00\x00\x00\x00\x00\x00\x03\x01\x01\x00\x02\x00' + '\x00\x2c\x00\x00\x00\x00\x00\x00\x00\x03\x01\x01\x00\x02\x00' +
@@ -109,6 +143,89 @@ describe('CAB enumeration and extraction', () => {
]); ]);
}); });
it('extracts the rust-cab single-block MSZIP fixture', async () => {
const listing = enumerateCabinet(MSZIP_CAB);
expect(listing.folders[0]!.compression).toEqual({
kind: 'mszip',
raw: 1,
label: 'MSZIP',
});
expect(listing.diagnostics).not.toEqual(
expect.arrayContaining([
expect.objectContaining({ code: 'cab-compression-unsupported' }),
])
);
const extracted = await extractCabinet(MSZIP_CAB);
expect(new TextDecoder().decode(extracted.extractedEntries[0]!.bytes)).toBe(
'Hello, world!\n'
);
});
it('retains the MSZIP dictionary across two CFDATA blocks', async () => {
expect(MULTI_BLOCK_MSZIP_CAB).toHaveLength(593);
expect(
createHash('sha256').update(MULTI_BLOCK_MSZIP_CAB).digest('hex')
).toBe('9f71b97623395492f401ddc152ca648e8416996b14092b0df42ad712c27f9cb2');
const listing = enumerateCabinet(MULTI_BLOCK_MSZIP_CAB);
expect(listing.folders[0]).toMatchObject({
dataBlockCount: 2,
compressedSize: 505,
uncompressedSize: 36_864,
compression: { kind: 'mszip' },
});
const extracted = await extractCabinet(MULTI_BLOCK_MSZIP_CAB);
const output = extracted.extractedEntries[0]!.bytes;
expect(output).toHaveLength(36_864);
expect(createHash('sha256').update(output).digest('hex')).toBe(
'cb441d1f6e7d9c5789a306c51a00a654c9b7715c2c3d624e3d2a75bd0af549ff'
);
expect(output).toEqual(
Uint8Array.from({ length: 36_864 }, (_, index) => index % 251)
);
const firstBlock = MULTI_BLOCK_MSZIP_CAB.subarray(80, 551);
const secondBlock = MULTI_BLOCK_MSZIP_CAB.subarray(559, 593);
const decoder = new MsZipDecoder();
expect(decoder.decompressNext(firstBlock, 32_768)).toEqual(
output.subarray(0, 32_768)
);
expect(decoder.decompressNext(secondBlock, 4_096)).toEqual(
output.subarray(32_768)
);
expectCode(
() => new MsZipDecoder().decompressNext(secondBlock, 4_096),
'mszip-invalid-deflate'
);
});
it('rejects malformed MSZIP framing, DEFLATE, and output sizes', async () => {
const signature = MSZIP_CAB.slice();
const signatureOffset = findAscii(signature, 'CK');
signature[signatureOffset] = 0;
await expect(extractCabinet(signature)).rejects.toMatchObject({
diagnostic: { code: 'mszip-invalid-signature' },
});
const deflate = MSZIP_CAB.slice();
deflate[findAscii(deflate, 'CK') + 2] = 0x07;
await expect(extractCabinet(deflate)).rejects.toMatchObject({
diagnostic: { code: 'mszip-invalid-deflate' },
});
const wrongOutputSize = MSZIP_CAB.slice();
const outputSizeOffset = findAscii(wrongOutputSize, 'CK') - 2;
new DataView(
wrongOutputSize.buffer,
wrongOutputSize.byteOffset,
wrongOutputSize.byteLength
).setUint16(outputSizeOffset, 15, true);
await expect(extractCabinet(wrongOutputSize)).rejects.toMatchObject({
diagnostic: { code: 'mszip-output-size-mismatch' },
});
});
it('retains an uncompressed LZX block across CAB chunks', () => { it('retains an uncompressed LZX block across CAB chunks', () => {
const decoder = new LzxDecoder(18, { maxOperations: 100 }); const decoder = new LzxDecoder(18, { maxOperations: 100 });
const first = binary( const first = binary(

View File

@@ -13,6 +13,7 @@ import { checkCancellation, yieldForCancellation } from './cancellation.js';
import { calculateDataBlockChecksum } from './checksum.js'; import { calculateDataBlockChecksum } from './checksum.js';
import { fail, warning } from './errors.js'; import { fail, warning } from './errors.js';
import { LzxDecoder } from './lzx/decoder.js'; import { LzxDecoder } from './lzx/decoder.js';
import { MsZipDecoder } from './mszip/decoder.js';
import { decodeCabFileName, normalizeCabPath } from './path.js'; import { decodeCabFileName, normalizeCabPath } from './path.js';
import { import {
resolveLimits, resolveLimits,
@@ -30,6 +31,7 @@ const FLAG_NEXT_CABINET = 0x0002;
const FLAG_RESERVE_PRESENT = 0x0004; const FLAG_RESERVE_PRESENT = 0x0004;
const ATTRIBUTE_NAME_IS_UTF8 = 0x0080; const ATTRIBUTE_NAME_IS_UTF8 = 0x0080;
const MAX_CAB_BLOCK_OUTPUT = 32 * 1024; const MAX_CAB_BLOCK_OUTPUT = 32 * 1024;
const MAX_MSZIP_BLOCK_INPUT = 32 * 1024 + 12;
interface CabDataBlock { interface CabDataBlock {
headerOffset: number; headerOffset: number;
@@ -458,6 +460,16 @@ function parseCabinet(
{ offset: headerOffset + 4, structure: 'CFDATA' } { offset: headerOffset + 4, structure: 'CFDATA' }
); );
} }
if (
folder.compression.kind === 'mszip' &&
compressedSize > MAX_MSZIP_BLOCK_INPUT
) {
fail(
'cab-invalid-mszip-block-size',
`MSZIP CAB data block size ${compressedSize} exceeds ${MAX_MSZIP_BLOCK_INPUT} bytes.`,
{ offset: headerOffset + 4, structure: 'CFDATA.cbData' }
);
}
if ( if (
folder.compression.kind === 'none' && folder.compression.kind === 'none' &&
compressedSize !== uncompressedSize compressedSize !== uncompressedSize
@@ -589,10 +601,7 @@ function parseCabinet(
); );
} }
for (const folder of folders) { for (const folder of folders) {
if ( if (folder.compression.kind === 'quantum') {
folder.compression.kind === 'mszip' ||
folder.compression.kind === 'quantum'
) {
diagnostics.push( diagnostics.push(
warning( warning(
'cab-compression-unsupported', 'cab-compression-unsupported',
@@ -646,10 +655,7 @@ export async function extractCabinet(
const extractedEntries = []; const extractedEntries = [];
for (const folder of parsed.folders) { for (const folder of parsed.folders) {
checkCancellation(options.cancellation); checkCancellation(options.cancellation);
if ( if (folder.compression.kind === 'quantum') {
folder.compression.kind === 'mszip' ||
folder.compression.kind === 'quantum'
) {
fail( fail(
'cab-compression-unsupported', 'cab-compression-unsupported',
`${folder.compression.label} extraction is not implemented.`, `${folder.compression.label} extraction is not implemented.`,
@@ -665,6 +671,8 @@ export async function extractCabinet(
checkCancellation: () => checkCancellation(options.cancellation), checkCancellation: () => checkCancellation(options.cancellation),
}) })
: undefined; : undefined;
const msZipDecoder =
folder.compression.kind === 'mszip' ? new MsZipDecoder() : undefined;
let outputOffset = 0; let outputOffset = 0;
for (const block of folder.blocks) { for (const block of folder.blocks) {
checkCancellation(options.cancellation); checkCancellation(options.cancellation);
@@ -688,9 +696,11 @@ export async function extractCabinet(
} }
} }
const output = const output =
decoder === undefined decoder !== undefined
? compressed ? decoder.decompressNext(compressed, block.uncompressedSize)
: decoder.decompressNext(compressed, block.uncompressedSize); : msZipDecoder !== undefined
? msZipDecoder.decompressNext(compressed, block.uncompressedSize)
: compressed;
if (output.length !== block.uncompressedSize) { if (output.length !== block.uncompressedSize) {
fail( fail(
'cab-output-size-mismatch', 'cab-output-size-mismatch',

View File

@@ -0,0 +1,95 @@
/*
* SPDX-License-Identifier: MIT
*
* MSZIP block framing and 32 KiB history semantics follow rust-cab 0.6.0.
* rust-cab commit c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2
* Copyright (c) 2017 Matthew D. Steele, MIT License.
* Raw DEFLATE decoding is provided by fflate 0.8.3 (MIT).
* Modified for fixed browser buffers and strict block/output validation.
*/
import { inflateSync } from 'fflate';
import { fail } from '../errors.js';
const MAX_BLOCK_OUTPUT = 32 * 1024;
const HISTORY_SIZE = 32 * 1024;
const SIGNATURE_FIRST = 0x43;
const SIGNATURE_SECOND = 0x4b;
export class MsZipDecoder {
private history = new Uint8Array(0);
decompressNext(chunk: Uint8Array, outputLength: number): Uint8Array {
if (
!Number.isInteger(outputLength) ||
outputLength <= 0 ||
outputLength > MAX_BLOCK_OUTPUT
) {
fail(
'mszip-invalid-chunk-size',
`MSZIP CAB chunk output size ${outputLength} is outside 1..${MAX_BLOCK_OUTPUT}.`,
{ structure: 'CFDATA.cbUncomp' }
);
}
if (
chunk.length < 2 ||
chunk[0] !== SIGNATURE_FIRST ||
chunk[1] !== SIGNATURE_SECOND
) {
fail(
'mszip-invalid-signature',
'MSZIP data block does not begin with the CK signature.',
{ structure: 'MSZIP block' }
);
}
let output: Uint8Array;
try {
// One extra byte lets us distinguish an exact result from a stream that
// expands beyond the trusted CFDATA declaration. fflate writes into this
// caller-owned buffer, so malformed input cannot trigger output growth.
output = inflateSync(chunk.subarray(2), {
dictionary: this.history.length === 0 ? undefined : this.history,
out: new Uint8Array(outputLength + 1),
});
} catch {
fail(
'mszip-invalid-deflate',
'MSZIP block contains an invalid raw DEFLATE stream.',
{ structure: 'MSZIP block' }
);
}
if (output.length !== outputLength) {
fail(
'mszip-output-size-mismatch',
`MSZIP block produced ${output.length} bytes; expected ${outputLength}.`,
{ structure: 'CFDATA.cbUncomp' }
);
}
this.updateHistory(output);
return output;
}
private updateHistory(output: Uint8Array): void {
const historyLength = Math.min(
HISTORY_SIZE,
this.history.length + output.length
);
const nextHistory = new Uint8Array(historyLength);
const oldBytes = Math.min(
this.history.length,
historyLength - output.length
);
if (oldBytes > 0) {
nextHistory.set(this.history.subarray(this.history.length - oldBytes));
}
nextHistory.set(
output.subarray(output.length - (historyLength - oldBytes)),
oldBytes
);
this.history = nextHistory;
}
}

View File

@@ -47,9 +47,33 @@ They are MIT-licensed; the exact upstream text is in
`LICENSES/rust-cab-MIT.txt`. `LICENSES/rust-cab-MIT.txt`.
The arrays cover an uncompressed single-entry CAB, an uncompressed two-entry The arrays cover an uncompressed single-entry CAB, an uncompressed two-entry
CAB, and an LZX `0x1303` CAB containing `hi.txt` and `bye.txt`. They are CAB CAB, a single-block MSZIP CAB, and an LZX `0x1303` CAB containing `hi.txt` and
unit fixtures, not OneNote exports, and are not evidence of `.onepkg` producer `bye.txt`. They are CAB unit fixtures, not OneNote exports, and are not evidence
compatibility. of `.onepkg` producer compatibility.
## Inline generated two-block MSZIP fixture
`src/onenote/onepkg/cabinet.test.ts` also contains a generated 593-byte CAB
with one 36,864-byte `history.bin` entry. The input byte at index `i` is
`i % 251`. It is split into a 32,768-byte block and a 4,096-byte block; the
second raw DEFLATE stream uses the first block as its preset 32 KiB dictionary,
so decoding it without folder history fails.
- Generator: CPython 3.13.14 `zlib.compressobj`, zlib 1.3.1, raw DEFLATE
(`wbits=-15`), level 9; the second call uses `zdict=first_block`. Each stream
is prefixed with `CK` and placed in a spec-shaped single-folder CAB with CAB
checksums.
- Generator invocation: `python3` with the documented deterministic input and
CAB layout above; the frozen Base64 bytes, sizes, checksums, and hashes are
asserted by the test. No notebook or third-party content is an input.
- CAB SHA-256:
`9f71b97623395492f401ddc152ca648e8416996b14092b0df42ad712c27f9cb2`.
- Extracted SHA-256:
`cb441d1f6e7d9c5789a306c51a00a654c9b7715c2c3d624e3d2a75bd0af549ff`.
- Licence: original test data under this project's MPL-2.0 licence.
This fixture establishes block-history behavior only. It is not a OneNote
export and is not evidence of `.onepkg` producer compatibility.
## Distribution policy ## Distribution policy

View File

@@ -94,6 +94,9 @@ describe('release archive', () => {
expect( expect(
createHash('sha256').update(files['LICENSES/lzxd-MIT.txt']!).digest('hex') createHash('sha256').update(files['LICENSES/lzxd-MIT.txt']!).digest('hex')
).toBe('c7a1dcbf839b1dfb72f24e1de12ab55f789db2c61d894ca9aa8b5f595490d9ca'); ).toBe('c7a1dcbf839b1dfb72f24e1de12ab55f789db2c61d894ca9aa8b5f595490d9ca');
expect(
files['LICENSES/THIRD-PARTY-LICENSES.txt']!.toString('utf8')
).toContain('fflate@0.8.3');
expect(files).not.toHaveProperty( expect(files).not.toHaveProperty(
'tests/fixtures/joplin-test.onepkg.base64' 'tests/fixtures/joplin-test.onepkg.base64'
); );