diff --git a/CHANGELOG.md b/CHANGELOG.md index 2a4a56e..4150404 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,17 +8,18 @@ - Add a worker-hosted native TypeScript parser for a desktop OneNote revision-store subset: bounded file-node and object-space traversal, property sets, page metadata, titles, timestamps, levels, and plain text. -- Add in-memory CAB enumeration and extraction for uncompressed and LZX +- Add in-memory CAB enumeration and extraction for uncompressed, MSZIP, and LZX folders, including checksums, strict paths, explicit resource limits, and cooperative cancellation between data blocks. - Open contained `.one` sections independently and expose package entries, section/page navigation, partial failures, and structured diagnostics. - Verify a Joplin `.one` golden with three pages and a Joplin LZX `0x1203` `.onepkg` golden with seven entries, five sections, and seven pages. Also - verify rust-cab uncompressed and LZX `0x1303` fixtures. + verify rust-cab uncompressed, MSZIP, and LZX `0x1303` fixtures plus a + generated two-block MSZIP dictionary-chain fixture. - Add deterministic static release packaging, SHA-256 sidecars, pinned-source provenance, fixture licensing, and security/support documentation. -Known unsupported areas are FSSHTTP-packaged `.one`, MSZIP and Quantum CAB -extraction, multi-cabinet archives, `.onetoc2` hierarchy/order interpretation, -visual content, attachments, layout/format fidelity, links, and exports. +Known unsupported areas are FSSHTTP-packaged `.one`, Quantum CAB extraction, +multi-cabinet archives, `.onetoc2` hierarchy/order interpretation, visual +content, attachments, layout/format fidelity, links, and exports. diff --git a/README.md b/README.md index 0839089..0823cf0 100644 --- a/README.md +++ b/README.md @@ -14,25 +14,28 @@ and is delivered to Toolbox Portal as an already-built, checksummed static ZIP. - Desktop revision-store `.one` sections from the tested OneNote family. The reader exposes section/page names, stable page IDs, created and updated timestamps when present, page level, and readable plain text. -- Single-cabinet `.onepkg` files whose CAB folders are uncompressed or LZX - compressed. Each contained desktop `.one` section is parsed independently, - so one unsupported section does not hide sections that did parse. +- Single-cabinet `.onepkg` files whose CAB folders are uncompressed, MSZIP, or + LZX compressed. Each contained desktop `.one` section is parsed + independently, so one unsupported section does not hide sections that did + parse. - Package entries, safe normalized paths, uncompressed sizes, compression metadata, section/page navigation, and structured diagnostics. The golden tests parse a real Joplin desktop `.one` file with three pages and a real Joplin `.onepkg` with seven entries, five sections, and seven pages. LZX -`0x1203` and `0x1303` data and an uncompressed CAB are covered by fixtures. +`0x1203` and `0x1303`, uncompressed CAB, and one- and two-block MSZIP data are +covered by fixtures. The MSZIP fixtures are CAB-level evidence, not a claim +about a particular OneNote `.onepkg` producer. This is a structural reader, not a full-fidelity OneNote renderer. Images, tables, ink, embedded files, layout, formatting, links, and exports are not -rendered. FSSHTTP-packaged `.one`, MSZIP and Quantum CAB extraction, -multi-cabinet archives, and `.onetoc2` hierarchy/order interpretation are not -supported. See [format support](docs/FORMAT_SUPPORT.md) for the precise matrix. +rendered. FSSHTTP-packaged `.one`, Quantum CAB extraction, multi-cabinet +archives, and `.onetoc2` hierarchy/order interpretation are not supported. See +[format support](docs/FORMAT_SUPPORT.md) for the precise matrix. There is no Rust/Wasm runtime, server, proxy, upload, telemetry, or notebook -storage. All parsing and CAB/LZX decompression are ordinary browser JavaScript -produced from the TypeScript source. +storage. All parsing and CAB/LZX/MSZIP decompression is ordinary browser +JavaScript produced from the TypeScript source. ## Use diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 5e23d07..7546b4f 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -4,6 +4,7 @@ The production application uses: - `@add-ideas/toolbox-contract` 0.1.1, Apache-2.0; - `@add-ideas/toolbox-shell-react` 0.1.1, Apache-2.0; +- fflate 0.8.3, MIT; - React, React DOM, and Scheduler, MIT. The release packager collects the complete installed runtime licence texts into @@ -22,6 +23,9 @@ adaptation of `lzxd` 0.2.5 (MIT OR Apache-2.0), pinned at option. Adapted modules carry provenance headers and the licence texts are in `LICENSES/rust-cab-MIT.txt` and `LICENSES/lzxd-MIT.txt`. +MSZIP raw DEFLATE decoding uses fflate 0.8.3 (MIT). Its complete installed +licence text is included in the generated runtime licence bundle. + The Base64-encoded `onenote_desktop.one` test fixture comes from Joplin at commit `28eb53bd9f0cd4ae0dec0c59a08037075a38302b` and is distributed under AGPL-3.0-or-later. Its licence text is in `LICENSES/AGPL-3.0.txt`. diff --git a/docs/FORMAT_SUPPORT.md b/docs/FORMAT_SUPPORT.md index 513365a..c2143c8 100644 --- a/docs/FORMAT_SUPPORT.md +++ b/docs/FORMAT_SUPPORT.md @@ -10,7 +10,7 @@ | `.onepkg`, single CAB, LZX | Supported subset | Real Joplin package: LZX `0x1203`, three CFDATA blocks, seven entries, five sections, seven pages | | CAB LZX `0x1303` | Supported extraction | Byte-for-byte rust-cab MIT unit fixture, two text entries | | CAB without compression | Supported extraction | Byte-for-byte rust-cab MIT unit fixture | -| CAB MSZIP | Unsupported | Metadata is recognized; extraction fails with `cab-compression-unsupported` | +| CAB MSZIP | Supported extraction | rust-cab single-block fixture plus a generated two-block dictionary-chain fixture | | CAB Quantum | Unsupported | Metadata is recognized; extraction fails with `cab-compression-unsupported` | | Multi-cabinet CAB / spanning files | Unsupported | Header flags and spanning folder indexes are rejected | | `.onetoc2` package entries | Listed only | Files are identified, but notebook hierarchy and section order are not interpreted | @@ -24,6 +24,12 @@ a controlled failure. The default LZX limit accepts CAB window sizes through 8 MiB, but compatibility claims are fixture-backed only for window bits 18 (`0x1203`) and 19 (`0x1303`). +MSZIP validates the `CK` prefix, limits each compressed block to 32 KiB + 12, +decodes into a fixed-size output buffer, requires the declared output length, +and carries at most 32 KiB of history between blocks in one folder. The MSZIP +tests establish CAB extraction behavior; no redistributable MSZIP-compressed +OneNote export is currently present, so they do not establish producer-specific +`.onepkg` compatibility. Trailing CAB data is ignored with a warning. Reserved CAB areas are bounded and skipped. CAB names are decoded as strict UTF-8; unmarked valid UTF-8 receives a warning, while invalid non-ASCII legacy-code-page names are rejected rather diff --git a/docs/PORTING.md b/docs/PORTING.md index c75f66f..7b6b123 100644 --- a/docs/PORTING.md +++ b/docs/PORTING.md @@ -12,10 +12,11 @@ The desktop `.one` reader adapts selected structures from the pinned The browser-native package path has separate MIT provenance: -| TypeScript module | Upstream path(s) | Revision and selected licence | Port scope and deviations | -| -------------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `src/onenote/onepkg/{cabinet,checksum}.ts` | rust-cab `src/{cabinet,folder,file,ctype,checksum}.rs` | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2`, MIT | CAB reading/checksum adapted for immutable browser bytes, strict normalized paths, checked ranges/checksums, whole-folder extraction, explicit limits, and cancellation/yield hooks; writers and MSZIP are not ported | -| `src/onenote/onepkg/lzx/{bit-reader,huffman,decoder}.ts` | lzxd 0.2.5 `src/{bitstream,tree,block,window,lib}.rs` | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd`, MIT option of MIT OR Apache-2.0 | Directly translated core with typed arrays, CAB CFDATA boundaries, persistent folder state, strict malformed-stream errors, and operation/window/cancellation limits; delta reference data is omitted | +| TypeScript module | Upstream path(s) | Revision and selected licence | Port scope and deviations | +| -------------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `src/onenote/onepkg/{cabinet,checksum}.ts` | rust-cab `src/{cabinet,folder,file,ctype,checksum}.rs` | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2`, MIT | CAB reading/checksum adapted for immutable browser bytes, strict normalized paths, checked ranges/checksums, whole-folder extraction, explicit limits, and cancellation/yield hooks; writers are not ported | +| `src/onenote/onepkg/mszip/decoder.ts` | rust-cab `src/mszip.rs`; fflate raw DEFLATE | rust-cab revision above; fflate 0.8.3, MIT | `CK` framing and persistent 32 KiB folder history follow rust-cab; raw DEFLATE uses fflate with a caller-owned bounded buffer and strict declared-size checks | +| `src/onenote/onepkg/lzx/{bit-reader,huffman,decoder}.ts` | lzxd 0.2.5 `src/{bitstream,tree,block,window,lib}.rs` | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd`, MIT option of MIT OR Apache-2.0 | Directly translated core with typed arrays, CAB CFDATA boundaries, persistent folder state, strict malformed-stream errors, and operation/window/cancellation limits; delta reference data is omitted | The bounded binary readers, format detection, parser limits/error model, path policy, worker protocol/DTOs, package-to-section boundary, and application UI diff --git a/docs/REFERENCE_IMPLEMENTATIONS.md b/docs/REFERENCE_IMPLEMENTATIONS.md index a5094da..d107e8f 100644 --- a/docs/REFERENCE_IMPLEMENTATIONS.md +++ b/docs/REFERENCE_IMPLEMENTATIONS.md @@ -2,13 +2,13 @@ Version 0.1.0 records these immutable implementation revisions: -| Repository | Exact revision | Use | -| ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------- | -| [`toolbox-sdk`](https://git.add-ideas.de/zemion/toolbox-sdk/src/commit/5e206baf42dfae5a00f2743fb2cc501bb12d9b7a) | `5e206baf42dfae5a00f2743fb2cc501bb12d9b7a` | App contract, React shell, manifest schema/testkit, and nested-path behavior | -| [`toolbox-portal`](https://git.add-ideas.de/zemion/toolbox-portal/src/commit/5d2e466ad61e36a0e2be2a2d8459b1a127f53f11) | `5d2e466ad61e36a0e2be2a2d8459b1a127f53f11` | Reviewed ZIP shape, exact checksum lock, and independent-app assembly model | -| [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop OneStore and MS-ONE structures adapted into TypeScript | -| [`rust-cab`](https://github.com/mdsteele/rust-cab/tree/c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2) | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2` | CAB structures, checksum, enumeration/extraction, and two unit fixtures | -| [`lzxd` 0.2.5](https://github.com/Lonami/lzxd/tree/4748e43594e3e30cff2ace3a6ad7a376c9816fdd) | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd` | LZX bitstream, Huffman, block/window, match, and E8 logic adapted into TypeScript | +| Repository | Exact revision | Use | +| ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------- | +| [`toolbox-sdk`](https://git.add-ideas.de/zemion/toolbox-sdk/src/commit/5e206baf42dfae5a00f2743fb2cc501bb12d9b7a) | `5e206baf42dfae5a00f2743fb2cc501bb12d9b7a` | App contract, React shell, manifest schema/testkit, and nested-path behavior | +| [`toolbox-portal`](https://git.add-ideas.de/zemion/toolbox-portal/src/commit/5d2e466ad61e36a0e2be2a2d8459b1a127f53f11) | `5d2e466ad61e36a0e2be2a2d8459b1a127f53f11` | Reviewed ZIP shape, exact checksum lock, and independent-app assembly model | +| [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop OneStore and MS-ONE structures adapted into TypeScript | +| [`rust-cab`](https://github.com/mdsteele/rust-cab/tree/c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2) | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2` | CAB structures, checksum, enumeration/extraction, MSZIP history, and unit fixtures | +| [`lzxd` 0.2.5](https://github.com/Lonami/lzxd/tree/4748e43594e3e30cff2ace3a6ad7a376c9816fdd) | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd` | LZX bitstream, Huffman, block/window, match, and E8 logic adapted into TypeScript | The SDK and Portal revisions were inspected from the local repositories. The three parser references were cloned and reviewed at the hashes above; a branch @@ -32,6 +32,9 @@ truth and the pinned projects as implementation references: - [[MS-CAB]: Cabinet File Format](https://download.microsoft.com/download/4/d/a/4da14f27-b4ef-4170-a6e6-5b1ef85b1baa/%5Bms-cab%5D.pdf), sections 2.1-2.4 (`CFHEADER`, `CFFOLDER`, `CFFILE`, and `CFDATA`) and 3.1 (checksum method). +- [[MS-MCI]: Compression and Decompression Data Structure](https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-mci/3048c471-f739-4d30-bcb3-a962cfe84a69), + MSZIP block framing, raw DEFLATE streams, 32 KiB block/history behavior, and + compressed-size bounds. - [[MS-PATCH]: LZX Delta Compression and Decompression](https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-patch/cc78752a-b4af-4eee-88cb-01f4d8a4c2bf), sections 2.1-2.7 (bitstream/window concepts, header/E8 translation, blocks, Huffman trees, pretree encoding, tokens, and decoding). `lzxd` 0.2.5 states diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 141ba76..fb72fd9 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -57,6 +57,7 @@ The CAB/package path applies these defaults: | Total folder output | 512 MiB | | Total extracted entry bytes | 512 MiB | | Folder compression ratio | 200:1 | +| One MSZIP compressed block | 32 KiB + 12 B | | LZX window | 8 MiB | | LZX decode operations | 1,000,000,000 | @@ -66,11 +67,13 @@ checksums, rejects overlapping/out-of-range data, rejects absolute/traversal, empty, control-character, and case-insensitive duplicate paths, and refuses multi-cabinet/spanning entries. -LZX checks its window, Huffman trees, block sizes, match ranges, output size, -and operation budget. CAB extraction checks cancellation and yields between -data blocks so the worker can process termination. Desktop OneStore traversal -checks fragment/reference cycles, nesting, object/property/reference counts, -and byte ranges. +MSZIP checks its `CK` framing and raw DEFLATE data, writes into a fixed caller +buffer with one overflow-detection byte, requires the exact declared output +size, and retains no more than 32 KiB of history. LZX checks its window, Huffman +trees, block sizes, match ranges, output size, and operation budget. CAB +extraction checks cancellation and yields between data blocks so the worker can +process termination. Desktop OneStore traversal checks fragment/reference +cycles, nesting, object/property/reference counts, and byte ranges. ## Rendering and links diff --git a/package-lock.json b/package-lock.json index 4e40d13..229286e 100644 --- a/package-lock.json +++ b/package-lock.json @@ -11,6 +11,7 @@ "dependencies": { "@add-ideas/toolbox-contract": "0.1.1", "@add-ideas/toolbox-shell-react": "0.1.1", + "fflate": "^0.8.3", "react": "^19.2.6", "react-dom": "^19.2.6" }, @@ -28,7 +29,6 @@ "eslint-config-prettier": "^10.1.8", "eslint-plugin-react-hooks": "^7.1.1", "eslint-plugin-react-refresh": "^0.5.2", - "fflate": "^0.8.3", "globals": "^17.6.0", "jsdom": "^29.1.1", "prettier": "^3.8.3", @@ -2342,7 +2342,6 @@ "version": "0.8.3", "resolved": "https://registry.npmjs.org/fflate/-/fflate-0.8.3.tgz", "integrity": "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==", - "dev": true, "license": "MIT" }, "node_modules/file-entry-cache": { diff --git a/package.json b/package.json index 22ea2f8..66cc28a 100644 --- a/package.json +++ b/package.json @@ -29,6 +29,7 @@ "dependencies": { "@add-ideas/toolbox-contract": "0.1.1", "@add-ideas/toolbox-shell-react": "0.1.1", + "fflate": "^0.8.3", "react": "^19.2.6", "react-dom": "^19.2.6" }, @@ -46,7 +47,6 @@ "eslint-config-prettier": "^10.1.8", "eslint-plugin-react-hooks": "^7.1.1", "eslint-plugin-react-refresh": "^0.5.2", - "fflate": "^0.8.3", "globals": "^17.6.0", "jsdom": "^29.1.1", "prettier": "^3.8.3", diff --git a/scripts/package-release.mjs b/scripts/package-release.mjs index d1bf2e0..29dc5ed 100644 --- a/scripts/package-release.mjs +++ b/scripts/package-release.mjs @@ -39,6 +39,7 @@ export async function collectReleaseMetadataFiles(projectRoot) { await createThirdPartyLicenseBundle(projectRoot, [ '@add-ideas/toolbox-contract', '@add-ideas/toolbox-shell-react', + 'fflate', 'react', 'react-dom', 'scheduler', diff --git a/src/onenote/onepkg/cabinet.test.ts b/src/onenote/onepkg/cabinet.test.ts index d4bb09d..1f90f70 100644 --- a/src/onenote/onepkg/cabinet.test.ts +++ b/src/onenote/onepkg/cabinet.test.ts @@ -1,9 +1,12 @@ +import { createHash } from 'node:crypto'; + import { describe, expect, it } from 'vitest'; import type { OneNoteSectionDto } from '../model/dto.js'; import { enumerateCabinet, extractCabinet } from './cabinet.js'; import { OnePkgError } from './errors.js'; import { LzxDecoder } from './lzx/decoder.js'; +import { MsZipDecoder } from './mszip/decoder.js'; import { openOneNotePackage } from './package.js'; /* @@ -33,6 +36,37 @@ const TWO_FILE_UNCOMPRESSED_CAB = binary( '\0\0\0\0\x1d\0\x1d\0Hello, world!\nSee you later!\n' ); +const MSZIP_CAB = binary( + 'MSCF\0\0\0\0\x61\0\0\0\0\0\0\0' + + '\x2c\0\0\0\0\0\0\0\x03\x01\x01\0\x01\0\0\0\x34\x12\0\0' + + '\x43\0\0\0\x01\0\x01\0' + + '\x0e\0\0\0\0\0\0\0\0\0\x6c\x22\xe7\x59\x01\0hi.txt\0' + + '\0\0\0\0\x16\0\x0e\0' + + 'CK\xf3H\xcd\xc9\xc9\xd7Q(\xcf/\xcaIQ\xe4\x02\x00$\xf2\x04\x94' +); + +/* + * Generated two-block CAB; generation details and hashes are recorded in + * tests/fixtures/README.md. Its second raw DEFLATE stream requires the first + * block's 32 KiB history. + */ +const MULTI_BLOCK_MSZIP_CAB = Uint8Array.from( + Buffer.from( + 'TVNDRgAAAABRAgAAAAAAACwAAAAAAAAAAwEBAAEAAADvvgAASAAAAAIAAQAAkAAAAAAAAAAA' + + 'AAAAACAAaGlzdG9yeS5iaW4Ak4QnWtcBAIBDS+3P1yIQAABAURQtRIpK2Tu7oU1SqRCS7BAa' + + '2iVZFdo0aBDaGVGRVLaWEYq21RSiRNmeevIX9/zBERAUGjZcWGTEyFGjx4iKiY+VkBwnNX6C' + + 'tMzESZNlp0yVk1dQVFJWUVVT19DUmqato6unb2A4fcbMWUaz58ydN3/BQmOTRaaLzZYsXWa+' + + 'fMVKC0urVdY2tqvt1tivdXB0cnZxdVvn7uG53svbZ8PGTZt9t2zdtn3Hzl279/jt9d8XEBgU' + + 'HLL/wMHQsPBDh48cPXb8RETkyVOnz0RFnz13/kJM7MW4+IRLl69cvXb9xs3EpOSUW6lpt+/c' + + 'Tc+4l3k/68HDR9k5uXn5BYVFj588ffa8uKS07EV5ReXLV1XVr9+8fff+w8ea2rr6hk+fv3z9' + + '9r3xR1Nzy8/Wtl+/2/90dP7919Xd09vXPzAoQJ06derUqVOnTp06derUqVOnTp06derUqVOn' + + 'Tp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOn' + + 'Tp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOnTp06derUqVOn' + + 'Tp06derUqQ/V/wM/Q7GFIgAAEENL7c+BDAAAAMAgf+t7fGWQurq6urq6urq6urq6urr6vx4=', + 'base64' + ) +); + const LZX_CAB = binary( '\x4d\x53\x43\x46\x00\x00\x00\x00\x97\x00\x00\x00\x00\x00\x00' + '\x00\x2c\x00\x00\x00\x00\x00\x00\x00\x03\x01\x01\x00\x02\x00' + @@ -109,6 +143,89 @@ describe('CAB enumeration and extraction', () => { ]); }); + it('extracts the rust-cab single-block MSZIP fixture', async () => { + const listing = enumerateCabinet(MSZIP_CAB); + expect(listing.folders[0]!.compression).toEqual({ + kind: 'mszip', + raw: 1, + label: 'MSZIP', + }); + expect(listing.diagnostics).not.toEqual( + expect.arrayContaining([ + expect.objectContaining({ code: 'cab-compression-unsupported' }), + ]) + ); + + const extracted = await extractCabinet(MSZIP_CAB); + expect(new TextDecoder().decode(extracted.extractedEntries[0]!.bytes)).toBe( + 'Hello, world!\n' + ); + }); + + it('retains the MSZIP dictionary across two CFDATA blocks', async () => { + expect(MULTI_BLOCK_MSZIP_CAB).toHaveLength(593); + expect( + createHash('sha256').update(MULTI_BLOCK_MSZIP_CAB).digest('hex') + ).toBe('9f71b97623395492f401ddc152ca648e8416996b14092b0df42ad712c27f9cb2'); + const listing = enumerateCabinet(MULTI_BLOCK_MSZIP_CAB); + expect(listing.folders[0]).toMatchObject({ + dataBlockCount: 2, + compressedSize: 505, + uncompressedSize: 36_864, + compression: { kind: 'mszip' }, + }); + + const extracted = await extractCabinet(MULTI_BLOCK_MSZIP_CAB); + const output = extracted.extractedEntries[0]!.bytes; + expect(output).toHaveLength(36_864); + expect(createHash('sha256').update(output).digest('hex')).toBe( + 'cb441d1f6e7d9c5789a306c51a00a654c9b7715c2c3d624e3d2a75bd0af549ff' + ); + expect(output).toEqual( + Uint8Array.from({ length: 36_864 }, (_, index) => index % 251) + ); + + const firstBlock = MULTI_BLOCK_MSZIP_CAB.subarray(80, 551); + const secondBlock = MULTI_BLOCK_MSZIP_CAB.subarray(559, 593); + const decoder = new MsZipDecoder(); + expect(decoder.decompressNext(firstBlock, 32_768)).toEqual( + output.subarray(0, 32_768) + ); + expect(decoder.decompressNext(secondBlock, 4_096)).toEqual( + output.subarray(32_768) + ); + expectCode( + () => new MsZipDecoder().decompressNext(secondBlock, 4_096), + 'mszip-invalid-deflate' + ); + }); + + it('rejects malformed MSZIP framing, DEFLATE, and output sizes', async () => { + const signature = MSZIP_CAB.slice(); + const signatureOffset = findAscii(signature, 'CK'); + signature[signatureOffset] = 0; + await expect(extractCabinet(signature)).rejects.toMatchObject({ + diagnostic: { code: 'mszip-invalid-signature' }, + }); + + const deflate = MSZIP_CAB.slice(); + deflate[findAscii(deflate, 'CK') + 2] = 0x07; + await expect(extractCabinet(deflate)).rejects.toMatchObject({ + diagnostic: { code: 'mszip-invalid-deflate' }, + }); + + const wrongOutputSize = MSZIP_CAB.slice(); + const outputSizeOffset = findAscii(wrongOutputSize, 'CK') - 2; + new DataView( + wrongOutputSize.buffer, + wrongOutputSize.byteOffset, + wrongOutputSize.byteLength + ).setUint16(outputSizeOffset, 15, true); + await expect(extractCabinet(wrongOutputSize)).rejects.toMatchObject({ + diagnostic: { code: 'mszip-output-size-mismatch' }, + }); + }); + it('retains an uncompressed LZX block across CAB chunks', () => { const decoder = new LzxDecoder(18, { maxOperations: 100 }); const first = binary( diff --git a/src/onenote/onepkg/cabinet.ts b/src/onenote/onepkg/cabinet.ts index 5c297bd..3fc9199 100644 --- a/src/onenote/onepkg/cabinet.ts +++ b/src/onenote/onepkg/cabinet.ts @@ -13,6 +13,7 @@ import { checkCancellation, yieldForCancellation } from './cancellation.js'; import { calculateDataBlockChecksum } from './checksum.js'; import { fail, warning } from './errors.js'; import { LzxDecoder } from './lzx/decoder.js'; +import { MsZipDecoder } from './mszip/decoder.js'; import { decodeCabFileName, normalizeCabPath } from './path.js'; import { resolveLimits, @@ -30,6 +31,7 @@ const FLAG_NEXT_CABINET = 0x0002; const FLAG_RESERVE_PRESENT = 0x0004; const ATTRIBUTE_NAME_IS_UTF8 = 0x0080; const MAX_CAB_BLOCK_OUTPUT = 32 * 1024; +const MAX_MSZIP_BLOCK_INPUT = 32 * 1024 + 12; interface CabDataBlock { headerOffset: number; @@ -458,6 +460,16 @@ function parseCabinet( { offset: headerOffset + 4, structure: 'CFDATA' } ); } + if ( + folder.compression.kind === 'mszip' && + compressedSize > MAX_MSZIP_BLOCK_INPUT + ) { + fail( + 'cab-invalid-mszip-block-size', + `MSZIP CAB data block size ${compressedSize} exceeds ${MAX_MSZIP_BLOCK_INPUT} bytes.`, + { offset: headerOffset + 4, structure: 'CFDATA.cbData' } + ); + } if ( folder.compression.kind === 'none' && compressedSize !== uncompressedSize @@ -589,10 +601,7 @@ function parseCabinet( ); } for (const folder of folders) { - if ( - folder.compression.kind === 'mszip' || - folder.compression.kind === 'quantum' - ) { + if (folder.compression.kind === 'quantum') { diagnostics.push( warning( 'cab-compression-unsupported', @@ -646,10 +655,7 @@ export async function extractCabinet( const extractedEntries = []; for (const folder of parsed.folders) { checkCancellation(options.cancellation); - if ( - folder.compression.kind === 'mszip' || - folder.compression.kind === 'quantum' - ) { + if (folder.compression.kind === 'quantum') { fail( 'cab-compression-unsupported', `${folder.compression.label} extraction is not implemented.`, @@ -665,6 +671,8 @@ export async function extractCabinet( checkCancellation: () => checkCancellation(options.cancellation), }) : undefined; + const msZipDecoder = + folder.compression.kind === 'mszip' ? new MsZipDecoder() : undefined; let outputOffset = 0; for (const block of folder.blocks) { checkCancellation(options.cancellation); @@ -688,9 +696,11 @@ export async function extractCabinet( } } const output = - decoder === undefined - ? compressed - : decoder.decompressNext(compressed, block.uncompressedSize); + decoder !== undefined + ? decoder.decompressNext(compressed, block.uncompressedSize) + : msZipDecoder !== undefined + ? msZipDecoder.decompressNext(compressed, block.uncompressedSize) + : compressed; if (output.length !== block.uncompressedSize) { fail( 'cab-output-size-mismatch', diff --git a/src/onenote/onepkg/mszip/decoder.ts b/src/onenote/onepkg/mszip/decoder.ts new file mode 100644 index 0000000..6f200c8 --- /dev/null +++ b/src/onenote/onepkg/mszip/decoder.ts @@ -0,0 +1,95 @@ +/* + * SPDX-License-Identifier: MIT + * + * MSZIP block framing and 32 KiB history semantics follow rust-cab 0.6.0. + * rust-cab commit c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2 + * Copyright (c) 2017 Matthew D. Steele, MIT License. + * Raw DEFLATE decoding is provided by fflate 0.8.3 (MIT). + * Modified for fixed browser buffers and strict block/output validation. + */ + +import { inflateSync } from 'fflate'; + +import { fail } from '../errors.js'; + +const MAX_BLOCK_OUTPUT = 32 * 1024; +const HISTORY_SIZE = 32 * 1024; +const SIGNATURE_FIRST = 0x43; +const SIGNATURE_SECOND = 0x4b; + +export class MsZipDecoder { + private history = new Uint8Array(0); + + decompressNext(chunk: Uint8Array, outputLength: number): Uint8Array { + if ( + !Number.isInteger(outputLength) || + outputLength <= 0 || + outputLength > MAX_BLOCK_OUTPUT + ) { + fail( + 'mszip-invalid-chunk-size', + `MSZIP CAB chunk output size ${outputLength} is outside 1..${MAX_BLOCK_OUTPUT}.`, + { structure: 'CFDATA.cbUncomp' } + ); + } + if ( + chunk.length < 2 || + chunk[0] !== SIGNATURE_FIRST || + chunk[1] !== SIGNATURE_SECOND + ) { + fail( + 'mszip-invalid-signature', + 'MSZIP data block does not begin with the CK signature.', + { structure: 'MSZIP block' } + ); + } + + let output: Uint8Array; + try { + // One extra byte lets us distinguish an exact result from a stream that + // expands beyond the trusted CFDATA declaration. fflate writes into this + // caller-owned buffer, so malformed input cannot trigger output growth. + output = inflateSync(chunk.subarray(2), { + dictionary: this.history.length === 0 ? undefined : this.history, + out: new Uint8Array(outputLength + 1), + }); + } catch { + fail( + 'mszip-invalid-deflate', + 'MSZIP block contains an invalid raw DEFLATE stream.', + { structure: 'MSZIP block' } + ); + } + + if (output.length !== outputLength) { + fail( + 'mszip-output-size-mismatch', + `MSZIP block produced ${output.length} bytes; expected ${outputLength}.`, + { structure: 'CFDATA.cbUncomp' } + ); + } + + this.updateHistory(output); + return output; + } + + private updateHistory(output: Uint8Array): void { + const historyLength = Math.min( + HISTORY_SIZE, + this.history.length + output.length + ); + const nextHistory = new Uint8Array(historyLength); + const oldBytes = Math.min( + this.history.length, + historyLength - output.length + ); + if (oldBytes > 0) { + nextHistory.set(this.history.subarray(this.history.length - oldBytes)); + } + nextHistory.set( + output.subarray(output.length - (historyLength - oldBytes)), + oldBytes + ); + this.history = nextHistory; + } +} diff --git a/tests/fixtures/README.md b/tests/fixtures/README.md index b7db36a..dd3d33a 100644 --- a/tests/fixtures/README.md +++ b/tests/fixtures/README.md @@ -47,9 +47,33 @@ They are MIT-licensed; the exact upstream text is in `LICENSES/rust-cab-MIT.txt`. The arrays cover an uncompressed single-entry CAB, an uncompressed two-entry -CAB, and an LZX `0x1303` CAB containing `hi.txt` and `bye.txt`. They are CAB -unit fixtures, not OneNote exports, and are not evidence of `.onepkg` producer -compatibility. +CAB, a single-block MSZIP CAB, and an LZX `0x1303` CAB containing `hi.txt` and +`bye.txt`. They are CAB unit fixtures, not OneNote exports, and are not evidence +of `.onepkg` producer compatibility. + +## Inline generated two-block MSZIP fixture + +`src/onenote/onepkg/cabinet.test.ts` also contains a generated 593-byte CAB +with one 36,864-byte `history.bin` entry. The input byte at index `i` is +`i % 251`. It is split into a 32,768-byte block and a 4,096-byte block; the +second raw DEFLATE stream uses the first block as its preset 32 KiB dictionary, +so decoding it without folder history fails. + +- Generator: CPython 3.13.14 `zlib.compressobj`, zlib 1.3.1, raw DEFLATE + (`wbits=-15`), level 9; the second call uses `zdict=first_block`. Each stream + is prefixed with `CK` and placed in a spec-shaped single-folder CAB with CAB + checksums. +- Generator invocation: `python3` with the documented deterministic input and + CAB layout above; the frozen Base64 bytes, sizes, checksums, and hashes are + asserted by the test. No notebook or third-party content is an input. +- CAB SHA-256: + `9f71b97623395492f401ddc152ca648e8416996b14092b0df42ad712c27f9cb2`. +- Extracted SHA-256: + `cb441d1f6e7d9c5789a306c51a00a654c9b7715c2c3d624e3d2a75bd0af549ff`. +- Licence: original test data under this project's MPL-2.0 licence. + +This fixture establishes block-history behavior only. It is not a OneNote +export and is not evidence of `.onepkg` producer compatibility. ## Distribution policy diff --git a/tests/release-archive.test.ts b/tests/release-archive.test.ts index 86eaa50..37d61b2 100644 --- a/tests/release-archive.test.ts +++ b/tests/release-archive.test.ts @@ -94,6 +94,9 @@ describe('release archive', () => { expect( createHash('sha256').update(files['LICENSES/lzxd-MIT.txt']!).digest('hex') ).toBe('c7a1dcbf839b1dfb72f24e1de12ab55f789db2c61d894ca9aa8b5f595490d9ca'); + expect( + files['LICENSES/THIRD-PARTY-LICENSES.txt']!.toString('utf8') + ).toContain('fflate@0.8.3'); expect(files).not.toHaveProperty( 'tests/fixtures/joplin-test.onepkg.base64' );