diff --git a/CHANGELOG.md b/CHANGELOG.md index 4980918..a90cf0a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,9 @@ - Add a worker-hosted native TypeScript parser for a desktop OneNote revision-store subset: bounded file-node and object-space traversal, property sets, page metadata, titles, timestamps, levels, and plain text. +- Add bounded FSSHTTPB packaging, storage-index/cell/revision/object-group + parsing and open unfragmented package-store `.one` sections through the same + content model. - Add in-memory CAB enumeration and extraction for uncompressed, MSZIP, Quantum, and LZX folders, including checksums, strict paths, explicit resource limits, and cooperative cancellation between data blocks. @@ -21,6 +24,5 @@ - Add deterministic static release packaging, SHA-256 sidecars, pinned-source provenance, fixture licensing, and security/support documentation. -Known unsupported areas are FSSHTTP-packaged `.one`, multi-cabinet archives, -`.onetoc2` hierarchy/order interpretation, visual content, attachments, -layout/format fidelity, links, and exports. +Known unsupported areas include fragmented FSSHTTPB data-element +reconstruction and untested OneNote producer variants. diff --git a/README.md b/README.md index c113b60..5eadfe6 100644 --- a/README.md +++ b/README.md @@ -14,15 +14,19 @@ and is delivered to Toolbox Portal as an already-built, checksummed static ZIP. - Desktop revision-store `.one` sections from the tested OneNote family. The reader exposes section/page names, stable page IDs, created and updated timestamps when present, page level, and readable plain text. +- Unfragmented FSSHTTPB-packaged `.one` sections matching the tested Joplin + family. Package cells, revision chains, object groups, and resolved object + and page-space references feed the same bounded content reader. - Single-cabinet `.onepkg` files whose CAB folders are uncompressed, MSZIP, - Quantum, or LZX compressed. Each contained desktop `.one` section is parsed - independently, so one unsupported section does not hide sections that did - parse. + Quantum, or LZX compressed. Each contained supported `.one` section is + parsed independently, so one unsupported section does not hide sections that + did parse. - Package entries, safe normalized paths, uncompressed sizes, compression metadata, section/page navigation, and structured diagnostics. -The golden tests parse a real Joplin desktop `.one` file with three pages and a -real Joplin `.onepkg` with seven entries, five sections, and seven pages. LZX +The golden tests parse a real Joplin desktop `.one` file with three pages, an +unfragmented 17,197-byte Joplin FSSHTTPB `.one` file with one page, and a real +Joplin `.onepkg` with seven entries, five sections, and seven pages. LZX `0x1203` and `0x1303`, uncompressed CAB, one- and two-block MSZIP data, and single- and multi-frame Quantum data are covered by fixtures. The MSZIP and Quantum fixtures are CAB-level evidence, not a claim about a particular @@ -30,8 +34,8 @@ OneNote `.onepkg` producer. This is a structural reader, not a full-fidelity OneNote renderer. Images, tables, ink, embedded files, layout, formatting, links, and exports are not -rendered. FSSHTTP-packaged `.one`, multi-cabinet archives, and `.onetoc2` -hierarchy/order interpretation are not supported. See +rendered. Fragmented FSSHTTPB data elements and some producer variants remain +unsupported. See [format support](docs/FORMAT_SUPPORT.md) for the precise matrix. There is no Rust/Wasm runtime, server, proxy, upload, telemetry, or notebook diff --git a/docs/FORMAT_SUPPORT.md b/docs/FORMAT_SUPPORT.md index 605adf1..4182658 100644 --- a/docs/FORMAT_SUPPORT.md +++ b/docs/FORMAT_SUPPORT.md @@ -5,7 +5,7 @@ | Input or feature | Status | Tested evidence and boundary | | ---------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------- | | Desktop revision-store `.one` | Supported subset | Real 34,904-byte Joplin golden; three pages | -| FSSHTTP-packaged `.one` | Unsupported | Header is detected and rejected with a controlled diagnostic | +| FSSHTTPB-packaged `.one` | Supported subset | Licensed 17,197-byte Joplin golden; one page; unfragmented data elements | | Unknown/legacy `.one` variants | Unsupported | File type and format GUIDs are validated before traversal | | `.onepkg`, single CAB, LZX | Supported subset | Real Joplin package: LZX `0x1203`, three CFDATA blocks, seven entries, five sections, seven pages | | CAB LZX `0x1303` | Supported extraction | Byte-for-byte rust-cab MIT unit fixture, two text entries | @@ -17,10 +17,19 @@ | Standalone `.onetoc2` | Unsupported in the app | The picker accepts only `.one` and `.onepkg` | “Supported subset” is intentionally narrower than “all files with this -extension.” A successful open requires the tested desktop OneStore structures -and content graph nodes. Malformed offsets, counts, references, CAB checksums, -paths, compressed data, and unsupported variants produce bounded diagnostics or -a controlled failure. +extension.” A successful open requires the tested desktop or FSSHTTPB OneStore +structures and content graph nodes. Malformed offsets, counts, references, CAB +checksums, paths, compressed data, and unsupported variants produce bounded +diagnostics or a controlled failure. + +FSSHTTPB support covers the OneStore packaging envelope, Storage Index and +Storage Manifest, header/data-root cells, bounded revision chains, Object Group +partitions, resolved object/page-space references, and available Object Data +BLOB references. Data Element Fragment records are recognized and bounded by +the low-level parser, but fragmented-element reconstruction is not implemented; +the store adapter rejects packages containing fragment records with a +controlled `UNSUPPORTED_FORMAT` error. Excluded object data cannot be +reconstructed from a standalone file and is likewise unsupported. The default LZX limit accepts CAB window sizes through 8 MiB, but compatibility claims are fixture-backed only for window bits 18 (`0x1203`) and 19 (`0x1303`). @@ -45,7 +54,7 @@ than guessed. ## Extracted OneNote data -The desktop section path currently returns: +The supported section paths currently return: - section display name; - page ID, title, created/updated timestamp when available, and page level; diff --git a/docs/PORTING.md b/docs/PORTING.md index 1fea08e..52f4ec5 100644 --- a/docs/PORTING.md +++ b/docs/PORTING.md @@ -3,12 +3,14 @@ The desktop `.one` reader adapts selected structures from the pinned `onenote.rs` implementation. It is not a line-for-line or complete port. -| TypeScript module | Upstream path(s) | Revision | Port scope and deviations | -| ---------------------------------------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -| `src/onenote/onestore/file-node.ts` | `onestore/desktop/file_structure/file_node_list_fragment.rs`, `file_node/*`, chunk references | `onenote.rs@5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected; immutable byte input, strict range checks, fragment/child cycle detection, and depth/count limits | -| `src/onenote/onestore/property-set.ts` | `onestore/shared/{object_prop_set,prop_set,property,object_stream_header}.rs` | same | Substantial; bounded vectors/counts/nesting and serializable discriminated unions | -| `src/onenote/onestore/desktop-store.ts` | `onestore/desktop/{file_node,file_structure,objects}/*` | same | Selected desktop section path; explicit missing-reference errors; file-data/attachment payloads skipped with diagnostics | -| `src/onenote/one/{constants,section}.ts` | `one/property*`, `one/property_set/*`, `onenote/{section,page,page_series,outline,rich_text}.rs` | same | Page metadata, titles, and plain text only; bounded graph traversal; visual/embedded content diagnosed | +| TypeScript module | Upstream path(s) | Revision | Port scope and deviations | +| ---------------------------------------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| `src/onenote/onestore/file-node.ts` | `onestore/desktop/file_structure/file_node_list_fragment.rs`, `file_node/*`, chunk references | `onenote.rs@5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected; immutable byte input, strict range checks, fragment/child cycle detection, and depth/count limits | +| `src/onenote/onestore/property-set.ts` | `onestore/shared/{object_prop_set,prop_set,property,object_stream_header}.rs` | same | Substantial; bounded vectors/counts/nesting and serializable discriminated unions | +| `src/onenote/onestore/desktop-store.ts` | `onestore/desktop/{file_node,file_structure,objects}/*` | same | Selected desktop section path; explicit missing-reference errors; file-data/attachment payloads skipped with diagnostics | +| `src/onenote/fsshttpb/*` | `fsshttpb/{data,data_element,packaging}.rs` | same | Native bounded package/header/data-element records, checked compact integers, zero-copy BLOB ranges; fragment records parsed but not assembled | +| `src/onenote/onestore/fsshttp-store.ts` | `onestore/fsshttpb/{header,mapping_table,object,object_space,revision,revision_role}.rs` | same | Unfragmented package-store adapter; bounded/cycle-checked revision chains, newest-revision precedence, resolved positional reference arrays | +| `src/onenote/one/{constants,section}.ts` | `one/property*`, `one/property_set/*`, `onenote/{section,page,page_series,outline,rich_text}.rs` | same | Page metadata, titles, and plain text only; bounded graph traversal; visual/embedded content diagnosed | The browser-native package path has separate MIT provenance: diff --git a/docs/REFERENCE_IMPLEMENTATIONS.md b/docs/REFERENCE_IMPLEMENTATIONS.md index b948ae7..d649428 100644 --- a/docs/REFERENCE_IMPLEMENTATIONS.md +++ b/docs/REFERENCE_IMPLEMENTATIONS.md @@ -6,7 +6,7 @@ Version 0.1.0 records these immutable implementation revisions: | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------- | | [`toolbox-sdk`](https://git.add-ideas.de/zemion/toolbox-sdk/src/commit/5e206baf42dfae5a00f2743fb2cc501bb12d9b7a) | `5e206baf42dfae5a00f2743fb2cc501bb12d9b7a` | App contract, React shell, manifest schema/testkit, and nested-path behavior | | [`toolbox-portal`](https://git.add-ideas.de/zemion/toolbox-portal/src/commit/5d2e466ad61e36a0e2be2a2d8459b1a127f53f11) | `5d2e466ad61e36a0e2be2a2d8459b1a127f53f11` | Reviewed ZIP shape, exact checksum lock, and independent-app assembly model | -| [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop OneStore and MS-ONE structures adapted into TypeScript | +| [`onenote.rs`](https://github.com/msiemens/onenote.rs/tree/5138a39a3f4e72b840932f9872fecde52fa9da60) | `5138a39a3f4e72b840932f9872fecde52fa9da60` | Selected desktop/FSSHTTPB OneStore and MS-ONE structures adapted into TypeScript | | [`rust-cab`](https://github.com/mdsteele/rust-cab/tree/c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2) | `c5839f5fdfa4c4e7cc9b22f570c79c96af0560e2` | CAB structures, checksum, enumeration/extraction, MSZIP history, and unit fixtures | | [`compcol`](https://github.com/KarpelesLab/compcol/tree/04a6db2aa7bd487a89c559631d79d1b384139f50) | `04a6db2aa7bd487a89c559631d79d1b384139f50` | Quantum bit reader, arithmetic models, LZ window, CAB sentinel behavior, and tests | | [`lzxd` 0.2.5](https://github.com/Lonami/lzxd/tree/4748e43594e3e30cff2ace3a6ad7a376c9816fdd) | `4748e43594e3e30cff2ace3a6ad7a376c9816fdd` | LZX bitstream, Huffman, block/window, match, and E8 logic adapted into TypeScript | @@ -24,7 +24,11 @@ truth and the pinned projects as implementation references: - [[MS-ONESTORE]: OneNote Revision Store File Format](https://learn.microsoft.com/en-us/openspecs/office_file_formats/ms-onestore/ae670cd2-4b38-4b24-82d1-87cfb2cc3725), particularly sections 2.2.4 (chunk references), 2.3.1 (header), 2.4 (file-node lists and fragments), 2.5.1-2.5.8 (object-space/revision - manifests), and 2.6.7 (property sets). + manifests), 2.6.7 (property sets), 2.7 (FSSHTTPB package-store objects), and + 2.8.1 (OneStore Packaging Structure). +- [[MS-FSSHTTPB]: Binary Requests for File Synchronization via SOAP Protocol](https://learn.microsoft.com/en-us/openspecs/sharepoint_protocols/ms-fsshttpb/99a25464-99b5-4262-a964-baabed2170eb), + particularly the stream-object headers/end markers and section 2.2.1.12 Data + Element Package structures used by the OneStore package envelope. - [[MS-ONE]: OneNote File Format](https://learn.microsoft.com/en-us/openspecs/office_file_formats/ms-one/73d22548-a613-4350-8c23-07d15576be50), particularly sections 2.1.8-2.1.13 (root objects, page object spaces, and property sets), 2.2.17-2.2.23 (section/page-series/outline/rich-text nodes), diff --git a/docs/SECURITY.md b/docs/SECURITY.md index caf6478..407dcc4 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -26,7 +26,7 @@ The application does not: ## Active default limits The main thread rejects a source larger than 512 MiB before reading it. The -desktop section parser then applies these independent defaults: +OneStore section parsers then apply these independent defaults: | Resource | Default limit | | ------------------------------------- | -------------------: | @@ -36,6 +36,7 @@ desktop section parser then applies these independent defaults: | Objects | 500,000 | | Properties per set / property nesting | 16,384 / 64 | | Stream IDs | 1,000,000 | +| FSSHTTPB revision-chain entries | 16,384 | | One property vector | 64 MiB | | Object references | 1,000,000 | | Content-graph depth | 256 | @@ -78,7 +79,12 @@ most the current and next compressed block so a synthetic `0xFF` sentinel can be consumed without buffering a complete folder. CAB extraction checks cancellation during Quantum work and yields between data blocks so the worker can process termination. Desktop OneStore traversal checks fragment/reference -cycles, nesting, object/property/reference counts, and byte ranges. +cycles, nesting, object/property/reference counts, and byte ranges. FSSHTTPB +parsing bounds every declared stream body and compact integer, rejects invalid +end markers and non-zero package tails, checks mapping/reference counts and +revision cycles, and retains BLOB payloads as bounded source ranges. Fragment +records are range-checked, but packages containing them are rejected because +fragmented-element reconstruction is not implemented. ## Rendering and links @@ -89,7 +95,7 @@ explicit user action, scheme validation, and safe navigation attributes. ## Known gaps and roadmap -- There is no wall-clock deadline. Synchronous desktop `.one` parsing cannot +- There is no wall-clock deadline. Synchronous OneStore `.one` parsing cannot observe cooperative cancellation until control returns; terminating the worker is the current hard stop. - The package flow holds the source, decompressed folder buffers, extracted