data-tools/docs/ARCHITECTURE.md

Architecture

Data Tools is a relocatable static React application. There is no application server and no runtime API.

Pipeline

1. The UI checks the local file size before decoding and passes text, the selected format and the optional filename to a fresh module worker.
2. The worker applies byte/character and structural preflight limits, including a streaming dimension pass before delimited parsing, then parses with format-specific defensive options and converts the result into a serialisable typed node model.
3. The model is measured again for node count and depth before it is returned. The shared worker-job protocol filters results by job ID, terminates a superseded worker and enforces a 30-second hard deadline. The UI retains the last successful model until a new one is ready.
4. Tree, table, flattened and query projections operate on the model with their own preview/result caps.
5. Serialisers create a new string and a structured list of notices, coercions and losses. Output is capped before it reaches copy or download actions.
6. Schema inference samples the same bounded model. Structural edits clone only the addressed path, remeasure the result, then serialize it through the source adapter for review before source replacement.

The node model has explicit null, boolean, string, number, date, array and ordered-object variants. Number nodes retain a source representation (exact, bigint or binary) and an exact display lexeme. Objects use ordered entry arrays rather than assigning untrusted keys to ordinary JavaScript objects.

XML deliberately does not pretend to be a JSON object. It is represented as explicit element records containing qualified name, namespace URI, attributes and ordered child records. Converting XML to another syntax therefore exposes this model and reports the convention.

Parsers

Runtime versions are pinned exactly: lossless-json 4.3.1, yaml 2.9.0, smol-toml 1.8.0, @xmldom/xmldom 0.9.12 and papaparse 5.7.0. No imported value is rendered as HTML or executed.

Bounded data-model and parser functions remain pure app modules because exact scalar spelling and XML namespace metadata are domain decisions. Generic byte-size display, safe downloads and cancellable worker lifecycle come from @add-ideas/toolbox-helpers 0.2.0.

src/core/adapters.ts is the public ordered capability catalogue. parse.ts and convert.ts expose concrete registries that must cover every descriptor; tests make missing import/export implementations fail visibly. schema.ts never turns exact decimal lexemes into binary range constraints. edit.ts implements immutable RFC 6901 traversal and rejects prototype-affecting keys, non-canonical array indices and any result outside the normal model ceilings.

An edit is not applied immediately: it is first passed through the current format's export adapter. The UI shows the proposed source and all conversion events, and requires a separate acknowledgement when any event is classified as loss.