4.3 KiB
Data Tools
Inspect, format, query, flatten and convert structured data locally in the browser. Data Tools is a standalone application in the add·ideas Toolbox; input text and files never leave the current browser.
Version 0.2.0
- Detect or explicitly select JSON, YAML 1.2, TOML, XML, CSV, TSV and NDJSON.
- Keep JSON integer and decimal lexemes exact, including values beyond IEEE-754 range.
- Inspect a bounded typed tree, a tabular projection and flattened scalar leaves.
- Query with RFC 6901 JSON Pointer or a deliberately small, non-evaluating path subset.
- Infer a bounded, sample-labelled JSON Schema with field presence, nullability, mixed types and exact-number provenance.
- Preview immutable RFC 6901 add/replace/remove edits, inspect the source-format serialization disclosure, and explicitly accept any loss before applying it.
- Format and convert between every supported output with a visible loss/coercion report.
- Drive import, export and format selectors from one ordered adapter catalogue with a concrete parser and writer registry for every advertised format.
- Export CSV/TSV with opt-in spreadsheet formula neutralisation, enabled by default.
- Open UTF-8 files or edit text directly; parsing runs through the shared cancellable disposable-worker protocol with a hard deadline, while the previous valid result remains visible until a replacement is ready.
Format semantics
| Format | v0.1 behaviour |
|---|---|
| JSON | Duplicate keys and prototype-affecting keys are rejected. Number lexemes are stored and emitted exactly. |
| YAML | YAML 1.2 core schema, unique string keys, BigInt integers, maximum 50 alias expansions. Custom tags and merge keys are rejected. |
| TOML | Integers are BigInt-backed and dates/times retain their TOML kind. Floats use the browser's binary representation and are diagnosed. |
| XML | DOCTYPE, entity declarations and XInclude are rejected before inert parsing. Elements, namespaces, attributes, text, CDATA, comments and processing instructions use an explicit ordered record model. |
| CSV / TSV | Fields remain strings. Quoting and line endings are parsed without dynamic typing. The first row can be used as table headings. |
| NDJSON | Every non-empty line is parsed as lossless JSON. Blank lines and the exact failing line are reported. |
The safe path subset supports $, .key, ['key'], [index] and [*]. It intentionally has no recursive descent, filters, expressions, functions or script evaluation.
Resource limits
The current production limits are 16 MiB of UTF-8 input, 8 million JavaScript characters, 100,000 model nodes, depth 128, 50,000 delimited rows, 2,000 columns, 250,000 cells, 1 million characters per field/edit value, 5,000 inference samples/fields, 4,096 query or edit-pointer characters, 1,000 query matches and 16 million generated characters. Disposable parses have a 30-second deadline. UI previews are smaller than the model and state when they are truncated.
See Architecture, Privacy and security and Accessibility for the design and threat model.
Development
Requires Node.js 22 and npm 11.
npm ci
npm run check
npm run test:browser
Release
npm run release:artifact creates a deterministic release/data-tools-0.2.0.zip and checksum sidecar.
Licence
GPL-3.0-or-later. Runtime dependencies retain their own licences; see THIRD_PARTY_NOTICES.md.