51 lines
3.8 KiB
Markdown
51 lines
3.8 KiB
Markdown
# Data Tools
|
|
|
|
Inspect, format, query, flatten and convert structured data locally in the browser. Data Tools is a standalone application in the [add·ideas Toolbox](https://git.add-ideas.de/lotobo/toolbox-portal); input text and files never leave the current browser.
|
|
|
|
## Version 0.1.0
|
|
|
|
- Detect or explicitly select JSON, YAML 1.2, TOML, XML, CSV, TSV and NDJSON.
|
|
- Keep JSON integer and decimal lexemes exact, including values beyond IEEE-754 range.
|
|
- Inspect a bounded typed tree, a tabular projection and flattened scalar leaves.
|
|
- Query with RFC 6901 JSON Pointer or a deliberately small, non-evaluating path subset.
|
|
- Format and convert between every supported output with a visible loss/coercion report.
|
|
- Export CSV/TSV with opt-in spreadsheet formula neutralisation, enabled by default.
|
|
- Open UTF-8 files or edit text directly; parsing runs in a disposable module worker and the previous valid result remains visible while a replacement is analysed.
|
|
|
|
### Format semantics
|
|
|
|
| Format | v0.1 behaviour |
|
|
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| JSON | Duplicate keys and prototype-affecting keys are rejected. Number lexemes are stored and emitted exactly. |
|
|
| YAML | YAML 1.2 core schema, unique string keys, BigInt integers, maximum 50 alias expansions. Custom tags and merge keys are rejected. |
|
|
| TOML | Integers are BigInt-backed and dates/times retain their TOML kind. Floats use the browser's binary representation and are diagnosed. |
|
|
| XML | `DOCTYPE`, entity declarations and XInclude are rejected before inert parsing. Elements, namespaces, attributes, text, CDATA, comments and processing instructions use an explicit ordered record model. |
|
|
| CSV / TSV | Fields remain strings. Quoting and line endings are parsed without dynamic typing. The first row can be used as table headings. |
|
|
| NDJSON | Every non-empty line is parsed as lossless JSON. Blank lines and the exact failing line are reported. |
|
|
|
|
The safe path subset supports `$`, `.key`, `['key']`, `[index]` and `[*]`. It intentionally has no recursive descent, filters, expressions, functions or script evaluation.
|
|
|
|
### Resource limits
|
|
|
|
The current production limits are 16 MiB of UTF-8 input, 8 million JavaScript characters, 100,000 model nodes, depth 128, 50,000 delimited rows, 2,000 columns, 250,000 cells, 1 million characters per field, 4,096 query characters, 1,000 query matches and 16 million generated characters. Disposable parses have a 30-second deadline. UI previews are smaller than the model and state when they are truncated.
|
|
|
|
See [Architecture](docs/ARCHITECTURE.md), [Privacy and security](docs/PRIVACY-SECURITY.md) and [Accessibility](docs/ACCESSIBILITY.md) for the design and threat model.
|
|
|
|
## Development
|
|
|
|
Requires Node.js 22 and npm 11.
|
|
|
|
```sh
|
|
npm ci
|
|
npm run check
|
|
npm run test:browser
|
|
```
|
|
|
|
## Release
|
|
|
|
`npm run release:artifact` creates a deterministic `release/data-tools-0.1.0.zip` and checksum sidecar.
|
|
|
|
## Licence
|
|
|
|
GPL-3.0-or-later. Runtime dependencies retain their own licences; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
|