# Data Tools

Inspect, format, query, flatten and convert structured data locally in the browser. Data Tools is a standalone application in the [add·ideas Toolbox](https://git.add-ideas.de/lotobo/toolbox-portal); input text and files never leave the current browser.

## Version 0.2.0

- Detect or explicitly select JSON, YAML 1.2, TOML, XML, CSV, TSV and NDJSON.
- Keep JSON integer and decimal lexemes exact, including values beyond IEEE-754 range.
- Inspect a bounded typed tree, a tabular projection and flattened scalar leaves.
- Query with RFC 6901 JSON Pointer or a deliberately small, non-evaluating path subset.
- Infer a bounded, sample-labelled JSON Schema with field presence, nullability, mixed types and exact-number provenance.
- Preview immutable RFC 6901 add/replace/remove edits, inspect the source-format serialization disclosure, and explicitly accept any loss before applying it.
- Format and convert between every supported output with a visible loss/coercion report.
- Drive import, export and format selectors from one ordered adapter catalogue with a concrete parser and writer registry for every advertised format.
- Export CSV/TSV with opt-in spreadsheet formula neutralisation, enabled by default.
- Open UTF-8 files or edit text directly; parsing runs through the shared
  cancellable disposable-worker protocol with a hard deadline, while the
  previous valid result remains visible until a replacement is ready.

### Format semantics

| Format    | v0.1 behaviour                                                                                                                                                                                           |
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| JSON      | Duplicate keys and prototype-affecting keys are rejected. Number lexemes are stored and emitted exactly.                                                                                                 |
| YAML      | YAML 1.2 core schema, unique string keys, BigInt integers, maximum 50 alias expansions. Custom tags and merge keys are rejected.                                                                         |
| TOML      | Integers are BigInt-backed and dates/times retain their TOML kind. Floats use the browser's binary representation and are diagnosed.                                                                     |
| XML       | `DOCTYPE`, entity declarations and XInclude are rejected before inert parsing. Elements, namespaces, attributes, text, CDATA, comments and processing instructions use an explicit ordered record model. |
| CSV / TSV | Fields remain strings. Quoting and line endings are parsed without dynamic typing. The first row can be used as table headings.                                                                          |
| NDJSON    | Every non-empty line is parsed as lossless JSON. Blank lines and the exact failing line are reported.                                                                                                    |

The safe path subset supports `$`, `.key`, `['key']`, `[index]` and `[*]`. It intentionally has no recursive descent, filters, expressions, functions or script evaluation.

### Resource limits

The current production limits are 16 MiB of UTF-8 input, 8 million JavaScript characters, 100,000 model nodes, depth 128, 50,000 delimited rows, 2,000 columns, 250,000 cells, 1 million characters per field/edit value, 5,000 inference samples/fields, 4,096 query or edit-pointer characters, 1,000 query matches and 16 million generated characters. Disposable parses have a 30-second deadline. UI previews are smaller than the model and state when they are truncated.

See [Architecture](docs/ARCHITECTURE.md), [Privacy and security](docs/PRIVACY-SECURITY.md) and [Accessibility](docs/ACCESSIBILITY.md) for the design and threat model.

## Development

Requires Node.js 22 and npm 11.

```sh
npm ci
npm run check
npm run test:browser
```

## Release

`npm run release:artifact` creates a deterministic `release/data-tools-0.2.0.zip` and checksum sidecar.

## Licence

GPL-3.0-or-later. Runtime dependencies retain their own licences; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).