@@ -1,5 +1,10 @@
|
||||
# Changelog
|
||||
|
||||
## 0.2.0 - 2026-09-02
|
||||
|
||||
- Add raw-input inference evidence, a visible bounded query AST and a local
|
||||
DuckDB-WASM adapter for broader analytical SQL without remote execution.
|
||||
|
||||
## 0.1.0 - 2026-09-01
|
||||
|
||||
- Add bounded JSON/CSV/NDJSON/XML parsing, SQL-like and path query engines.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+19
-2
@@ -2,9 +2,17 @@
|
||||
|
||||
Query Tools is a GPL-3.0-or-later, local-first workbench for bounded JSON, CSV,
|
||||
NDJSON and static XML. It provides an intentionally small SQL-like row language
|
||||
and JSONPath-like value language, result table/tree/JSON views, explain plans,
|
||||
and JSONPath-like value language, parsed SQL/path syntax trees, result
|
||||
table/tree/JSON views, explain plans,
|
||||
local saved queries, and JSON/CSV/NDJSON export. No input is uploaded and no
|
||||
query is evaluated as code.
|
||||
query is evaluated as application JavaScript.
|
||||
|
||||
CSV import has explicit `none`, `safe`, and `aggressive` inference modes and
|
||||
retains every raw cell beside inferred-value evidence. JSON import records raw
|
||||
unsafe-integer and precision-risk number lexemes. A separately selected
|
||||
DuckDB-WASM mode runs bounded, read-only `SELECT` queries in a disposable local
|
||||
worker using same-origin bundled MVP WASM assets; external access and automatic
|
||||
extension loading/installing are disabled.
|
||||
|
||||
SQL subset: `SELECT`, AND-connected `WHERE` comparisons/`CONTAINS`, `GROUP BY`,
|
||||
`COUNT`, `SUM`, `AVG`, `MIN`, `MAX`, `ORDER BY`, and `LIMIT`. Path subset:
|
||||
@@ -13,6 +21,15 @@ not compatibility with full SQL, JSONPath, or JMESPath.
|
||||
|
||||
Limits: 2 MiB source, 10,000 rows/path values, 200 fields, 200,000 structured
|
||||
nodes, depth 32, and 20,000 query characters. XML DTD/entities are rejected.
|
||||
DuckDB mode accepts at most 10,000 rows, returns at most 1,000 rows, uses a 128
|
||||
MiB database memory limit, and has a 30-second startup ceiling followed by a
|
||||
ten-second query deadline. It is not a database connection and does not expose
|
||||
files, network resources, extensions, DDL, DML, `COPY`, or `PRAGMA`.
|
||||
|
||||
Static hosting must serve `.wasm` files as `application/wasm`, cache hashed
|
||||
assets immutably, permit `'wasm-unsafe-eval'` for same-origin scripts, and allow
|
||||
same-origin/blob workers while retaining `connect-src 'self'`. The production
|
||||
browser test verifies those headers against the locally bundled DuckDB asset.
|
||||
|
||||
Run `npm ci`, then `npm run check`, `npm run test:browser`, and
|
||||
`npm run package:release -- --force`. Copyright © 2026 Albrecht Degering.
|
||||
|
||||
+16
-1
@@ -1,3 +1,18 @@
|
||||
# Security
|
||||
|
||||
Report vulnerabilities privately through the Gitea repository owner. Never attach confidential datasets publicly. Inputs are untrusted, bounded, rendered inertly, and never executed; XML DTD/entities and dangerous JSON keys are rejected.
|
||||
Report vulnerabilities privately through the Gitea repository owner. Never
|
||||
attach confidential datasets publicly. Inputs are untrusted, bounded, rendered
|
||||
inertly, and never evaluated as application code; XML DTD/entities and dangerous
|
||||
JSON keys are rejected.
|
||||
|
||||
The built-in query languages use fixed parsers and interpreters without `eval`
|
||||
or dynamic function construction. DuckDB mode accepts one parsed `SELECT`
|
||||
statement, rejects semicolons and mutation, extension, attachment, file and
|
||||
configuration keywords, then executes against a bounded in-memory table in a
|
||||
disposable locally bundled WASM worker. External access and extension
|
||||
installation/loading are disabled; startup/query deadlines terminate the worker
|
||||
and stale results cannot replace newer state.
|
||||
|
||||
Deployment must serve the bundled WASM with `application/wasm` and a CSP that
|
||||
allows same-origin WASM and workers while keeping `connect-src 'self'`. Browser
|
||||
release tests verify those headers and fail on third-party requests.
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
# Source identity
|
||||
|
||||
- Project: Query Tools
|
||||
- Version: 0.1.0
|
||||
- Version: 0.2.0
|
||||
- Repository: https://git.add-ideas.de/lotobo/query-tools
|
||||
- Licence: GPL-3.0-or-later
|
||||
- Toolbox id: `de.add-ideas.query-tools`
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
# Third-party notices
|
||||
|
||||
Runtime dependencies are bundled locally: React/React DOM (MIT), @xmldom/xmldom (MIT), and add·ideas Toolbox Contract/Shell/Helpers (GPL-3.0-or-later). Exact texts are generated into `LICENSES/npm-runtime-licenses.txt`.
|
||||
Runtime dependencies are bundled locally: React/React DOM (MIT),
|
||||
@xmldom/xmldom (MIT), DuckDB-WASM and Apache Arrow JavaScript (MIT), add·ideas
|
||||
Toolbox Contract/Shell 0.3.0 (Apache-2.0), and Toolbox Helpers 0.2.0
|
||||
(GPL-3.0-or-later). No DuckDB asset is fetched at runtime. Exact texts are
|
||||
generated into `LICENSES/npm-runtime-licenses.txt`.
|
||||
|
||||
@@ -1,3 +1,21 @@
|
||||
# Architecture
|
||||
|
||||
`core/data.ts` performs bounded static parsing into JSON-compatible values. `core/query.ts` tokenizes a fixed grammar, interprets field paths and returns an explain plan; it never compiles or evaluates code. React retains the last valid dataset/result after errors. Relative assets and a same-origin service worker support nested offline deployment.
|
||||
`core/data.ts` performs bounded static parsing into JSON-compatible values while
|
||||
retaining raw numeric and cell-inference evidence. `core/query.ts` tokenizes the
|
||||
fixed SQL-like and path grammars, exposes their bounded syntax trees, interprets
|
||||
field paths and returns an explain plan; it never compiles or evaluates code.
|
||||
|
||||
`core/duckdb.ts` is a separately selected adapter for the locally bundled
|
||||
single-threaded DuckDB-WASM MVP build. A lexical security gate accepts exactly
|
||||
one `SELECT` statement and rejects semicolons plus mutation, attachment,
|
||||
extension, file and configuration keywords before DuckDB performs its complete
|
||||
SQL parse. The adapter serializes at most 10,000 bounded rows into an in-memory
|
||||
table, disables external access and extension auto-install/auto-load, caps
|
||||
database memory at 128 MiB, limits results to 1,000 rows and terminates the
|
||||
disposable worker after completion, cancellation or a hard deadline.
|
||||
|
||||
React retains the last valid dataset/result after errors and suppresses stale
|
||||
asynchronous results. Relative assets and a same-origin service worker support
|
||||
nested offline deployment. The static host serves hashed WASM as
|
||||
`application/wasm` with immutable caching and grants only the CSP allowances
|
||||
needed for same-origin WASM and workers.
|
||||
|
||||
@@ -1,3 +1,13 @@
|
||||
# Privacy and security
|
||||
|
||||
Data stays in browser memory. Saved query text uses localStorage only. There is no telemetry, remote schema resolution, query execution endpoint, URL fetch, eval or dynamic function construction. Exports require an explicit action. Limits are documented in README.
|
||||
Data stays in browser memory. Saved query text uses localStorage only. There is
|
||||
no telemetry, remote schema resolution, query execution endpoint, URL fetch,
|
||||
`eval` or dynamic function construction. Exports require an explicit action.
|
||||
|
||||
DuckDB mode loads the bounded dataset into an in-memory table inside a
|
||||
disposable worker using same-origin bundled JavaScript and WASM. External access,
|
||||
extension installation and extension auto-loading are disabled. The app exposes
|
||||
no database connection or file/network query surface; completion, cancellation
|
||||
and hard deadlines terminate the worker. The host CSP retains
|
||||
`connect-src 'self'`, and browser tests fail on third-party requests. Input,
|
||||
result, memory and time limits are documented in README.
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
const CACHE = "query-tools-v0.1.0",
|
||||
const CACHE = "query-tools-v0.2.0",
|
||||
APP = [
|
||||
"./",
|
||||
"./index.html",
|
||||
|
||||
+18
-1
@@ -3,7 +3,7 @@
|
||||
"schemaVersion": 1,
|
||||
"id": "de.add-ideas.query-tools",
|
||||
"name": "Query Tools",
|
||||
"version": "0.1.0",
|
||||
"version": "0.2.0",
|
||||
"description": "Query structured data locally.",
|
||||
"entry": "./",
|
||||
"icon": "./favicon.svg",
|
||||
@@ -21,6 +21,23 @@
|
||||
"crossOriginIsolated": false,
|
||||
"topLevelContext": false
|
||||
},
|
||||
"io": {
|
||||
"accepts": [
|
||||
{ "mediaType": "application/json", "extensions": [".json"] },
|
||||
{
|
||||
"mediaType": "application/x-ndjson",
|
||||
"extensions": [".ndjson", ".jsonl"]
|
||||
},
|
||||
{ "mediaType": "text/csv", "extensions": [".csv"] },
|
||||
{ "mediaType": "application/xml", "extensions": [".xml"] }
|
||||
],
|
||||
"produces": [
|
||||
{ "mediaType": "application/json", "extensions": [".json"] },
|
||||
{ "mediaType": "application/x-ndjson", "extensions": [".ndjson"] },
|
||||
{ "mediaType": "text/csv", "extensions": [".csv"] }
|
||||
]
|
||||
},
|
||||
"capabilities": { "required": [], "optional": ["workers", "webassembly"] },
|
||||
"privacy": {
|
||||
"processing": "local",
|
||||
"fileUploads": true,
|
||||
|
||||
Reference in New Issue
Block a user