regex-tools/README.md

# Regex Tools

Regex Tools is a production-oriented, local-first browser workbench for
developing, understanding, testing and applying regular expressions. It is an
independent static React/Vite application and a module of add·ideas Toolbox.

Patterns, replacement templates, test text and project files stay in the
browser. There is no account, backend, upload, remote execution, telemetry,
runtime CDN, remote font, or cloud-save service.

## Current milestone

Version 0.1.0 is the first production flavour slice: ECMAScript.

- Syntax acceptance: `@eslint-community/regexpp` 4.12.2, ECMAScript 2025
  grammar; application-owned structural explanations are partial and
  feature-matrix tested.
- Execution engine: the current browser's native ECMAScript `RegExp`.
- Native and editor offsets: UTF-16 code units.
- Parsing and execution run in separate workers.
- Actual execution can be terminated by killing its worker.
- The worker is recreated after timeout, crash or cancellation.
- Capture ranges use the engine's `d` indices result.
- Named, numbered, optional, empty and repeated-final captures are represented.
- Match, replacement, typed list/extraction and unit-test modes are available.
- Project JSON import/export and optional IndexedDB save are available.
- Interactive test text is not persisted by default.

No licensed parser is used, referenced at runtime, or required to build the
project. This is the community build and will grow flavour by flavour using
reviewed open-source syntax providers and actual named engines.

## The two trees

The **explanation tree** is a deterministic structural explanation generated
from the application-owned normalized syntax tree. It is not an engine trace.

The **extraction tree** contains actual engine matches and captures. Its
hierarchy uses syntactic capture nesting while preserving actual engine ranges.
ECMAScript exposes only the final retained capture of a repeated group; Regex
Tools says so and never invents capture history.

Browser ECMAScript engines do not expose their internal backtracking trace.
Version 0.1.0 therefore presents no ECMAScript execution trace and no
educational trace disguised as one.

## Workbench modes

- **Match** — live highlighting, extraction tree and bounded capture table.
- **Replace** — native ECMAScript `RegExp` matches with bounded,
  specification-compatible string substitution; output completeness is
  reported separately from bounded per-match original/result, changed-range and
  token-contribution previews that synchronize the replacement, pattern,
  subject and output editors.
- **List** — non-executable typed templates with text, CSV, JSON and NDJSON
  export; export is disabled when engine collection or bounded rendering is
  incomplete.
- **Unit tests** — editable and cloneable versioned presence, count, full-match,
  capture, replacement, timing and expected-timeout assertions. Capture a
  complete current result, select only cases to rerun, filter failures, and
  import/export a standalone validated JSON suite; execution stays in isolated
  workers.

Corpus, comparison, debugging, analysis, generation, minimization, formatting
and code generation remain roadmap items. Controls for them are not exposed in
this release.

## Flags and iteration

Pattern and flags are stored separately; delimiters are presentation only.
Native `g` and `y` behavior is preserved. The optional **Scan all** action is
explicit. When neither `g` nor `y` is present it adds `g` only to that execution
request, visibly records the internal flag, and does not change the saved user
flags. The `d` flag may likewise be added internally to obtain exact indices
without changing matching semantics.

Zero-length global iteration advances using the selected Unicode semantics and
cannot loop forever.

## Limits and timeout behavior

Important defaults:

| Limit                          |            Default |
| ------------------------------ | -----------------: |
| Live parse debounce            |             120 ms |
| Live execution debounce        |             220 ms |
| Live execution timeout         |             250 ms |
| Manual execution timeout       |                2 s |
| Advanced maximum timeout       |               10 s |
| Pattern hard limit             | 64 Ki UTF-16 units |
| Interactive subject hard limit |             16 MiB |
| Replacement template           | 64 Ki UTF-16 units |
| List template                  | 16 Ki UTF-16 units |
| Per-match replacement preview  |        200 matches |
| Replacement contributions      |              2,000 |
| Maximum matches                |             10,000 |
| Maximum capture groups         |              1,000 |
| Maximum capture rows           |            100,000 |
| Replacement preview            |             64 MiB |
| Project JSON document          |       32 MiB UTF-8 |
| Standalone test-suite JSON     |       32 MiB UTF-8 |
| In-memory unit-test suite      |       32 MiB UTF-8 |
| Tests per suite                |              1,000 |
| Retained message per test      |        2 KiB UTF-8 |
| Unit-test suite wall time      |               60 s |

A timeout terminates actual execution. It is reported as **timed out**, never as
“no match”. The next request creates a fresh worker.

Replacement applies exactly the authoritative matches returned under the match
and capture-row limits. If either collection limit is reached, later subject
text remains unchanged and the output is explicitly reported as partial. The
separate output-byte limit reports a truncated output preview.

## Project persistence

Project schema version 1 stores flavour, pattern, flags, options, replacement,
list template, unit tests (including explicit scan-all behavior and supported
per-test timeouts) and selected UI state. Import rebuilds a validated plain
object; it never evaluates code and pauses live execution until review.
Workbench-wide manual timeout changes are not persisted, and unsupported
project resource overrides are rejected.

Interactive test text is excluded from local save and export unless explicitly
enabled. Unit-test definitions are deliberate project data, but their subjects
require separate explicit opt-ins for JSON export and IndexedDB save.
Standalone test-suite JSON likewise omits subjects by default, is schema- and
field-validated on import, and appends at most 1,000 total tests. Add, update,
clone and append-import also preflight the complete in-memory suite against a
32 MiB aggregate bound. Corpus content is not supported or persisted in this
release. Import, export and IndexedDB save enforce one 32 MiB aggregate UTF-8
JSON-document limit in addition to per-field limits. The latest local save is
read through an `updatedAt` index cursor; loading it does not materialize the
complete project store.

## Development

Requirements: Node.js 22 or later and npm 11.

```sh
npm install
npm run dev
```

Quality and release commands:

```sh
npm run typecheck
npm run lint
npm run format:check
npm test
npm run test:conformance
npm run test:browser
npm run engines:build
npm run engines:verify
npm run build
npm run toolbox:check
npm run check
npm run release:artifact
```

The normal build performs no network download. `vite.config.ts` uses
`base: "./"`; all workers and assets are self-hosted and nested-path safe.

## Toolbox SDK and Portal

The application uses Toolbox SDK 0.2.2 `AppShell`. Its canonical typed/static
manifest source is `src/toolbox/manifest.source.json`; generation fails if its
version or source identity drifts.

The deterministic release ZIP can be consumed by Toolbox Portal using an exact
artifact URL, manifest ID, version and SHA-256. Portal builds consume the
artifact; they do not build Regex Tools source.

Version 0.1.0 has no WebAssembly. Its minimum CSP needs self-hosted scripts and
workers, not `wasm-unsafe-eval`. See
[`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).

## Flavour roadmap

The next flavour is official PCRE2 10.47 WebAssembly. A technical spike from the
signed official tag has already demonstrated Unicode matching, named captures,
substitution, automatic and explicit callouts, and match/depth/heap limits.
PCRE2 is not shipped until the application-owned bridge, byte/UTF-16 offsets,
trace caps, browser tests and source/licence material meet the acceptance gate.

Python, Go, Rust, .NET, Java and legacy PCRE follow only through their actual
named runtimes. No flavour is emulated through JavaScript and relabelled.

## Licensing

Regex Tools original source is GPL-3.0-or-later, copyright © 2026 Albrecht
Degering. Shipped dependencies retain their own compatible licences and
notices; see [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and
[`LICENSES/`](LICENSES/).

No regex101 application source, branding, generated explanation prose,
reference text or visual design is copied. RegexLib and RGXP.RU fixtures are not
redistributed because a suitable fixture licence was not established. No
RegexHub fixture is shipped in v0.1.0.

## Known limitations

- ECMAScript syntax coverage follows regexpp's ECMAScript 2025 grammar; a later
  browser proposal may execute before the provider understands it.
- Exact consumed-length bounds for ECMAScript Unicode properties of strings
  remain unknown and are reported as such rather than guessed.
- regexpp does not provide tolerant AST recovery. Malformed input receives an
  exact reported error position as a zero-width range and an application-owned
  error node, not a fabricated multi-character span or partial provider AST.
- Native JavaScript compile errors do not consistently expose a machine-readable
  source offset; the provider diagnostic supplies the range.
- Capture history and actual ECMAScript execution tracing are unavailable.
- Capture-table rendering is paged at 200 rows while aggregate collection is
  bounded separately.
- Explanation/extraction trees and editor decorations show bounded 2,000-node
  previews with explicit totals; normalized results and table/export data stay
  separate from those render-only caps.
- Extraction-tree values show at most 100 UTF-16 units and capture-table cells
  at most 512. UI-only shortening shows the exact range length; engine-clipped
  values are labelled as prefixes of that range. Copy and eligible JSON export
  continue to use the separately bounded engine value.
- Replacement parsing is complete below its 64 Ki input limit, while editor
  marks, token rows and diagnostics have explicit presentation caps. List token
  chips are similarly bounded, and list rendering stops at 250,000 token
  evaluations.
- Per-match replacement presentation renders at most 200 matches and 2,000
  token contributions under a 50,000 token-evaluation budget. Bounded
  replacement output remains available separately and is labelled partial if
  match collection reached a limit.
- Result-value previews are capped per value and overall. List export is
  disabled whenever a bounded preview would otherwise be mistaken for complete
  data.
- Unit-test failure messages retain at most 2 KiB UTF-8 per case, so 1,000
  results stay below the 2 MiB log budget. Oversized exact replacement results
  offer a lightweight `should-match` draft when one is valid and fits the suite.
- Benchmark, risk analysis, generated cases, minimization and corpus processing
  are not in this release.

Architecture, security, performance, provenance and flavour details live in
[`docs/`](docs/).