# Regex Tools

Regex Tools is a production-oriented, local-first browser workbench for
developing, understanding, testing and applying regular expressions. It is an
independent static React/Vite application and a module of add·ideas Toolbox.

Patterns, replacement templates, test text and project files stay in the
browser. There is no account, backend, upload, remote execution, telemetry,
runtime CDN, remote font, or cloud-save service.

## Current release — 0.3.0

The v0.3.0 release contains four executable engine slices:

- the current browser's native ECMAScript `RegExp`;
- official PCRE2 10.47 as a self-hosted 8-bit WebAssembly pack;
- CPython 3.14.2 `re` in the self-hosted Pyodide 314.0.3 runtime; and
- TeaVM 0.15.0's `java.util.regex` class-library implementation as a
  self-hosted ES2015 module.

The Java flavour is TeaVM class-library code compiled to JavaScript. It is not
OpenJDK and is not advertised as an OpenJDK compatibility oracle.

- Syntax acceptance: `@eslint-community/regexpp` 4.12.2, ECMAScript 2025
  grammar; application-owned structural explanations are partial and
  feature-matrix tested.
- ECMAScript native and editor offsets are UTF-16 code units.
- PCRE2 execution and substitution: pinned official PCRE2 10.47 compiled to a
  self-hosted 8-bit WebAssembly pack with UTF/UCP always enabled.
- PCRE2 native UTF-8 byte ranges are retained and normalized to exact editor
  UTF-16 ranges; inputs with lone surrogates are rejected instead of rewritten.
- PCRE2 match-step, depth and heap limits are configurable inside hard caps.
- PCRE2 automatic callouts are copied through a separate ABI operation and
  displayed from a dedicated killable trace worker under 50,000-event and
  10 MiB serialized-data hard caps.
- Python compilation, matching and replacement use the bundled CPython `re`
  module. Native code-point ranges are retained and normalized to exact editor
  UTF-16 ranges.
- Java compilation, matching and replacement use TeaVM's
  `java.util.regex.Pattern` and `Matcher`. Native ranges are UTF-16 code units.
- TeaVM 0.15.0 replacement supports its actual single-digit `$n` subset.
  `$12` is therefore `$1` followed by literal `2`; Java SE `${name}`
  replacement is reported as unavailable rather than presented as OpenJDK
  behavior.
- Python and Java use deliberately partial application-owned lexical syntax
  providers; their actual runtime compiler remains authoritative.
- Parsing and execution run in separate workers.
- Actual execution can be terminated by killing its worker.
- The worker is recreated after timeout, crash or cancellation.
- ECMAScript capture ranges use the engine's `d` indices result; the other
  flavours return their native range records through bounded bridges.
- Named, numbered, optional, empty and repeated-final captures are represented.
- Match, replacement, typed list/extraction, corpus/apply and unit-test modes
  are available.
- ECMAScript-specific advisory static risk findings, bounded dynamic growth
  probes and cold/warm benchmarks run locally; timing and growth execution is
  isolated in a separately killable worker.
- ECMAScript and PCRE2 can be compared through two exact, independently timed
  requests without translating patterns, flags or replacements. Incomplete or
  rejected sides are explicitly not comparable.
- The reviewed PCRE2 C17 generator emits exact UTF-8 byte arrays, configured
  limits and error handling for the PCRE2 10.47 8-bit API.
- A cancellable, deterministic subject minimizer verifies saved-test failures,
  capture-range failures, exact timeouts and ECMAScript/PCRE2 mismatches against
  the selected real workers under explicit evaluation and wall budgets.
- Deterministic ECMAScript test-case generation samples the normalized AST from
  an explicit seed, reports structural coverage and unsupported constructs, and
  labels cases only after verification by the selected real engine worker.
- Project JSON import/export and optional IndexedDB save are available.
- Interactive test text is not persisted by default.

No licensed parser is used, referenced at runtime, or required to build the
project. This is the community build and will grow flavour by flavour using
reviewed open-source syntax providers and actual named engines.

## The two trees

The **explanation tree** is a deterministic structural explanation generated
from the application-owned normalized syntax tree. It is not an engine trace.

The **extraction tree** contains actual engine matches and captures. Its
hierarchy uses syntactic capture nesting while preserving actual engine ranges.
ECMAScript exposes only the final retained capture of a repeated group; Regex
Tools says so and never invents capture history.

Browser ECMAScript exposes no actual engine trace. PCRE2 has a separate bounded
automatic-callout viewer. Its event fields are reported by PCRE2, while
“forward”, “same position” and “apparent backtrack” are visibly derived only
from adjacent reported subject positions. The callout stream is not described
as a complete record of every internal engine action.

## Workbench modes

- **Match** — live highlighting, extraction tree and bounded capture table.
- **Replace** — engine-native bounded substitution for all four executable
  flavours;
  output completeness is reported separately from bounded per-match
  original/result, changed-range and token-contribution previews that
  synchronize the replacement, pattern, subject and output editors.
- **List** — non-executable typed templates with text, CSV, JSON and NDJSON
  export; export is disabled when engine collection or bounded rendering is
  incomplete.
- **Corpus / Apply** — bounded multi-file UTF-8 and pasted-text input processed
  sequentially in killable workers. Whole-document and independent-line
  semantics are explicit; line apply preserves original separators. Progress,
  cancellation, bounded capture samples, replacement previews, per-document
  exact/partial summaries, content-free JSON/CSV/NDJSON reports and
  privacy-gated exact-output ZIP/downloads are available. Original files are
  never modified.
- **Unit tests** — editable and cloneable versioned presence, count, full-match,
  capture, replacement, timing and expected-timeout assertions. Capture a
  complete current result, select only cases to rerun, filter failures, and
  import/export a standalone validated JSON suite; execution stays in isolated
  workers.

The auxiliary **PCRE trace**, ECMAScript **Analysis**, **Compare & code**,
**Generate cases**, **Minimize** and **Format** workspaces are separate from
normal matching, replacement, tests and corpus execution. Trace collection is
never included in benchmark measurements.
Comparison retains native ranges while aligning actual results by normalized
editor UTF-16 ranges. Agreement is described only as evidence for the current
subject, never as proof of pattern equivalence.
Case generation is bounded sampling, not a proof of language coverage. Version
1 supports only a complete accepted ECMAScript normalized AST; the partial
PCRE2 provider is reported as unavailable rather than passed through or
relabeled. See
[`docs/GENERATED_CASES.md`](docs/GENERATED_CASES.md).
Minimization verifies every accepted candidate against the exact selected
worker request and claims only local minimality under its documented
Unicode-scalar transforms; see
[`docs/MINIMIZATION.md`](docs/MINIMIZATION.md).
Formatting is currently ECMAScript-only and changes only parser-reported
literal slash/control representations. Apply requires independent reparse,
actual-engine current-subject/replacement comparison, every applicable exact
unit test and explicit confirmation; see
[`docs/FORMATTING.md`](docs/FORMATTING.md).

## Flags and iteration

Pattern and flags are stored separately; delimiters are presentation only.
ECMAScript preserves native `g` and `y`; PCRE2 offers the application-level
`g` iteration flag plus `i`, `m`, `s`, `x`, `U` and `J`. PCRE2 UTF/UCP mode is
mandatory because browser strings are Unicode and partial-byte ranges cannot be
represented safely in the editor. Python exposes application-level `g` plus
CPython `re` flags `a`, `i`, `m`, `s` and `x`. Java exposes application-level
`g` plus TeaVM `Pattern` flags `i`, `d`, `m`, `s`, `u`, `x` and the documented
`U` compatibility request. The optional **Scan all** action adds only an
internal iteration request and does not change saved flags. ECMAScript may add
internal `d` to obtain exact indices without changing match semantics.

Zero-length global iteration advances using the selected Unicode semantics and
cannot loop forever.

## Limits and timeout behavior

Important defaults:

| Limit                           |            Default |
| ------------------------------- | -----------------: |
| Live parse debounce             |             120 ms |
| Live execution debounce         |             220 ms |
| Live execution timeout          |             250 ms |
| Manual execution timeout        |                2 s |
| Advanced maximum timeout        |               10 s |
| Pattern hard limit              | 64 Ki UTF-16 units |
| Interactive subject hard limit  |             16 MiB |
| Corpus documents                |                256 |
| Per corpus document             |             16 MiB |
| Aggregate corpus input          |            256 MiB |
| Independent corpus lines        |            100,000 |
| Aggregate corpus matches        |            100,000 |
| Capture summaries per document  |                 32 |
| Samples per capture             |                  3 |
| Retained applied output         |             64 MiB |
| Applied ZIP                     |             68 MiB |
| Corpus batch wall time          |              5 min |
| Replacement template            | 64 Ki UTF-16 units |
| List template                   | 16 Ki UTF-16 units |
| Per-match replacement preview   |        200 matches |
| Replacement contributions       |              2,000 |
| Maximum matches                 |             10,000 |
| Maximum capture groups          |              1,000 |
| Maximum capture rows            |            100,000 |
| Replacement preview             |             64 MiB |
| Project JSON document           |       32 MiB UTF-8 |
| Standalone test-suite JSON      |       32 MiB UTF-8 |
| In-memory unit-test suite       |       32 MiB UTF-8 |
| Tests per suite                 |              1,000 |
| Retained message per test       |        2 KiB UTF-8 |
| Unit-test suite wall time       |               60 s |
| PCRE2 trace events              |             50,000 |
| PCRE2 serialized trace          |             10 MiB |
| Comparison retained differences |              5,000 |
| Comparison retained alignments  |              2,000 |
| Static analysis traversal       |       20,000 nodes |
| Static analysis findings        |                100 |
| Growth probe steps              |                 24 |
| Analysis aggregate wall time    |               30 s |
| Generated verified cases        |                 48 |
| Generated candidate attempts    |                192 |
| Generation aggregate wall time  |               15 s |
| Generation aggregate candidates |              1 MiB |
| Minimizer subject               |        1 MiB UTF-8 |
| Minimizer candidate evaluations |              2,000 |
| Minimizer aggregate wall time   |               60 s |
| Formatter validation tests      |              1,000 |
| Formatter test wall time        |               60 s |
| Retained minimizer history      |                200 |

Analysis defaults to three warm-up plus 15 measured samples (caps: 100 and
1,000), with a 2-second per-sample worker limit that can be raised to 10
seconds. Growth generation defaults to 1 MiB, preflights the central 16 MiB
subject cap and rejects more than 10,000,000 repetitions. Replacement timing
is skipped above an 8 Mi UTF-16-unit output estimate.

A timeout terminates actual execution. It is reported as **timed out**, never as
“no match”. The next request creates a fresh worker.

Replacement applies exactly the authoritative matches returned under the match
and capture-row limits. If either collection limit is reached, later subject
text remains unchanged and the output is explicitly reported as partial. The
separate output-byte limit reports a truncated output preview.

## Project persistence

Project schema version 1 stores flavour, pattern, flags, options, replacement,
list template, unit tests (including explicit scan-all behavior and supported
per-test timeouts) and selected UI state. Import rebuilds a validated plain
object; it never evaluates code and pauses live execution until review.
Workbench-wide manual timeout changes are not persisted, and unsupported
project resource overrides are rejected.

Interactive test text is excluded from local save and export unless explicitly
enabled. Unit-test definitions are deliberate project data, but their subjects
require separate explicit opt-ins for JSON export and IndexedDB save.
Standalone test-suite JSON likewise omits subjects by default, is schema- and
field-validated on import, and appends at most 1,000 total tests. Add, update,
clone and append-import also preflight the complete in-memory suite against a
32 MiB aggregate bound. Added generated tests retain their validated generator
version, seed, candidate ID and intended outcome; editing one removes that
provenance. Schema v1 can remember that the corpus workspace was selected, but
corpus documents, contents, outputs and results are always ephemeral and
excluded from JSON and IndexedDB. Imports that try to inject those fields are
rejected. Import, export and IndexedDB save enforce one 32 MiB aggregate UTF-8
JSON-document limit in addition to per-field limits. The latest local save is
read through an `updatedAt` index cursor; loading it does not materialize the
complete project store.

## Development

Requirements: Node.js 22 or later and npm 11.

```sh
npm install
npm run dev
```

Quality and release commands:

```sh
npm run typecheck
npm run lint
npm run format:check
npm test
npm run test:conformance
npm run test:browser
npm run engines:build
npm run engines:verify
npm run engines:pcre2:build -- --source-dir /path/to/pcre2 --emcc /path/to/emcc
npm run engines:pcre2:verify
npm run engines:pcre2:install
node scripts/python-engine-pack.mjs node_modules/pyodide .engine-build/python
node scripts/install-python-engine.mjs .engine-build/python
node scripts/java-engine-pack.mjs
node scripts/install-java-engine.mjs
npm run build
npm run toolbox:check
npm run check
npm run release:artifact
```

The normal build performs no network download. `vite.config.ts` uses
`base: "./"`; all workers and assets are self-hosted and nested-path safe.

## Toolbox SDK and Portal

The application uses Toolbox SDK 0.2.2 `AppShell`. Its canonical typed/static
manifest source is `src/toolbox/manifest.source.json`; generation fails if its
version or source identity drifts.

The deterministic release ZIP can be consumed by Toolbox Portal using an exact
artifact URL, manifest ID, version and SHA-256. Portal builds consume the
artifact; they do not build Regex Tools source.

The PCRE2 and Python workers load only self-hosted JavaScript, WebAssembly and
Python-standard-library assets; Java loads its self-hosted ES module.
Deployment must serve `.wasm` as `application/wasm` and allow WebAssembly
compilation in `script-src`; see
[`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).

## Flavour support and boundaries

PCRE2's application-owned ABI provides bounded compilation, matching, native
substitution and separate automatic-callout tracing, copied caller-owned
records, deterministic allocation cleanup and explicit result completeness.
The exact source/toolchain build is reproducible offline and the verified pack
is bundled with its metadata, checksums and licence.

The Python slice executes actual CPython 3.14.2 `re` in Pyodide 314.0.3; it does
not translate a Python pattern into ECMAScript. Pyodide startup is a separate
worker-load operation and is excluded from the normal execution deadline.

The Java slice executes TeaVM 0.15.0's own `java.util.regex` class library. It
does not translate a Java pattern into ECMAScript, but it is also not OpenJDK:
in particular, TeaVM 0.15.0 does not implement
`Pattern.UNICODE_CHARACTER_CLASS`. The UI reports the `U` flag as a
compatibility request and preserves TeaVM's predefined-class and word-boundary
behaviour.

Generated-case version 1 is ECMAScript-only. It traverses the complete
application-owned ECMAScript AST under node, depth, attempt, byte, repetition
and wall-time caps, then verifies every retained label through the actual
browser `RegExp` adapter. PCRE2 generation remains unavailable until its syntax
provider can expose a complete normalized tree.

Pattern formatting is likewise ECMAScript-only. It consumes the complete
regexpp token model and does not reinterpret the partial PCRE2, Python or Java
structures. Those flavours remain unavailable until complete grammar-backed
implementations pass equivalent idempotence, reparse, engine and test gates.

The first advertised generated-code target is the PCRE2 10.47 8-bit C API.
Its deterministic match and replacement fixtures compile and execute against
the exact official release. See
[`docs/COMPARISON_AND_CODEGEN.md`](docs/COMPARISON_AND_CODEGEN.md).

Go, Rust, .NET and legacy PCRE follow only through their actual named runtimes.
No flavour is emulated through JavaScript and relabelled.

## Licensing

Regex Tools original source is GPL-3.0-or-later, copyright © 2026 Albrecht
Degering. Shipped dependencies retain their own compatible licences and
notices; see [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and
[`LICENSES/`](LICENSES/).

No regex101 application source, branding, generated explanation prose,
reference text or visual design is copied. RegexLib and RGXP.RU fixtures are not
redistributed because a suitable fixture licence was not established. No
RegexHub fixture is shipped in v0.3.0.

## Known limitations

- ECMAScript syntax coverage follows regexpp's ECMAScript 2025 grammar; a later
  browser proposal may execute before the provider understands it.
- Exact consumed-length bounds for ECMAScript Unicode properties of strings
  remain unknown and are reported as such rather than guessed.
- regexpp does not provide tolerant AST recovery. Malformed input receives an
  exact reported error position as a zero-width range and an application-owned
  error node, not a fabricated multi-character span or partial provider AST.
- Native JavaScript compile errors do not consistently expose a machine-readable
  source offset; the provider diagnostic supplies the range.
- Capture history is unavailable. Browser ECMAScript exposes no engine trace;
  PCRE2 exposes bounded automatic callouts, not a complete internal trace.
  Movement classifications are derived and labelled as such.
- Static risk heuristics and generated-input observations apply only to
  ECMAScript 2025 and are advisory. They are never proof of safety or of a
  pattern’s general time complexity; analysis settings and results are
  ephemeral.
- PCRE2 structural explanations are deliberately partial. The application
  recognizes common capture forms and PCRE2-only constructs for reference, but
  the actual PCRE2 compiler is authoritative. Branch-reset capture numbering is
  omitted from the explanation rather than guessed.
- Python and Java structural explanations are deliberately partial lexical
  models. CPython `re` and TeaVM `Pattern` respectively remain authoritative
  for compilation, group count and matching.
- Python reports native Unicode code-point offsets and converts them at valid
  boundaries to editor UTF-16 ranges. Java and ECMAScript report native UTF-16
  offsets; PCRE2 reports native UTF-8 bytes. Native units remain visible.
- TeaVM `java.util.regex` is not OpenJDK. Its `U` compatibility request does
  not add OpenJDK's `UNICODE_CHARACTER_CLASS` semantics.
- Capture-table rendering is paged at 200 rows while aggregate collection is
  bounded separately.
- Explanation/extraction trees and editor decorations show bounded 2,000-node
  previews with explicit totals; normalized results and table/export data stay
  separate from those render-only caps.
- Extraction-tree values show at most 100 UTF-16 units and capture-table cells
  at most 512. UI-only shortening shows the exact range length; engine-clipped
  values are labelled as prefixes of that range. Copy and eligible JSON export
  continue to use the separately bounded engine value.
- Replacement parsing is complete below its 64 Ki input limit, while editor
  marks, token rows and diagnostics have explicit presentation caps. List token
  chips are similarly bounded, and list rendering stops at 250,000 token
  evaluations.
- Per-match replacement presentation renders at most 200 matches and 2,000
  token contributions under a 50,000 token-evaluation budget. Bounded
  replacement output remains available separately and is labelled partial if
  match collection reached a limit.
- Result-value previews are capped per value and overall. List export is
  disabled whenever a bounded preview would otherwise be mistaken for complete
  data.
- Unit-test failure messages retain at most 2 KiB UTF-8 per case, so 1,000
  results stay below the 2 MiB log budget. Oversized exact replacement results
  offer a lightweight `should-match` draft when one is valid and fits the suite.
- Corpus input accepts recognized UTF-8 text only. It does not auto-detect
  legacy encodings or recursively traverse directories.
- Corpus capture extraction is a bounded per-document summary: full match plus
  at most 31 capture groups, three 160-unit samples each. Content-free reports
  include counts and clipping metadata, never the samples themselves.
- Generated cases deliberately sample only ECMAScript and report
  backreferences, lookaround, boundaries, complex Unicode sets and other
  unsupported constraints instead of claiming exhaustive coverage.
- Pattern formatting is intentionally narrow and ECMAScript-only. Its bounded
  source/candidate and unit-test validation is evidence for the checked
  snapshots, not a proof of equivalence over all subjects.
- Cross-flavour comparison, PCRE2 C generation and subject minimization remain
  scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting
  Python or Java does not silently substitute one of those engines.

Architecture, security, performance, provenance and flavour details live in
[`docs/`](docs/).
