regex-tools/README.md

Regex Tools

Regex Tools is a production-oriented, local-first browser workbench for developing, understanding, testing and applying regular expressions. It is an independent static React/Vite application and a module of add·ideas Toolbox.

Patterns, replacement templates, test text and project files stay in the browser. There is no account, backend, upload, remote execution, telemetry, runtime CDN, remote font, or cloud-save service.

Current release — 0.4.0

The v0.4.0 release contains twelve selectable, executable profiles:

• JavaScript (ECMAScript) — the current browser's native RegExp, with ECMAScript 2025 syntax supplied by regexpp 4.12.2;
• PCRE2 — official standalone PCRE2 10.47 as a self-hosted 8-bit WebAssembly pack;
• PHP — actual PHP 8.5.8 preg_* in @php-wasm/web-8-5 3.1.46; this runtime reports PCRE2 10.44 and is deliberately distinct from the standalone PCRE2 10.47 profile;
• Perl — actual Perl 5.28.1 in WebPerl 0.09-beta, prominently identified as a legacy beta compatibility target rather than current desktop Perl;
• Python — CPython 3.14.2 re in self-hosted Pyodide 314.0.3;
• Ruby — CRuby 4.0.0 Regexp in the minimal ruby.wasm 2.9.3-2.9.4 runtime;
• Java — TeaVM 0.15.0's java.util.regex class-library implementation;
• C++ — Emscripten 6.0.4 libc++ std::wregex, using the C++ modified ECMAScript grammar by default and exposing the standard alternate grammar choices;
• Go — Go 1.26.5 standard-library regexp with RE2 syntax;
• Rust — the Rust regex crate 1.13.1, built with rustc 1.97.1;
• .NET — .NET 10.0.10 System.Text.RegularExpressions in browser WebAssembly with invariant-culture execution; and
• Scala compatibility — Scala/JVM regex syntax and replacement semantics through the same TeaVM java.util.regex engine as the Java profile. No Scala runtime or Scala Regex wrapper API is shipped.

Every profile executes the named runtime. Java is TeaVM class-library code, not OpenJDK. Scala is an explicitly labelled Java-compatibility profile, not a relabeled Scala engine.

• Syntax acceptance: @eslint-community/regexpp 4.12.2, ECMAScript 2025 grammar; application-owned structural explanations are partial and feature-matrix tested.
• ECMAScript native and editor offsets are UTF-16 code units.
• PCRE2 execution and substitution: pinned official PCRE2 10.47 compiled to a self-hosted 8-bit WebAssembly pack with UTF/UCP always enabled.
• PCRE2 native UTF-8 byte ranges are retained and normalized to exact editor UTF-16 ranges; inputs with lone surrogates are rejected instead of rewritten.
• PCRE2 match-step, depth and heap limits are configurable inside hard caps.
• PCRE2 automatic callouts are copied through a separate ABI operation and displayed from a dedicated killable trace worker under 50,000-event and 10 MiB serialized-data hard caps.
• PHP compilation and match selection use actual preg_match() in PHP 8.5.8. A fixed byte-bounded expander reproduces PHP replacement tokens and is checked against native preg_replace() at load. PHP returns native UTF-8 byte ranges and reports its own PCRE2 10.44 identity.
• The legacy Perl bridge passes user values as JSON data to fixed application code, disables runtime regex evaluation, rejects code assertions and exposes a pre-tokenized, deliberately bounded replacement grammar. Replacement bytes stream from native spans to a separate fixed MEMFS file instead of being materialized or JSON-escaped; the worker validates the exact bounded UTF-8 file before returning it. Its prebuilt WebPerl loader has a dormant optional interoperability eval site; the fixed engine path is verified without broad CSP 'unsafe-eval'.
• Python compilation, matching and replacement use the bundled CPython re module. Native code-point ranges are retained and normalized to exact editor UTF-16 ranges. Bounded replacement UTF-8 crosses the JSON bridge as canonical base64; the adapter validates the declared byte count, cap and UTF-8 before exposing output.
• Ruby compilation and matching use CRuby's Regexp; native String#gsub and String#sub block iteration drives replacement matching, while a fixed byte-bounded implementation of CRuby 4.0.0's rb_reg_regsub template grammar produces the output prefix. Native character ranges are retained as Unicode code-point offsets. Requests and replacement output use separate bounded files in the worker-local WASI memory filesystem; only bounded metadata returns as inert JSON, avoiding both output escaping amplification and ruby.wasm's JavaScript-eval conversion path.
• Java compilation, matching and replacement use TeaVM's java.util.regex.Pattern and Matcher. Native ranges are UTF-16 code units.
• TeaVM 0.15.0 replacement supports its actual single-digit $n subset. $12 is therefore $1 followed by literal 2; Java SE ${name} replacement is reported as unavailable rather than presented as OpenJDK behavior.
• C++ compilation and matching use libc++ std::wregex; a fixed byte-bounded formatter reproduces match.format grammar and checks parity with that native API at load. Its 32-bit wchar_t ranges are Unicode code points.
• Go and Rust return native UTF-8 byte offsets from their real runtimes. Their fixed bounded expanders are parity-checked against Regexp.ExpandString and Captures::expand, then return output through predictable base64 envelopes. Their adapters measure exact encoded request bytes, and their native JSON caps cover worst-case escaping for every accepted input field.
• .NET returns native UTF-16 offsets and executes with a bounded native regex timeout in addition to the killable worker boundary.
• Python, Java and all eight v0.4.0 community additions use deliberately partial application-owned lexical syntax providers; their actual runtime compiler remains authoritative.
• Parsing and execution run in separate workers.
• Actual execution can be terminated by killing its worker.
• The worker is recreated after timeout, crash or cancellation.
• ECMAScript capture ranges use the engine's d indices result; every other profile returns its native range records through a bounded fixed bridge.
• Named, numbered, optional, empty and repeated-final captures are represented.
• Match, replacement, typed list/extraction, corpus/apply and unit-test modes are available.
• ECMAScript-specific advisory static risk findings, bounded dynamic growth probes and cold/warm benchmarks run locally; timing and growth execution is isolated in a separately killable worker.
• ECMAScript and PCRE2 can be compared through two exact, independently timed requests without translating patterns, flags or replacements. Incomplete or rejected sides are explicitly not comparable.
• The reviewed PCRE2 C17 generator emits exact UTF-8 byte arrays, configured limits and error handling for the PCRE2 10.47 8-bit API.
• A cancellable, deterministic subject minimizer verifies saved-test failures, capture-range failures, exact timeouts and ECMAScript/PCRE2 mismatches against the selected real workers under explicit evaluation and wall budgets.
• Deterministic ECMAScript test-case generation samples the normalized AST from an explicit seed, reports structural coverage and unsupported constructs, and labels cases only after verification by the selected real engine worker.
• Project JSON import/export and optional IndexedDB save are available.
• Interactive test text is not persisted by default.

No licensed parser is used, referenced at runtime, or required to build the project. This is the community build and will grow flavour by flavour using reviewed open-source syntax providers and actual named engines.

The two trees

The explanation tree is a deterministic structural explanation generated from the application-owned normalized syntax tree. It is not an engine trace.

The extraction tree contains actual engine matches and captures. Its hierarchy uses syntactic capture nesting while preserving actual engine ranges. ECMAScript exposes only the final retained capture of a repeated group; Regex Tools says so and never invents capture history.

Browser ECMAScript exposes no actual engine trace. PCRE2 has a separate bounded automatic-callout viewer. Its event fields are reported by PCRE2, while “forward”, “same position” and “apparent backtrack” are visibly derived only from adjacent reported subject positions. The callout stream is not described as a complete record of every internal engine action.

Workbench modes

• Match — live highlighting, extraction tree and bounded capture table.
• Replace — bounded profile-authoritative substitution for all twelve executable profiles; the legacy Perl profile uses its documented fixed non-executable template grammar, and profiles that use fixed bounded expanders identify and parity-check them against their named runtime; output completeness is reported separately from bounded per-match original/result, changed-range and token-contribution previews that synchronize the replacement, pattern, subject and output editors.
• List — non-executable typed templates with text, CSV, JSON and NDJSON export; export is disabled when engine collection or bounded rendering is incomplete.
• Corpus / Apply — bounded multi-file UTF-8 and pasted-text input processed sequentially in killable workers. Whole-document and independent-line semantics are explicit; line apply preserves original separators. Progress, cancellation, bounded capture samples, replacement previews, per-document exact/partial summaries, content-free JSON/CSV/NDJSON reports and privacy-gated exact-output ZIP/downloads are available. Original files are never modified.
• Unit tests — editable and cloneable versioned presence, count, full-match, capture, replacement, timing and expected-timeout assertions. Capture a complete current result, select only cases to rerun, filter failures, and import/export a standalone validated JSON suite; execution stays in isolated workers.

The auxiliary PCRE trace, ECMAScript Analysis, Compare & code, Generate cases, Minimize and Format workspaces are separate from normal matching, replacement, tests and corpus execution. Trace collection is never included in benchmark measurements. Comparison retains native ranges while aligning actual results by normalized editor UTF-16 ranges. Agreement is described only as evidence for the current subject, never as proof of pattern equivalence. Case generation is bounded sampling, not a proof of language coverage. Version 1 supports only a complete accepted ECMAScript normalized AST; the partial PCRE2 provider is reported as unavailable rather than passed through or relabeled. See docs/GENERATED_CASES.md. Minimization verifies every accepted candidate against the exact selected worker request and claims only local minimality under its documented Unicode-scalar transforms; see docs/MINIMIZATION.md. Formatting is currently ECMAScript-only and changes only parser-reported literal slash/control representations. Apply requires independent reparse, actual-engine current-subject/replacement comparison, every applicable exact unit test and explicit confirmation; see docs/FORMATTING.md.

Flags and iteration

Pattern and flags are stored separately; delimiters are presentation only. Each profile exposes only its reviewed allowlist, and application-level g means bounded iteration where the underlying API has no compile-time global flag. PHP delimiters/modifiers, Go and Rust inline options, C++ grammar selection, .NET options and the legacy Perl character-set modes are mapped by their own adapters rather than reused across profiles. Standalone PCRE2 always uses UTF/UCP; PHP controls UTF through its exposed u modifier and reports its own PCRE2 version. The optional Scan all action adds only an internal iteration request and does not change saved flags. ECMAScript may add internal d to obtain exact indices without changing match semantics. The exact flag matrix is in docs/FLAVOUR_SUPPORT.md.

Zero-length global iteration advances using the selected Unicode semantics and cannot loop forever.

Limits and timeout behavior

Important defaults:

Limit	Default
Live parse debounce	120 ms
Live execution debounce	220 ms
Live execution timeout	250 ms
Manual execution timeout	2 s
Advanced maximum timeout	10 s
Pattern hard limit	64 Ki UTF-16 units
Interactive subject hard limit	16 MiB
Corpus documents	256
Per corpus document	16 MiB
Aggregate corpus input	256 MiB
Independent corpus lines	100,000
Aggregate corpus matches	100,000
Capture summaries per document	32
Samples per capture	3
Retained applied output	64 MiB
Applied ZIP	68 MiB
Corpus batch wall time	5 min
Replacement template	64 Ki UTF-16 units
List template	16 Ki UTF-16 units
Per-match replacement preview	200 matches
Replacement contributions	2,000
Maximum matches	10,000
Maximum capture groups	1,000
Maximum capture rows	100,000
Replacement preview	64 MiB
Project JSON document	32 MiB UTF-8
Standalone test-suite JSON	32 MiB UTF-8
In-memory unit-test suite	32 MiB UTF-8
Tests per suite	1,000
Retained message per test	2 KiB UTF-8
Unit-test suite wall time	60 s
PCRE2 trace events	50,000
PCRE2 serialized trace	10 MiB
Comparison retained differences	5,000
Comparison retained alignments	2,000
Static analysis traversal	20,000 nodes
Static analysis findings	100
Growth probe steps	24
Analysis aggregate wall time	30 s
Generated verified cases	48
Generated candidate attempts	192
Generation aggregate wall time	15 s
Generation aggregate candidates	1 MiB
Minimizer subject	1 MiB UTF-8
Minimizer candidate evaluations	2,000
Minimizer aggregate wall time	60 s
Formatter validation tests	1,000
Formatter test wall time	60 s
Retained minimizer history	200

Analysis defaults to three warm-up plus 15 measured samples (caps: 100 and 1,000), with a 2-second per-sample worker limit that can be raised to 10 seconds. Growth generation defaults to 1 MiB, preflights the central 16 MiB subject cap and rejects more than 10,000,000 repetitions. Replacement timing is skipped above an 8 Mi UTF-16-unit output estimate.

A timeout terminates actual execution. It is reported as timed out, never as “no match”. The next request creates a fresh worker.

Replacement applies exactly the authoritative matches returned under the match and capture-row limits. If either collection limit is reached, later subject text remains unchanged and the output is explicitly reported as partial. The separate output-byte limit reports a truncated output preview.

Project persistence

Project schema version 1 stores flavour, pattern, flags, options, replacement, list template, unit tests (including explicit scan-all behavior and supported per-test timeouts) and selected UI state. Import rebuilds a validated plain object; it never evaluates code and pauses live execution until review. Workbench-wide manual timeout changes are not persisted, and unsupported project resource overrides are rejected.

Interactive test text is excluded from local save and export unless explicitly enabled. Unit-test definitions are deliberate project data, but their subjects require separate explicit opt-ins for JSON export and IndexedDB save. Standalone test-suite JSON likewise omits subjects by default, is schema- and field-validated on import, and appends at most 1,000 total tests. Add, update, clone and append-import also preflight the complete in-memory suite against a 32 MiB aggregate bound. Added generated tests retain their validated generator version, seed, candidate ID and intended outcome; editing one removes that provenance. Schema v1 can remember that the corpus workspace was selected, but corpus documents, contents, outputs and results are always ephemeral and excluded from JSON and IndexedDB. Imports that try to inject those fields are rejected. Import, export and IndexedDB save enforce one 32 MiB aggregate UTF-8 JSON-document limit in addition to per-field limits. The latest local save is read through an updatedAt index cursor; loading it does not materialize the complete project store.

Development

Requirements: Node.js 22 or later and npm 11.

npm install
npm run dev

Quality and release commands:

npm run typecheck
npm run lint
npm run format:check
npm test
npm run test:conformance
npm run test:browser
npm run engines:build
npm run engines:verify
npm run engines:pcre2:build -- --source-dir /path/to/pcre2 --emcc /path/to/emcc
npm run engines:pcre2:verify
npm run engines:pcre2:install
node scripts/python-engine-pack.mjs node_modules/pyodide .engine-build/python
node scripts/install-python-engine.mjs .engine-build/python
node scripts/java-engine-pack.mjs
node scripts/install-java-engine.mjs
node scripts/php-engine-pack.mjs
node scripts/install-php-engine.mjs
node scripts/perl-engine-pack.mjs --archive /path/to/webperl_prebuilt_v0.09-beta.zip
node scripts/install-perl-engine.mjs
node scripts/ruby-engine-pack.mjs
node scripts/install-ruby-engine.mjs
node scripts/cpp-engine-pack.mjs --empp /path/to/em++
node scripts/install-cpp-engine.mjs
node scripts/build-go-engine.mjs /path/to/go
node scripts/install-go-engine.mjs
node scripts/build-rust-engine.mjs /path/to/rust-toolchain
node scripts/install-rust-engine.mjs
node scripts/dotnet-engine-pack.mjs --dotnet /path/to/dotnet
node scripts/install-dotnet-engine.mjs
npm run build
npm run toolbox:check
npm run check
npm run release:artifact

The normal build performs no network download. vite.config.ts uses base: "./"; all workers and assets are self-hosted and nested-path safe.

Toolbox SDK and Portal

The application uses Toolbox SDK 0.2.2 AppShell. Its canonical typed/static manifest source is src/toolbox/manifest.source.json; generation fails if its version or source identity drifts.

The deterministic release ZIP can be consumed by Toolbox Portal using an exact artifact URL, manifest ID, version and SHA-256. Portal builds consume the artifact; they do not build Regex Tools source.

All runtime assets are self-hosted below the release path. Deployment must serve .wasm as application/wasm, serve module/worker files with a JavaScript MIME type and allow WebAssembly compilation in script-src; see docs/PORTAL_REQUIREMENTS.md.

Flavour support and boundaries

PCRE2's application-owned ABI provides bounded compilation, matching, native substitution and separate automatic-callout tracing, copied caller-owned records, deterministic allocation cleanup and explicit result completeness. The exact source/toolchain build is reproducible offline and the verified pack is bundled with its metadata, checksums and licence.

The Python slice executes actual CPython 3.14.2 re in Pyodide 314.0.3; it does not translate a Python pattern into ECMAScript. Pyodide startup is a separate worker-load operation and is excluded from the normal execution deadline. Its checked-in pack contains a machine-verified SOURCE-MANIFEST.json with the complete base-runtime inventory, primary source hashes or immutable Git trees, the vendored Error Stack Parser and StackFrame source ports, all Pyodide CPython/Emscripten patches and an independently anchored legal file for every component. Optional wheels named by Pyodide's package catalogue are not shipped.

PHP executes actual PHP 8.5.8 preg_*; the embedded PHP runtime reports PCRE2 10.44. Selecting PHP therefore does not execute the separately built PCRE2 10.47 pack. Perl executes the unmodified WebPerl 0.09-beta runtime containing Perl 5.28.1 and is kept visibly legacy because that upstream beta was released in 2019.

Ruby, C++, Go, Rust and .NET execute their named WebAssembly runtimes. Their native range units are preserved and normalized at checked boundaries. The C++ profile defaults to libc++'s modified ECMAScript grammar; its checked-in source manifest binds the exact module to the traced Emscripten runtime, musl, emmalloc, compiler-rt, libc++ and libc++abi inputs and records libunwind as an audited non-selection. Go and Rust provide non-backtracking engines that deliberately exclude look-around and backreferences; .NET uses System.Text.RegularExpressions 10.0.10 with invariant culture.

The Java slice executes TeaVM 0.15.0's own java.util.regex class library. It does not translate a Java pattern into ECMAScript, but it is also not OpenJDK: in particular, TeaVM 0.15.0 does not implement Pattern.UNICODE_CHARACTER_CLASS. The UI reports the U flag as a compatibility request and preserves TeaVM's predefined-class and word-boundary behaviour.

Scala reuses that exact TeaVM engine as a clearly named Scala/JVM compatibility profile. It does not ship or claim Scala compiler/runtime, scala.util.matching wrapper behavior or OpenJDK parity.

Generated-case version 1 is ECMAScript-only. It traverses the complete application-owned ECMAScript AST under node, depth, attempt, byte, repetition and wall-time caps, then verifies every retained label through the actual browser RegExp adapter. PCRE2 generation remains unavailable until its syntax provider can expose a complete normalized tree.

Pattern formatting is likewise ECMAScript-only. It consumes the complete regexpp token model and does not reinterpret any partial lexical structure. Other profiles remain unavailable until complete grammar-backed implementations pass equivalent idempotence, reparse, engine and test gates.

The first advertised generated-code target is the PCRE2 10.47 8-bit C API. Its deterministic match and replacement fixtures compile and execute against the exact official release. See docs/COMPARISON_AND_CODEGEN.md.

No profile is emulated through JavaScript and relabelled. The sole shared runtime is explicit: Scala compatibility intentionally delegates to the verified TeaVM Java engine and labels that boundary in the selector and result identity.

Licensing

Regex Tools original source is GPL-3.0-or-later, copyright © 2026 Albrecht Degering. Shipped dependencies retain their own compatible licences and notices; see THIRD_PARTY_NOTICES.md and LICENSES/.

The PHP 8.5.8 codebase is distributed under the elected PHP License version 4 (BSD-3-Clause) option, while WordPress Playground's @php-wasm packages retain their separate GPL-2.0-or-later notice. Each runtime pack contains the exact applicable licence and notice material recorded in its metadata.

No regex101 application source, branding, generated explanation prose, reference text or visual design is copied. RegexLib and RGXP.RU fixtures are not redistributed because a suitable fixture licence was not established. No RegexHub fixture is shipped in v0.4.0.

Known limitations

• ECMAScript syntax coverage follows regexpp's ECMAScript 2025 grammar; a later browser proposal may execute before the provider understands it.
• Exact consumed-length bounds for ECMAScript Unicode properties of strings remain unknown and are reported as such rather than guessed.
• regexpp does not provide tolerant AST recovery. Malformed input receives an exact reported error position as a zero-width range and an application-owned error node, not a fabricated multi-character span or partial provider AST.
• Native JavaScript compile errors do not consistently expose a machine-readable source offset; the provider diagnostic supplies the range.
• Capture history is unavailable. Browser ECMAScript exposes no engine trace; PCRE2 exposes bounded automatic callouts, not a complete internal trace. Movement classifications are derived and labelled as such.
• Static risk heuristics and generated-input observations apply only to ECMAScript 2025 and are advisory. They are never proof of safety or of a pattern’s general time complexity; analysis settings and results are ephemeral.
• PCRE2 structural explanations are deliberately partial. The application recognizes common capture forms and PCRE2-only constructs for reference, but the actual PCRE2 compiler is authoritative. Branch-reset capture numbering is omitted from the explanation rather than guessed.
• Python, Java, PHP, Perl, Ruby, C++, Go, Rust, .NET and Scala compatibility structural explanations are deliberately partial lexical models. The selected runtime remains authoritative for compilation, group count and matching.
• Python, Ruby, C++ and Perl report Unicode code-point offsets; Java, Scala, .NET and ECMAScript report UTF-16 offsets; standalone PCRE2, PHP, Go and Rust report UTF-8 bytes. Every range is checked before conversion and its native unit remains visible.
• TeaVM java.util.regex is not OpenJDK. Its U compatibility request does not add OpenJDK's UNICODE_CHARACTER_CLASS semantics; the same boundary applies to the Scala compatibility profile.
• WebPerl 0.09-beta and Perl 5.28.1 are intentionally legacy. The shipped upstream loader contains a dormant optional JavaScript-interoperability eval site, although the fixed bridge does not import or invoke it.
• Capture-table rendering is paged at 200 rows while aggregate collection is bounded separately.
• Explanation/extraction trees and editor decorations show bounded 2,000-node previews with explicit totals; normalized results and table/export data stay separate from those render-only caps.
• Extraction-tree values show at most 100 UTF-16 units and capture-table cells at most 512. UI-only shortening shows the exact range length; engine-clipped values are labelled as prefixes of that range. Copy and eligible JSON export continue to use the separately bounded engine value.
• Replacement parsing is complete only for the providers that advertise that coverage; the v0.4.0 community providers label their lexical replacement models as partial and defer semantics to the runtime. Editor marks, token rows and diagnostics have explicit presentation caps. List token chips are similarly bounded, and list rendering stops at 250,000 token evaluations.
• Per-match replacement presentation renders at most 200 matches and 2,000 token contributions under a 50,000 token-evaluation budget. Bounded replacement output remains available separately and is labelled partial if match collection reached a limit.
• Result-value previews are capped per value and overall. List export is disabled whenever a bounded preview would otherwise be mistaken for complete data.
• Unit-test failure messages retain at most 2 KiB UTF-8 per case, so 1,000 results stay below the 2 MiB log budget. Oversized exact replacement results offer a lightweight should-match draft when one is valid and fits the suite.
• Corpus input accepts recognized UTF-8 text only. It does not auto-detect legacy encodings or recursively traverse directories.
• Corpus capture extraction is a bounded per-document summary: full match plus at most 31 capture groups, three 160-unit samples each. Content-free reports include counts and clipping metadata, never the samples themselves.
• Generated cases deliberately sample only ECMAScript and report backreferences, lookaround, boundaries, complex Unicode sets and other unsupported constraints instead of claiming exhaustive coverage.
• Pattern formatting is intentionally narrow and ECMAScript-only. Its bounded source/candidate and unit-test validation is evidence for the checked snapshots, not a proof of equivalence over all subjects.
• Cross-flavour comparison, PCRE2 C generation and subject minimization remain scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting any other profile does not silently substitute one of those engines.

Architecture, security, performance, provenance and flavour details live in docs/.