feat: add multi-engine regex flavour support

This commit is contained in:
2026-07-27 11:43:51 +02:00
parent 7079cde15f
commit 7f3a91ad37
340 changed files with 643286 additions and 483 deletions

201
README.md
View File

@@ -8,18 +8,37 @@ Patterns, replacement templates, test text and project files stay in the
browser. There is no account, backend, upload, remote execution, telemetry,
runtime CDN, remote font, or cloud-save service.
## Current release — 0.3.0
## Current release — 0.4.0
The v0.3.0 release contains four executable engine slices:
The v0.4.0 release contains twelve selectable, executable profiles:
- the current browser's native ECMAScript `RegExp`;
- official PCRE2 10.47 as a self-hosted 8-bit WebAssembly pack;
- CPython 3.14.2 `re` in the self-hosted Pyodide 314.0.3 runtime; and
- TeaVM 0.15.0's `java.util.regex` class-library implementation as a
self-hosted ES2015 module.
- **JavaScript (ECMAScript)** — the current browser's native `RegExp`, with
ECMAScript 2025 syntax supplied by regexpp 4.12.2;
- **PCRE2** — official standalone PCRE2 10.47 as a self-hosted 8-bit
WebAssembly pack;
- **PHP** — actual PHP 8.5.8 `preg_*` in `@php-wasm/web-8-5` 3.1.46; this
runtime reports PCRE2 10.44 and is deliberately distinct from the standalone
PCRE2 10.47 profile;
- **Perl** — actual Perl 5.28.1 in WebPerl 0.09-beta, prominently identified as
a legacy beta compatibility target rather than current desktop Perl;
- **Python** — CPython 3.14.2 `re` in self-hosted Pyodide 314.0.3;
- **Ruby** — CRuby 4.0.0 `Regexp` in the minimal ruby.wasm
2.9.3-2.9.4 runtime;
- **Java** — TeaVM 0.15.0's `java.util.regex` class-library implementation;
- **C++** — Emscripten 6.0.4 libc++ `std::wregex`, using the C++ modified
ECMAScript grammar by default and exposing the standard alternate grammar
choices;
- **Go** — Go 1.26.5 standard-library `regexp` with RE2 syntax;
- **Rust** — the Rust `regex` crate 1.13.1, built with rustc 1.97.1;
- **.NET** — .NET 10.0.10 `System.Text.RegularExpressions` in browser
WebAssembly with invariant-culture execution; and
- **Scala compatibility** — Scala/JVM regex syntax and replacement semantics
through the same TeaVM `java.util.regex` engine as the Java profile. No Scala
runtime or Scala `Regex` wrapper API is shipped.
The Java flavour is TeaVM class-library code compiled to JavaScript. It is not
OpenJDK and is not advertised as an OpenJDK compatibility oracle.
Every profile executes the named runtime. Java is TeaVM class-library code,
not OpenJDK. Scala is an explicitly labelled Java-compatibility profile, not a
relabeled Scala engine.
- Syntax acceptance: `@eslint-community/regexpp` 4.12.2, ECMAScript 2025
grammar; application-owned structural explanations are partial and
@@ -33,22 +52,55 @@ OpenJDK and is not advertised as an OpenJDK compatibility oracle.
- PCRE2 automatic callouts are copied through a separate ABI operation and
displayed from a dedicated killable trace worker under 50,000-event and
10 MiB serialized-data hard caps.
- PHP compilation and match selection use actual `preg_match()` in PHP 8.5.8.
A fixed byte-bounded expander reproduces PHP replacement tokens and is
checked against native `preg_replace()` at load. PHP returns native UTF-8
byte ranges and reports its own PCRE2 10.44 identity.
- The legacy Perl bridge passes user values as JSON data to fixed application
code, disables runtime regex evaluation, rejects code assertions and exposes
a pre-tokenized, deliberately bounded replacement grammar. Replacement bytes
stream from native spans to a separate fixed MEMFS file instead of being
materialized or JSON-escaped; the worker validates the exact bounded UTF-8
file before returning it. Its prebuilt WebPerl loader has a dormant optional
interoperability `eval` site; the fixed engine path is verified without broad
CSP `'unsafe-eval'`.
- Python compilation, matching and replacement use the bundled CPython `re`
module. Native code-point ranges are retained and normalized to exact editor
UTF-16 ranges.
UTF-16 ranges. Bounded replacement UTF-8 crosses the JSON bridge as canonical
base64; the adapter validates the declared byte count, cap and UTF-8 before
exposing output.
- Ruby compilation and matching use CRuby's `Regexp`; native `String#gsub`
and `String#sub` block iteration drives replacement matching, while a fixed
byte-bounded implementation of CRuby 4.0.0's `rb_reg_regsub` template
grammar produces the output prefix. Native character ranges are retained as
Unicode code-point offsets. Requests and replacement output use separate
bounded files in the worker-local WASI memory filesystem; only bounded
metadata returns as inert JSON, avoiding both output escaping amplification
and ruby.wasm's JavaScript-eval conversion path.
- Java compilation, matching and replacement use TeaVM's
`java.util.regex.Pattern` and `Matcher`. Native ranges are UTF-16 code units.
- TeaVM 0.15.0 replacement supports its actual single-digit `$n` subset.
`$12` is therefore `$1` followed by literal `2`; Java SE `${name}`
replacement is reported as unavailable rather than presented as OpenJDK
behavior.
- Python and Java use deliberately partial application-owned lexical syntax
providers; their actual runtime compiler remains authoritative.
- C++ compilation and matching use libc++ `std::wregex`; a fixed byte-bounded
formatter reproduces `match.format` grammar and checks parity with that
native API at load. Its 32-bit `wchar_t` ranges are Unicode code points.
- Go and Rust return native UTF-8 byte offsets from their real runtimes. Their
fixed bounded expanders are parity-checked against `Regexp.ExpandString` and
`Captures::expand`, then return output through predictable base64 envelopes.
Their adapters measure exact encoded request bytes, and their native JSON
caps cover worst-case escaping for every accepted input field.
- .NET returns native UTF-16 offsets and executes with a bounded native regex
timeout in addition to the killable worker boundary.
- Python, Java and all eight v0.4.0 community additions use deliberately
partial application-owned lexical syntax providers; their actual runtime
compiler remains authoritative.
- Parsing and execution run in separate workers.
- Actual execution can be terminated by killing its worker.
- The worker is recreated after timeout, crash or cancellation.
- ECMAScript capture ranges use the engine's `d` indices result; the other
flavours return their native range records through bounded bridges.
- ECMAScript capture ranges use the engine's `d` indices result; every other
profile returns its native range records through a bounded fixed bridge.
- Named, numbered, optional, empty and repeated-final captures are represented.
- Match, replacement, typed list/extraction, corpus/apply and unit-test modes
are available.
@@ -92,8 +144,10 @@ as a complete record of every internal engine action.
## Workbench modes
- **Match** — live highlighting, extraction tree and bounded capture table.
- **Replace** — engine-native bounded substitution for all four executable
flavours;
- **Replace** — bounded profile-authoritative substitution for all twelve
executable profiles; the legacy Perl profile uses its documented fixed
non-executable template grammar, and profiles that use fixed bounded
expanders identify and parity-check them against their named runtime;
output completeness is reported separately from bounded per-match
original/result, changed-range and token-contribution previews that
synchronize the replacement, pattern, subject and output editors.
@@ -138,15 +192,16 @@ unit test and explicit confirmation; see
## Flags and iteration
Pattern and flags are stored separately; delimiters are presentation only.
ECMAScript preserves native `g` and `y`; PCRE2 offers the application-level
`g` iteration flag plus `i`, `m`, `s`, `x`, `U` and `J`. PCRE2 UTF/UCP mode is
mandatory because browser strings are Unicode and partial-byte ranges cannot be
represented safely in the editor. Python exposes application-level `g` plus
CPython `re` flags `a`, `i`, `m`, `s` and `x`. Java exposes application-level
`g` plus TeaVM `Pattern` flags `i`, `d`, `m`, `s`, `u`, `x` and the documented
`U` compatibility request. The optional **Scan all** action adds only an
internal iteration request and does not change saved flags. ECMAScript may add
internal `d` to obtain exact indices without changing match semantics.
Each profile exposes only its reviewed allowlist, and application-level `g`
means bounded iteration where the underlying API has no compile-time global
flag. PHP delimiters/modifiers, Go and Rust inline options, C++ grammar
selection, .NET options and the legacy Perl character-set modes are mapped by
their own adapters rather than reused across profiles. Standalone PCRE2 always
uses UTF/UCP; PHP controls UTF through its exposed `u` modifier and reports its
own PCRE2 version. The optional **Scan all** action adds only an internal
iteration request and does not change saved flags. ECMAScript may add internal
`d` to obtain exact indices without changing match semantics. The exact flag
matrix is in [`docs/FLAVOUR_SUPPORT.md`](docs/FLAVOUR_SUPPORT.md).
Zero-length global iteration advances using the selected Unicode semantics and
cannot loop forever.
@@ -273,6 +328,20 @@ node scripts/python-engine-pack.mjs node_modules/pyodide .engine-build/python
node scripts/install-python-engine.mjs .engine-build/python
node scripts/java-engine-pack.mjs
node scripts/install-java-engine.mjs
node scripts/php-engine-pack.mjs
node scripts/install-php-engine.mjs
node scripts/perl-engine-pack.mjs --archive /path/to/webperl_prebuilt_v0.09-beta.zip
node scripts/install-perl-engine.mjs
node scripts/ruby-engine-pack.mjs
node scripts/install-ruby-engine.mjs
node scripts/cpp-engine-pack.mjs --empp /path/to/em++
node scripts/install-cpp-engine.mjs
node scripts/build-go-engine.mjs /path/to/go
node scripts/install-go-engine.mjs
node scripts/build-rust-engine.mjs /path/to/rust-toolchain
node scripts/install-rust-engine.mjs
node scripts/dotnet-engine-pack.mjs --dotnet /path/to/dotnet
node scripts/install-dotnet-engine.mjs
npm run build
npm run toolbox:check
npm run check
@@ -292,10 +361,9 @@ The deterministic release ZIP can be consumed by Toolbox Portal using an exact
artifact URL, manifest ID, version and SHA-256. Portal builds consume the
artifact; they do not build Regex Tools source.
The PCRE2 and Python workers load only self-hosted JavaScript, WebAssembly and
Python-standard-library assets; Java loads its self-hosted ES module.
Deployment must serve `.wasm` as `application/wasm` and allow WebAssembly
compilation in `script-src`; see
All runtime assets are self-hosted below the release path. Deployment must
serve `.wasm` as `application/wasm`, serve module/worker files with a
JavaScript MIME type and allow WebAssembly compilation in `script-src`; see
[`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).
## Flavour support and boundaries
@@ -309,6 +377,27 @@ is bundled with its metadata, checksums and licence.
The Python slice executes actual CPython 3.14.2 `re` in Pyodide 314.0.3; it does
not translate a Python pattern into ECMAScript. Pyodide startup is a separate
worker-load operation and is excluded from the normal execution deadline.
Its checked-in pack contains a machine-verified `SOURCE-MANIFEST.json` with the
complete base-runtime inventory, primary source hashes or immutable Git trees,
the vendored Error Stack Parser and StackFrame source ports, all Pyodide
CPython/Emscripten patches and an independently anchored legal file for every
component. Optional wheels named by Pyodide's package catalogue are not
shipped.
PHP executes actual PHP 8.5.8 `preg_*`; the embedded PHP runtime reports PCRE2
10.44. Selecting PHP therefore does not execute the separately built PCRE2
10.47 pack. Perl executes the unmodified WebPerl 0.09-beta runtime containing
Perl 5.28.1 and is kept visibly legacy because that upstream beta was released
in 2019.
Ruby, C++, Go, Rust and .NET execute their named WebAssembly runtimes. Their
native range units are preserved and normalized at checked boundaries. The
C++ profile defaults to libc++'s modified ECMAScript grammar; its checked-in
source manifest binds the exact module to the traced Emscripten runtime, musl,
emmalloc, compiler-rt, libc++ and libc++abi inputs and records libunwind as an
audited non-selection. Go and Rust provide non-backtracking engines that
deliberately exclude look-around and backreferences; .NET uses
`System.Text.RegularExpressions` 10.0.10 with invariant culture.
The Java slice executes TeaVM 0.15.0's own `java.util.regex` class library. It
does not translate a Java pattern into ECMAScript, but it is also not OpenJDK:
@@ -317,6 +406,10 @@ in particular, TeaVM 0.15.0 does not implement
compatibility request and preserves TeaVM's predefined-class and word-boundary
behaviour.
Scala reuses that exact TeaVM engine as a clearly named Scala/JVM compatibility
profile. It does not ship or claim Scala compiler/runtime, `scala.util.matching`
wrapper behavior or OpenJDK parity.
Generated-case version 1 is ECMAScript-only. It traverses the complete
application-owned ECMAScript AST under node, depth, attempt, byte, repetition
and wall-time caps, then verifies every retained label through the actual
@@ -324,8 +417,8 @@ browser `RegExp` adapter. PCRE2 generation remains unavailable until its syntax
provider can expose a complete normalized tree.
Pattern formatting is likewise ECMAScript-only. It consumes the complete
regexpp token model and does not reinterpret the partial PCRE2, Python or Java
structures. Those flavours remain unavailable until complete grammar-backed
regexpp token model and does not reinterpret any partial lexical structure.
Other profiles remain unavailable until complete grammar-backed
implementations pass equivalent idempotence, reparse, engine and test gates.
The first advertised generated-code target is the PCRE2 10.47 8-bit C API.
@@ -333,8 +426,10 @@ Its deterministic match and replacement fixtures compile and execute against
the exact official release. See
[`docs/COMPARISON_AND_CODEGEN.md`](docs/COMPARISON_AND_CODEGEN.md).
Go, Rust, .NET and legacy PCRE follow only through their actual named runtimes.
No flavour is emulated through JavaScript and relabelled.
No profile is emulated through JavaScript and relabelled. The sole shared
runtime is explicit: Scala compatibility intentionally delegates to the
verified TeaVM Java engine and labels that boundary in the selector and result
identity.
## Licensing
@@ -343,10 +438,15 @@ Degering. Shipped dependencies retain their own compatible licences and
notices; see [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and
[`LICENSES/`](LICENSES/).
The PHP 8.5.8 codebase is distributed under the elected PHP License version 4
(`BSD-3-Clause`) option, while WordPress Playground's `@php-wasm` packages
retain their separate GPL-2.0-or-later notice. Each runtime pack contains the
exact applicable licence and notice material recorded in its metadata.
No regex101 application source, branding, generated explanation prose,
reference text or visual design is copied. RegexLib and RGXP.RU fixtures are not
redistributed because a suitable fixture licence was not established. No
RegexHub fixture is shipped in v0.3.0.
RegexHub fixture is shipped in v0.4.0.
## Known limitations
@@ -370,14 +470,20 @@ RegexHub fixture is shipped in v0.3.0.
recognizes common capture forms and PCRE2-only constructs for reference, but
the actual PCRE2 compiler is authoritative. Branch-reset capture numbering is
omitted from the explanation rather than guessed.
- Python and Java structural explanations are deliberately partial lexical
models. CPython `re` and TeaVM `Pattern` respectively remain authoritative
for compilation, group count and matching.
- Python reports native Unicode code-point offsets and converts them at valid
boundaries to editor UTF-16 ranges. Java and ECMAScript report native UTF-16
offsets; PCRE2 reports native UTF-8 bytes. Native units remain visible.
- Python, Java, PHP, Perl, Ruby, C++, Go, Rust, .NET and Scala compatibility
structural explanations are deliberately partial lexical models. The
selected runtime remains authoritative for compilation, group count and
matching.
- Python, Ruby, C++ and Perl report Unicode code-point offsets; Java, Scala,
.NET and ECMAScript report UTF-16 offsets; standalone PCRE2, PHP, Go and Rust
report UTF-8 bytes. Every range is checked before conversion and its native
unit remains visible.
- TeaVM `java.util.regex` is not OpenJDK. Its `U` compatibility request does
not add OpenJDK's `UNICODE_CHARACTER_CLASS` semantics.
not add OpenJDK's `UNICODE_CHARACTER_CLASS` semantics; the same boundary
applies to the Scala compatibility profile.
- WebPerl 0.09-beta and Perl 5.28.1 are intentionally legacy. The shipped
upstream loader contains a dormant optional JavaScript-interoperability
`eval` site, although the fixed bridge does not import or invoke it.
- Capture-table rendering is paged at 200 rows while aggregate collection is
bounded separately.
- Explanation/extraction trees and editor decorations show bounded 2,000-node
@@ -387,10 +493,11 @@ RegexHub fixture is shipped in v0.3.0.
at most 512. UI-only shortening shows the exact range length; engine-clipped
values are labelled as prefixes of that range. Copy and eligible JSON export
continue to use the separately bounded engine value.
- Replacement parsing is complete below its 64 Ki input limit, while editor
marks, token rows and diagnostics have explicit presentation caps. List token
chips are similarly bounded, and list rendering stops at 250,000 token
evaluations.
- Replacement parsing is complete only for the providers that advertise that
coverage; the v0.4.0 community providers label their lexical replacement
models as partial and defer semantics to the runtime. Editor marks, token
rows and diagnostics have explicit presentation caps. List token chips are
similarly bounded, and list rendering stops at 250,000 token evaluations.
- Per-match replacement presentation renders at most 200 matches and 2,000
token contributions under a 50,000 token-evaluation budget. Bounded
replacement output remains available separately and is labelled partial if
@@ -413,8 +520,8 @@ RegexHub fixture is shipped in v0.3.0.
source/candidate and unit-test validation is evidence for the checked
snapshots, not a proof of equivalence over all subjects.
- Cross-flavour comparison, PCRE2 C generation and subject minimization remain
scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting
Python or Java does not silently substitute one of those engines.
scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting any
other profile does not silently substitute one of those engines.
Architecture, security, performance, provenance and flavour details live in
[`docs/`](docs/).