feat: add multi-engine regex flavour support
This commit is contained in:
201
README.md
201
README.md
@@ -8,18 +8,37 @@ Patterns, replacement templates, test text and project files stay in the
|
||||
browser. There is no account, backend, upload, remote execution, telemetry,
|
||||
runtime CDN, remote font, or cloud-save service.
|
||||
|
||||
## Current release — 0.3.0
|
||||
## Current release — 0.4.0
|
||||
|
||||
The v0.3.0 release contains four executable engine slices:
|
||||
The v0.4.0 release contains twelve selectable, executable profiles:
|
||||
|
||||
- the current browser's native ECMAScript `RegExp`;
|
||||
- official PCRE2 10.47 as a self-hosted 8-bit WebAssembly pack;
|
||||
- CPython 3.14.2 `re` in the self-hosted Pyodide 314.0.3 runtime; and
|
||||
- TeaVM 0.15.0's `java.util.regex` class-library implementation as a
|
||||
self-hosted ES2015 module.
|
||||
- **JavaScript (ECMAScript)** — the current browser's native `RegExp`, with
|
||||
ECMAScript 2025 syntax supplied by regexpp 4.12.2;
|
||||
- **PCRE2** — official standalone PCRE2 10.47 as a self-hosted 8-bit
|
||||
WebAssembly pack;
|
||||
- **PHP** — actual PHP 8.5.8 `preg_*` in `@php-wasm/web-8-5` 3.1.46; this
|
||||
runtime reports PCRE2 10.44 and is deliberately distinct from the standalone
|
||||
PCRE2 10.47 profile;
|
||||
- **Perl** — actual Perl 5.28.1 in WebPerl 0.09-beta, prominently identified as
|
||||
a legacy beta compatibility target rather than current desktop Perl;
|
||||
- **Python** — CPython 3.14.2 `re` in self-hosted Pyodide 314.0.3;
|
||||
- **Ruby** — CRuby 4.0.0 `Regexp` in the minimal ruby.wasm
|
||||
2.9.3-2.9.4 runtime;
|
||||
- **Java** — TeaVM 0.15.0's `java.util.regex` class-library implementation;
|
||||
- **C++** — Emscripten 6.0.4 libc++ `std::wregex`, using the C++ modified
|
||||
ECMAScript grammar by default and exposing the standard alternate grammar
|
||||
choices;
|
||||
- **Go** — Go 1.26.5 standard-library `regexp` with RE2 syntax;
|
||||
- **Rust** — the Rust `regex` crate 1.13.1, built with rustc 1.97.1;
|
||||
- **.NET** — .NET 10.0.10 `System.Text.RegularExpressions` in browser
|
||||
WebAssembly with invariant-culture execution; and
|
||||
- **Scala compatibility** — Scala/JVM regex syntax and replacement semantics
|
||||
through the same TeaVM `java.util.regex` engine as the Java profile. No Scala
|
||||
runtime or Scala `Regex` wrapper API is shipped.
|
||||
|
||||
The Java flavour is TeaVM class-library code compiled to JavaScript. It is not
|
||||
OpenJDK and is not advertised as an OpenJDK compatibility oracle.
|
||||
Every profile executes the named runtime. Java is TeaVM class-library code,
|
||||
not OpenJDK. Scala is an explicitly labelled Java-compatibility profile, not a
|
||||
relabeled Scala engine.
|
||||
|
||||
- Syntax acceptance: `@eslint-community/regexpp` 4.12.2, ECMAScript 2025
|
||||
grammar; application-owned structural explanations are partial and
|
||||
@@ -33,22 +52,55 @@ OpenJDK and is not advertised as an OpenJDK compatibility oracle.
|
||||
- PCRE2 automatic callouts are copied through a separate ABI operation and
|
||||
displayed from a dedicated killable trace worker under 50,000-event and
|
||||
10 MiB serialized-data hard caps.
|
||||
- PHP compilation and match selection use actual `preg_match()` in PHP 8.5.8.
|
||||
A fixed byte-bounded expander reproduces PHP replacement tokens and is
|
||||
checked against native `preg_replace()` at load. PHP returns native UTF-8
|
||||
byte ranges and reports its own PCRE2 10.44 identity.
|
||||
- The legacy Perl bridge passes user values as JSON data to fixed application
|
||||
code, disables runtime regex evaluation, rejects code assertions and exposes
|
||||
a pre-tokenized, deliberately bounded replacement grammar. Replacement bytes
|
||||
stream from native spans to a separate fixed MEMFS file instead of being
|
||||
materialized or JSON-escaped; the worker validates the exact bounded UTF-8
|
||||
file before returning it. Its prebuilt WebPerl loader has a dormant optional
|
||||
interoperability `eval` site; the fixed engine path is verified without broad
|
||||
CSP `'unsafe-eval'`.
|
||||
- Python compilation, matching and replacement use the bundled CPython `re`
|
||||
module. Native code-point ranges are retained and normalized to exact editor
|
||||
UTF-16 ranges.
|
||||
UTF-16 ranges. Bounded replacement UTF-8 crosses the JSON bridge as canonical
|
||||
base64; the adapter validates the declared byte count, cap and UTF-8 before
|
||||
exposing output.
|
||||
- Ruby compilation and matching use CRuby's `Regexp`; native `String#gsub`
|
||||
and `String#sub` block iteration drives replacement matching, while a fixed
|
||||
byte-bounded implementation of CRuby 4.0.0's `rb_reg_regsub` template
|
||||
grammar produces the output prefix. Native character ranges are retained as
|
||||
Unicode code-point offsets. Requests and replacement output use separate
|
||||
bounded files in the worker-local WASI memory filesystem; only bounded
|
||||
metadata returns as inert JSON, avoiding both output escaping amplification
|
||||
and ruby.wasm's JavaScript-eval conversion path.
|
||||
- Java compilation, matching and replacement use TeaVM's
|
||||
`java.util.regex.Pattern` and `Matcher`. Native ranges are UTF-16 code units.
|
||||
- TeaVM 0.15.0 replacement supports its actual single-digit `$n` subset.
|
||||
`$12` is therefore `$1` followed by literal `2`; Java SE `${name}`
|
||||
replacement is reported as unavailable rather than presented as OpenJDK
|
||||
behavior.
|
||||
- Python and Java use deliberately partial application-owned lexical syntax
|
||||
providers; their actual runtime compiler remains authoritative.
|
||||
- C++ compilation and matching use libc++ `std::wregex`; a fixed byte-bounded
|
||||
formatter reproduces `match.format` grammar and checks parity with that
|
||||
native API at load. Its 32-bit `wchar_t` ranges are Unicode code points.
|
||||
- Go and Rust return native UTF-8 byte offsets from their real runtimes. Their
|
||||
fixed bounded expanders are parity-checked against `Regexp.ExpandString` and
|
||||
`Captures::expand`, then return output through predictable base64 envelopes.
|
||||
Their adapters measure exact encoded request bytes, and their native JSON
|
||||
caps cover worst-case escaping for every accepted input field.
|
||||
- .NET returns native UTF-16 offsets and executes with a bounded native regex
|
||||
timeout in addition to the killable worker boundary.
|
||||
- Python, Java and all eight v0.4.0 community additions use deliberately
|
||||
partial application-owned lexical syntax providers; their actual runtime
|
||||
compiler remains authoritative.
|
||||
- Parsing and execution run in separate workers.
|
||||
- Actual execution can be terminated by killing its worker.
|
||||
- The worker is recreated after timeout, crash or cancellation.
|
||||
- ECMAScript capture ranges use the engine's `d` indices result; the other
|
||||
flavours return their native range records through bounded bridges.
|
||||
- ECMAScript capture ranges use the engine's `d` indices result; every other
|
||||
profile returns its native range records through a bounded fixed bridge.
|
||||
- Named, numbered, optional, empty and repeated-final captures are represented.
|
||||
- Match, replacement, typed list/extraction, corpus/apply and unit-test modes
|
||||
are available.
|
||||
@@ -92,8 +144,10 @@ as a complete record of every internal engine action.
|
||||
## Workbench modes
|
||||
|
||||
- **Match** — live highlighting, extraction tree and bounded capture table.
|
||||
- **Replace** — engine-native bounded substitution for all four executable
|
||||
flavours;
|
||||
- **Replace** — bounded profile-authoritative substitution for all twelve
|
||||
executable profiles; the legacy Perl profile uses its documented fixed
|
||||
non-executable template grammar, and profiles that use fixed bounded
|
||||
expanders identify and parity-check them against their named runtime;
|
||||
output completeness is reported separately from bounded per-match
|
||||
original/result, changed-range and token-contribution previews that
|
||||
synchronize the replacement, pattern, subject and output editors.
|
||||
@@ -138,15 +192,16 @@ unit test and explicit confirmation; see
|
||||
## Flags and iteration
|
||||
|
||||
Pattern and flags are stored separately; delimiters are presentation only.
|
||||
ECMAScript preserves native `g` and `y`; PCRE2 offers the application-level
|
||||
`g` iteration flag plus `i`, `m`, `s`, `x`, `U` and `J`. PCRE2 UTF/UCP mode is
|
||||
mandatory because browser strings are Unicode and partial-byte ranges cannot be
|
||||
represented safely in the editor. Python exposes application-level `g` plus
|
||||
CPython `re` flags `a`, `i`, `m`, `s` and `x`. Java exposes application-level
|
||||
`g` plus TeaVM `Pattern` flags `i`, `d`, `m`, `s`, `u`, `x` and the documented
|
||||
`U` compatibility request. The optional **Scan all** action adds only an
|
||||
internal iteration request and does not change saved flags. ECMAScript may add
|
||||
internal `d` to obtain exact indices without changing match semantics.
|
||||
Each profile exposes only its reviewed allowlist, and application-level `g`
|
||||
means bounded iteration where the underlying API has no compile-time global
|
||||
flag. PHP delimiters/modifiers, Go and Rust inline options, C++ grammar
|
||||
selection, .NET options and the legacy Perl character-set modes are mapped by
|
||||
their own adapters rather than reused across profiles. Standalone PCRE2 always
|
||||
uses UTF/UCP; PHP controls UTF through its exposed `u` modifier and reports its
|
||||
own PCRE2 version. The optional **Scan all** action adds only an internal
|
||||
iteration request and does not change saved flags. ECMAScript may add internal
|
||||
`d` to obtain exact indices without changing match semantics. The exact flag
|
||||
matrix is in [`docs/FLAVOUR_SUPPORT.md`](docs/FLAVOUR_SUPPORT.md).
|
||||
|
||||
Zero-length global iteration advances using the selected Unicode semantics and
|
||||
cannot loop forever.
|
||||
@@ -273,6 +328,20 @@ node scripts/python-engine-pack.mjs node_modules/pyodide .engine-build/python
|
||||
node scripts/install-python-engine.mjs .engine-build/python
|
||||
node scripts/java-engine-pack.mjs
|
||||
node scripts/install-java-engine.mjs
|
||||
node scripts/php-engine-pack.mjs
|
||||
node scripts/install-php-engine.mjs
|
||||
node scripts/perl-engine-pack.mjs --archive /path/to/webperl_prebuilt_v0.09-beta.zip
|
||||
node scripts/install-perl-engine.mjs
|
||||
node scripts/ruby-engine-pack.mjs
|
||||
node scripts/install-ruby-engine.mjs
|
||||
node scripts/cpp-engine-pack.mjs --empp /path/to/em++
|
||||
node scripts/install-cpp-engine.mjs
|
||||
node scripts/build-go-engine.mjs /path/to/go
|
||||
node scripts/install-go-engine.mjs
|
||||
node scripts/build-rust-engine.mjs /path/to/rust-toolchain
|
||||
node scripts/install-rust-engine.mjs
|
||||
node scripts/dotnet-engine-pack.mjs --dotnet /path/to/dotnet
|
||||
node scripts/install-dotnet-engine.mjs
|
||||
npm run build
|
||||
npm run toolbox:check
|
||||
npm run check
|
||||
@@ -292,10 +361,9 @@ The deterministic release ZIP can be consumed by Toolbox Portal using an exact
|
||||
artifact URL, manifest ID, version and SHA-256. Portal builds consume the
|
||||
artifact; they do not build Regex Tools source.
|
||||
|
||||
The PCRE2 and Python workers load only self-hosted JavaScript, WebAssembly and
|
||||
Python-standard-library assets; Java loads its self-hosted ES module.
|
||||
Deployment must serve `.wasm` as `application/wasm` and allow WebAssembly
|
||||
compilation in `script-src`; see
|
||||
All runtime assets are self-hosted below the release path. Deployment must
|
||||
serve `.wasm` as `application/wasm`, serve module/worker files with a
|
||||
JavaScript MIME type and allow WebAssembly compilation in `script-src`; see
|
||||
[`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).
|
||||
|
||||
## Flavour support and boundaries
|
||||
@@ -309,6 +377,27 @@ is bundled with its metadata, checksums and licence.
|
||||
The Python slice executes actual CPython 3.14.2 `re` in Pyodide 314.0.3; it does
|
||||
not translate a Python pattern into ECMAScript. Pyodide startup is a separate
|
||||
worker-load operation and is excluded from the normal execution deadline.
|
||||
Its checked-in pack contains a machine-verified `SOURCE-MANIFEST.json` with the
|
||||
complete base-runtime inventory, primary source hashes or immutable Git trees,
|
||||
the vendored Error Stack Parser and StackFrame source ports, all Pyodide
|
||||
CPython/Emscripten patches and an independently anchored legal file for every
|
||||
component. Optional wheels named by Pyodide's package catalogue are not
|
||||
shipped.
|
||||
|
||||
PHP executes actual PHP 8.5.8 `preg_*`; the embedded PHP runtime reports PCRE2
|
||||
10.44. Selecting PHP therefore does not execute the separately built PCRE2
|
||||
10.47 pack. Perl executes the unmodified WebPerl 0.09-beta runtime containing
|
||||
Perl 5.28.1 and is kept visibly legacy because that upstream beta was released
|
||||
in 2019.
|
||||
|
||||
Ruby, C++, Go, Rust and .NET execute their named WebAssembly runtimes. Their
|
||||
native range units are preserved and normalized at checked boundaries. The
|
||||
C++ profile defaults to libc++'s modified ECMAScript grammar; its checked-in
|
||||
source manifest binds the exact module to the traced Emscripten runtime, musl,
|
||||
emmalloc, compiler-rt, libc++ and libc++abi inputs and records libunwind as an
|
||||
audited non-selection. Go and Rust provide non-backtracking engines that
|
||||
deliberately exclude look-around and backreferences; .NET uses
|
||||
`System.Text.RegularExpressions` 10.0.10 with invariant culture.
|
||||
|
||||
The Java slice executes TeaVM 0.15.0's own `java.util.regex` class library. It
|
||||
does not translate a Java pattern into ECMAScript, but it is also not OpenJDK:
|
||||
@@ -317,6 +406,10 @@ in particular, TeaVM 0.15.0 does not implement
|
||||
compatibility request and preserves TeaVM's predefined-class and word-boundary
|
||||
behaviour.
|
||||
|
||||
Scala reuses that exact TeaVM engine as a clearly named Scala/JVM compatibility
|
||||
profile. It does not ship or claim Scala compiler/runtime, `scala.util.matching`
|
||||
wrapper behavior or OpenJDK parity.
|
||||
|
||||
Generated-case version 1 is ECMAScript-only. It traverses the complete
|
||||
application-owned ECMAScript AST under node, depth, attempt, byte, repetition
|
||||
and wall-time caps, then verifies every retained label through the actual
|
||||
@@ -324,8 +417,8 @@ browser `RegExp` adapter. PCRE2 generation remains unavailable until its syntax
|
||||
provider can expose a complete normalized tree.
|
||||
|
||||
Pattern formatting is likewise ECMAScript-only. It consumes the complete
|
||||
regexpp token model and does not reinterpret the partial PCRE2, Python or Java
|
||||
structures. Those flavours remain unavailable until complete grammar-backed
|
||||
regexpp token model and does not reinterpret any partial lexical structure.
|
||||
Other profiles remain unavailable until complete grammar-backed
|
||||
implementations pass equivalent idempotence, reparse, engine and test gates.
|
||||
|
||||
The first advertised generated-code target is the PCRE2 10.47 8-bit C API.
|
||||
@@ -333,8 +426,10 @@ Its deterministic match and replacement fixtures compile and execute against
|
||||
the exact official release. See
|
||||
[`docs/COMPARISON_AND_CODEGEN.md`](docs/COMPARISON_AND_CODEGEN.md).
|
||||
|
||||
Go, Rust, .NET and legacy PCRE follow only through their actual named runtimes.
|
||||
No flavour is emulated through JavaScript and relabelled.
|
||||
No profile is emulated through JavaScript and relabelled. The sole shared
|
||||
runtime is explicit: Scala compatibility intentionally delegates to the
|
||||
verified TeaVM Java engine and labels that boundary in the selector and result
|
||||
identity.
|
||||
|
||||
## Licensing
|
||||
|
||||
@@ -343,10 +438,15 @@ Degering. Shipped dependencies retain their own compatible licences and
|
||||
notices; see [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and
|
||||
[`LICENSES/`](LICENSES/).
|
||||
|
||||
The PHP 8.5.8 codebase is distributed under the elected PHP License version 4
|
||||
(`BSD-3-Clause`) option, while WordPress Playground's `@php-wasm` packages
|
||||
retain their separate GPL-2.0-or-later notice. Each runtime pack contains the
|
||||
exact applicable licence and notice material recorded in its metadata.
|
||||
|
||||
No regex101 application source, branding, generated explanation prose,
|
||||
reference text or visual design is copied. RegexLib and RGXP.RU fixtures are not
|
||||
redistributed because a suitable fixture licence was not established. No
|
||||
RegexHub fixture is shipped in v0.3.0.
|
||||
RegexHub fixture is shipped in v0.4.0.
|
||||
|
||||
## Known limitations
|
||||
|
||||
@@ -370,14 +470,20 @@ RegexHub fixture is shipped in v0.3.0.
|
||||
recognizes common capture forms and PCRE2-only constructs for reference, but
|
||||
the actual PCRE2 compiler is authoritative. Branch-reset capture numbering is
|
||||
omitted from the explanation rather than guessed.
|
||||
- Python and Java structural explanations are deliberately partial lexical
|
||||
models. CPython `re` and TeaVM `Pattern` respectively remain authoritative
|
||||
for compilation, group count and matching.
|
||||
- Python reports native Unicode code-point offsets and converts them at valid
|
||||
boundaries to editor UTF-16 ranges. Java and ECMAScript report native UTF-16
|
||||
offsets; PCRE2 reports native UTF-8 bytes. Native units remain visible.
|
||||
- Python, Java, PHP, Perl, Ruby, C++, Go, Rust, .NET and Scala compatibility
|
||||
structural explanations are deliberately partial lexical models. The
|
||||
selected runtime remains authoritative for compilation, group count and
|
||||
matching.
|
||||
- Python, Ruby, C++ and Perl report Unicode code-point offsets; Java, Scala,
|
||||
.NET and ECMAScript report UTF-16 offsets; standalone PCRE2, PHP, Go and Rust
|
||||
report UTF-8 bytes. Every range is checked before conversion and its native
|
||||
unit remains visible.
|
||||
- TeaVM `java.util.regex` is not OpenJDK. Its `U` compatibility request does
|
||||
not add OpenJDK's `UNICODE_CHARACTER_CLASS` semantics.
|
||||
not add OpenJDK's `UNICODE_CHARACTER_CLASS` semantics; the same boundary
|
||||
applies to the Scala compatibility profile.
|
||||
- WebPerl 0.09-beta and Perl 5.28.1 are intentionally legacy. The shipped
|
||||
upstream loader contains a dormant optional JavaScript-interoperability
|
||||
`eval` site, although the fixed bridge does not import or invoke it.
|
||||
- Capture-table rendering is paged at 200 rows while aggregate collection is
|
||||
bounded separately.
|
||||
- Explanation/extraction trees and editor decorations show bounded 2,000-node
|
||||
@@ -387,10 +493,11 @@ RegexHub fixture is shipped in v0.3.0.
|
||||
at most 512. UI-only shortening shows the exact range length; engine-clipped
|
||||
values are labelled as prefixes of that range. Copy and eligible JSON export
|
||||
continue to use the separately bounded engine value.
|
||||
- Replacement parsing is complete below its 64 Ki input limit, while editor
|
||||
marks, token rows and diagnostics have explicit presentation caps. List token
|
||||
chips are similarly bounded, and list rendering stops at 250,000 token
|
||||
evaluations.
|
||||
- Replacement parsing is complete only for the providers that advertise that
|
||||
coverage; the v0.4.0 community providers label their lexical replacement
|
||||
models as partial and defer semantics to the runtime. Editor marks, token
|
||||
rows and diagnostics have explicit presentation caps. List token chips are
|
||||
similarly bounded, and list rendering stops at 250,000 token evaluations.
|
||||
- Per-match replacement presentation renders at most 200 matches and 2,000
|
||||
token contributions under a 50,000 token-evaluation budget. Bounded
|
||||
replacement output remains available separately and is labelled partial if
|
||||
@@ -413,8 +520,8 @@ RegexHub fixture is shipped in v0.3.0.
|
||||
source/candidate and unit-test validation is evidence for the checked
|
||||
snapshots, not a proof of equivalence over all subjects.
|
||||
- Cross-flavour comparison, PCRE2 C generation and subject minimization remain
|
||||
scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting
|
||||
Python or Java does not silently substitute one of those engines.
|
||||
scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting any
|
||||
other profile does not silently substitute one of those engines.
|
||||
|
||||
Architecture, security, performance, provenance and flavour details live in
|
||||
[`docs/`](docs/).
|
||||
|
||||
Reference in New Issue
Block a user