feat: add Python and Java regex engines

This commit is contained in:
2026-07-27 02:02:21 +02:00
parent 4951c966d4
commit 7079cde15f
95 changed files with 8866 additions and 251 deletions

103
README.md
View File

@@ -8,16 +8,23 @@ Patterns, replacement templates, test text and project files stay in the
browser. There is no account, backend, upload, remote execution, telemetry,
runtime CDN, remote font, or cloud-save service.
## Current release — 0.2.0
## Current release — 0.3.0
The v0.2.0 release contains two real production engine slices: ECMAScript and
PCRE2 10.47.
The v0.3.0 release contains four executable engine slices:
- the current browser's native ECMAScript `RegExp`;
- official PCRE2 10.47 as a self-hosted 8-bit WebAssembly pack;
- CPython 3.14.2 `re` in the self-hosted Pyodide 314.0.3 runtime; and
- TeaVM 0.15.0's `java.util.regex` class-library implementation as a
self-hosted ES2015 module.
The Java flavour is TeaVM class-library code compiled to JavaScript. It is not
OpenJDK and is not advertised as an OpenJDK compatibility oracle.
- Syntax acceptance: `@eslint-community/regexpp` 4.12.2, ECMAScript 2025
grammar; application-owned structural explanations are partial and
feature-matrix tested.
- Execution engine: the current browser's native ECMAScript `RegExp`.
- Native and editor offsets: UTF-16 code units.
- ECMAScript native and editor offsets are UTF-16 code units.
- PCRE2 execution and substitution: pinned official PCRE2 10.47 compiled to a
self-hosted 8-bit WebAssembly pack with UTF/UCP always enabled.
- PCRE2 native UTF-8 byte ranges are retained and normalized to exact editor
@@ -26,10 +33,22 @@ PCRE2 10.47.
- PCRE2 automatic callouts are copied through a separate ABI operation and
displayed from a dedicated killable trace worker under 50,000-event and
10 MiB serialized-data hard caps.
- Python compilation, matching and replacement use the bundled CPython `re`
module. Native code-point ranges are retained and normalized to exact editor
UTF-16 ranges.
- Java compilation, matching and replacement use TeaVM's
`java.util.regex.Pattern` and `Matcher`. Native ranges are UTF-16 code units.
- TeaVM 0.15.0 replacement supports its actual single-digit `$n` subset.
`$12` is therefore `$1` followed by literal `2`; Java SE `${name}`
replacement is reported as unavailable rather than presented as OpenJDK
behavior.
- Python and Java use deliberately partial application-owned lexical syntax
providers; their actual runtime compiler remains authoritative.
- Parsing and execution run in separate workers.
- Actual execution can be terminated by killing its worker.
- The worker is recreated after timeout, crash or cancellation.
- Capture ranges use the engine's `d` indices result.
- ECMAScript capture ranges use the engine's `d` indices result; the other
flavours return their native range records through bounded bridges.
- Named, numbered, optional, empty and repeated-final captures are represented.
- Match, replacement, typed list/extraction, corpus/apply and unit-test modes
are available.
@@ -73,7 +92,8 @@ as a complete record of every internal engine action.
## Workbench modes
- **Match** — live highlighting, extraction tree and bounded capture table.
- **Replace** — engine-native bounded substitution for ECMAScript and PCRE2;
- **Replace** — engine-native bounded substitution for all four executable
flavours;
output completeness is reported separately from bounded per-match
original/result, changed-range and token-contribution previews that
synchronize the replacement, pattern, subject and output editors.
@@ -121,10 +141,12 @@ Pattern and flags are stored separately; delimiters are presentation only.
ECMAScript preserves native `g` and `y`; PCRE2 offers the application-level
`g` iteration flag plus `i`, `m`, `s`, `x`, `U` and `J`. PCRE2 UTF/UCP mode is
mandatory because browser strings are Unicode and partial-byte ranges cannot be
represented safely in the editor. The optional **Scan all** action adds only an
internal iteration flag to that request and does not change saved flags.
ECMAScript may add internal `d` to obtain exact indices without changing match
semantics.
represented safely in the editor. Python exposes application-level `g` plus
CPython `re` flags `a`, `i`, `m`, `s` and `x`. Java exposes application-level
`g` plus TeaVM `Pattern` flags `i`, `d`, `m`, `s`, `u`, `x` and the documented
`U` compatibility request. The optional **Scan all** action adds only an
internal iteration request and does not change saved flags. ECMAScript may add
internal `d` to obtain exact indices without changing match semantics.
Zero-length global iteration advances using the selected Unicode semantics and
cannot loop forever.
@@ -247,6 +269,10 @@ npm run engines:verify
npm run engines:pcre2:build -- --source-dir /path/to/pcre2 --emcc /path/to/emcc
npm run engines:pcre2:verify
npm run engines:pcre2:install
node scripts/python-engine-pack.mjs node_modules/pyodide .engine-build/python
node scripts/install-python-engine.mjs .engine-build/python
node scripts/java-engine-pack.mjs
node scripts/install-java-engine.mjs
npm run build
npm run toolbox:check
npm run check
@@ -266,18 +292,30 @@ The deterministic release ZIP can be consumed by Toolbox Portal using an exact
artifact URL, manifest ID, version and SHA-256. Portal builds consume the
artifact; they do not build Regex Tools source.
The PCRE2 worker loads self-hosted JavaScript and WebAssembly only. Deployment
must serve `.wasm` as `application/wasm` and allow WebAssembly compilation in
`script-src`; see [`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).
The PCRE2 and Python workers load only self-hosted JavaScript, WebAssembly and
Python-standard-library assets; Java loads its self-hosted ES module.
Deployment must serve `.wasm` as `application/wasm` and allow WebAssembly
compilation in `script-src`; see
[`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).
## Flavour roadmap
## Flavour support and boundaries
PCRE2 10.47 is the second implemented flavour. Its application-owned ABI
provides bounded compilation, matching, native substitution and separate
automatic-callout tracing, copied caller-owned records, deterministic
allocation cleanup and explicit result completeness. The exact
source/toolchain build is reproducible offline and the verified pack is bundled
with its metadata, checksums and licence.
PCRE2's application-owned ABI provides bounded compilation, matching, native
substitution and separate automatic-callout tracing, copied caller-owned
records, deterministic allocation cleanup and explicit result completeness.
The exact source/toolchain build is reproducible offline and the verified pack
is bundled with its metadata, checksums and licence.
The Python slice executes actual CPython 3.14.2 `re` in Pyodide 314.0.3; it does
not translate a Python pattern into ECMAScript. Pyodide startup is a separate
worker-load operation and is excluded from the normal execution deadline.
The Java slice executes TeaVM 0.15.0's own `java.util.regex` class library. It
does not translate a Java pattern into ECMAScript, but it is also not OpenJDK:
in particular, TeaVM 0.15.0 does not implement
`Pattern.UNICODE_CHARACTER_CLASS`. The UI reports the `U` flag as a
compatibility request and preserves TeaVM's predefined-class and word-boundary
behaviour.
Generated-case version 1 is ECMAScript-only. It traverses the complete
application-owned ECMAScript AST under node, depth, attempt, byte, repetition
@@ -286,17 +324,17 @@ browser `RegExp` adapter. PCRE2 generation remains unavailable until its syntax
provider can expose a complete normalized tree.
Pattern formatting is likewise ECMAScript-only. It consumes the complete
regexpp token model and does not reinterpret the partial PCRE2 structure.
PCRE2 formatting remains unavailable until a complete grammar-backed
implementation passes equivalent idempotence, reparse, engine and test gates.
regexpp token model and does not reinterpret the partial PCRE2, Python or Java
structures. Those flavours remain unavailable until complete grammar-backed
implementations pass equivalent idempotence, reparse, engine and test gates.
The first advertised generated-code target is the PCRE2 10.47 8-bit C API.
Its deterministic match and replacement fixtures compile and execute against
the exact official release. See
[`docs/COMPARISON_AND_CODEGEN.md`](docs/COMPARISON_AND_CODEGEN.md).
Python, Go, Rust, .NET, Java and legacy PCRE follow only through their actual
named runtimes. No flavour is emulated through JavaScript and relabelled.
Go, Rust, .NET and legacy PCRE follow only through their actual named runtimes.
No flavour is emulated through JavaScript and relabelled.
## Licensing
@@ -308,7 +346,7 @@ notices; see [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and
No regex101 application source, branding, generated explanation prose,
reference text or visual design is copied. RegexLib and RGXP.RU fixtures are not
redistributed because a suitable fixture licence was not established. No
RegexHub fixture is shipped in v0.2.0.
RegexHub fixture is shipped in v0.3.0.
## Known limitations
@@ -332,6 +370,14 @@ RegexHub fixture is shipped in v0.2.0.
recognizes common capture forms and PCRE2-only constructs for reference, but
the actual PCRE2 compiler is authoritative. Branch-reset capture numbering is
omitted from the explanation rather than guessed.
- Python and Java structural explanations are deliberately partial lexical
models. CPython `re` and TeaVM `Pattern` respectively remain authoritative
for compilation, group count and matching.
- Python reports native Unicode code-point offsets and converts them at valid
boundaries to editor UTF-16 ranges. Java and ECMAScript report native UTF-16
offsets; PCRE2 reports native UTF-8 bytes. Native units remain visible.
- TeaVM `java.util.regex` is not OpenJDK. Its `U` compatibility request does
not add OpenJDK's `UNICODE_CHARACTER_CLASS` semantics.
- Capture-table rendering is paged at 200 rows while aggregate collection is
bounded separately.
- Explanation/extraction trees and editor decorations show bounded 2,000-node
@@ -366,6 +412,9 @@ RegexHub fixture is shipped in v0.2.0.
- Pattern formatting is intentionally narrow and ECMAScript-only. Its bounded
source/candidate and unit-test validation is evidence for the checked
snapshots, not a proof of equivalence over all subjects.
- Cross-flavour comparison, PCRE2 C generation and subject minimization remain
scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting
Python or Java does not silently substitute one of those engines.
Architecture, security, performance, provenance and flavour details live in
[`docs/`](docs/).