feat: add Python and Java regex engines
This commit is contained in:
103
README.md
103
README.md
@@ -8,16 +8,23 @@ Patterns, replacement templates, test text and project files stay in the
|
||||
browser. There is no account, backend, upload, remote execution, telemetry,
|
||||
runtime CDN, remote font, or cloud-save service.
|
||||
|
||||
## Current release — 0.2.0
|
||||
## Current release — 0.3.0
|
||||
|
||||
The v0.2.0 release contains two real production engine slices: ECMAScript and
|
||||
PCRE2 10.47.
|
||||
The v0.3.0 release contains four executable engine slices:
|
||||
|
||||
- the current browser's native ECMAScript `RegExp`;
|
||||
- official PCRE2 10.47 as a self-hosted 8-bit WebAssembly pack;
|
||||
- CPython 3.14.2 `re` in the self-hosted Pyodide 314.0.3 runtime; and
|
||||
- TeaVM 0.15.0's `java.util.regex` class-library implementation as a
|
||||
self-hosted ES2015 module.
|
||||
|
||||
The Java flavour is TeaVM class-library code compiled to JavaScript. It is not
|
||||
OpenJDK and is not advertised as an OpenJDK compatibility oracle.
|
||||
|
||||
- Syntax acceptance: `@eslint-community/regexpp` 4.12.2, ECMAScript 2025
|
||||
grammar; application-owned structural explanations are partial and
|
||||
feature-matrix tested.
|
||||
- Execution engine: the current browser's native ECMAScript `RegExp`.
|
||||
- Native and editor offsets: UTF-16 code units.
|
||||
- ECMAScript native and editor offsets are UTF-16 code units.
|
||||
- PCRE2 execution and substitution: pinned official PCRE2 10.47 compiled to a
|
||||
self-hosted 8-bit WebAssembly pack with UTF/UCP always enabled.
|
||||
- PCRE2 native UTF-8 byte ranges are retained and normalized to exact editor
|
||||
@@ -26,10 +33,22 @@ PCRE2 10.47.
|
||||
- PCRE2 automatic callouts are copied through a separate ABI operation and
|
||||
displayed from a dedicated killable trace worker under 50,000-event and
|
||||
10 MiB serialized-data hard caps.
|
||||
- Python compilation, matching and replacement use the bundled CPython `re`
|
||||
module. Native code-point ranges are retained and normalized to exact editor
|
||||
UTF-16 ranges.
|
||||
- Java compilation, matching and replacement use TeaVM's
|
||||
`java.util.regex.Pattern` and `Matcher`. Native ranges are UTF-16 code units.
|
||||
- TeaVM 0.15.0 replacement supports its actual single-digit `$n` subset.
|
||||
`$12` is therefore `$1` followed by literal `2`; Java SE `${name}`
|
||||
replacement is reported as unavailable rather than presented as OpenJDK
|
||||
behavior.
|
||||
- Python and Java use deliberately partial application-owned lexical syntax
|
||||
providers; their actual runtime compiler remains authoritative.
|
||||
- Parsing and execution run in separate workers.
|
||||
- Actual execution can be terminated by killing its worker.
|
||||
- The worker is recreated after timeout, crash or cancellation.
|
||||
- Capture ranges use the engine's `d` indices result.
|
||||
- ECMAScript capture ranges use the engine's `d` indices result; the other
|
||||
flavours return their native range records through bounded bridges.
|
||||
- Named, numbered, optional, empty and repeated-final captures are represented.
|
||||
- Match, replacement, typed list/extraction, corpus/apply and unit-test modes
|
||||
are available.
|
||||
@@ -73,7 +92,8 @@ as a complete record of every internal engine action.
|
||||
## Workbench modes
|
||||
|
||||
- **Match** — live highlighting, extraction tree and bounded capture table.
|
||||
- **Replace** — engine-native bounded substitution for ECMAScript and PCRE2;
|
||||
- **Replace** — engine-native bounded substitution for all four executable
|
||||
flavours;
|
||||
output completeness is reported separately from bounded per-match
|
||||
original/result, changed-range and token-contribution previews that
|
||||
synchronize the replacement, pattern, subject and output editors.
|
||||
@@ -121,10 +141,12 @@ Pattern and flags are stored separately; delimiters are presentation only.
|
||||
ECMAScript preserves native `g` and `y`; PCRE2 offers the application-level
|
||||
`g` iteration flag plus `i`, `m`, `s`, `x`, `U` and `J`. PCRE2 UTF/UCP mode is
|
||||
mandatory because browser strings are Unicode and partial-byte ranges cannot be
|
||||
represented safely in the editor. The optional **Scan all** action adds only an
|
||||
internal iteration flag to that request and does not change saved flags.
|
||||
ECMAScript may add internal `d` to obtain exact indices without changing match
|
||||
semantics.
|
||||
represented safely in the editor. Python exposes application-level `g` plus
|
||||
CPython `re` flags `a`, `i`, `m`, `s` and `x`. Java exposes application-level
|
||||
`g` plus TeaVM `Pattern` flags `i`, `d`, `m`, `s`, `u`, `x` and the documented
|
||||
`U` compatibility request. The optional **Scan all** action adds only an
|
||||
internal iteration request and does not change saved flags. ECMAScript may add
|
||||
internal `d` to obtain exact indices without changing match semantics.
|
||||
|
||||
Zero-length global iteration advances using the selected Unicode semantics and
|
||||
cannot loop forever.
|
||||
@@ -247,6 +269,10 @@ npm run engines:verify
|
||||
npm run engines:pcre2:build -- --source-dir /path/to/pcre2 --emcc /path/to/emcc
|
||||
npm run engines:pcre2:verify
|
||||
npm run engines:pcre2:install
|
||||
node scripts/python-engine-pack.mjs node_modules/pyodide .engine-build/python
|
||||
node scripts/install-python-engine.mjs .engine-build/python
|
||||
node scripts/java-engine-pack.mjs
|
||||
node scripts/install-java-engine.mjs
|
||||
npm run build
|
||||
npm run toolbox:check
|
||||
npm run check
|
||||
@@ -266,18 +292,30 @@ The deterministic release ZIP can be consumed by Toolbox Portal using an exact
|
||||
artifact URL, manifest ID, version and SHA-256. Portal builds consume the
|
||||
artifact; they do not build Regex Tools source.
|
||||
|
||||
The PCRE2 worker loads self-hosted JavaScript and WebAssembly only. Deployment
|
||||
must serve `.wasm` as `application/wasm` and allow WebAssembly compilation in
|
||||
`script-src`; see [`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).
|
||||
The PCRE2 and Python workers load only self-hosted JavaScript, WebAssembly and
|
||||
Python-standard-library assets; Java loads its self-hosted ES module.
|
||||
Deployment must serve `.wasm` as `application/wasm` and allow WebAssembly
|
||||
compilation in `script-src`; see
|
||||
[`docs/PORTAL_REQUIREMENTS.md`](docs/PORTAL_REQUIREMENTS.md).
|
||||
|
||||
## Flavour roadmap
|
||||
## Flavour support and boundaries
|
||||
|
||||
PCRE2 10.47 is the second implemented flavour. Its application-owned ABI
|
||||
provides bounded compilation, matching, native substitution and separate
|
||||
automatic-callout tracing, copied caller-owned records, deterministic
|
||||
allocation cleanup and explicit result completeness. The exact
|
||||
source/toolchain build is reproducible offline and the verified pack is bundled
|
||||
with its metadata, checksums and licence.
|
||||
PCRE2's application-owned ABI provides bounded compilation, matching, native
|
||||
substitution and separate automatic-callout tracing, copied caller-owned
|
||||
records, deterministic allocation cleanup and explicit result completeness.
|
||||
The exact source/toolchain build is reproducible offline and the verified pack
|
||||
is bundled with its metadata, checksums and licence.
|
||||
|
||||
The Python slice executes actual CPython 3.14.2 `re` in Pyodide 314.0.3; it does
|
||||
not translate a Python pattern into ECMAScript. Pyodide startup is a separate
|
||||
worker-load operation and is excluded from the normal execution deadline.
|
||||
|
||||
The Java slice executes TeaVM 0.15.0's own `java.util.regex` class library. It
|
||||
does not translate a Java pattern into ECMAScript, but it is also not OpenJDK:
|
||||
in particular, TeaVM 0.15.0 does not implement
|
||||
`Pattern.UNICODE_CHARACTER_CLASS`. The UI reports the `U` flag as a
|
||||
compatibility request and preserves TeaVM's predefined-class and word-boundary
|
||||
behaviour.
|
||||
|
||||
Generated-case version 1 is ECMAScript-only. It traverses the complete
|
||||
application-owned ECMAScript AST under node, depth, attempt, byte, repetition
|
||||
@@ -286,17 +324,17 @@ browser `RegExp` adapter. PCRE2 generation remains unavailable until its syntax
|
||||
provider can expose a complete normalized tree.
|
||||
|
||||
Pattern formatting is likewise ECMAScript-only. It consumes the complete
|
||||
regexpp token model and does not reinterpret the partial PCRE2 structure.
|
||||
PCRE2 formatting remains unavailable until a complete grammar-backed
|
||||
implementation passes equivalent idempotence, reparse, engine and test gates.
|
||||
regexpp token model and does not reinterpret the partial PCRE2, Python or Java
|
||||
structures. Those flavours remain unavailable until complete grammar-backed
|
||||
implementations pass equivalent idempotence, reparse, engine and test gates.
|
||||
|
||||
The first advertised generated-code target is the PCRE2 10.47 8-bit C API.
|
||||
Its deterministic match and replacement fixtures compile and execute against
|
||||
the exact official release. See
|
||||
[`docs/COMPARISON_AND_CODEGEN.md`](docs/COMPARISON_AND_CODEGEN.md).
|
||||
|
||||
Python, Go, Rust, .NET, Java and legacy PCRE follow only through their actual
|
||||
named runtimes. No flavour is emulated through JavaScript and relabelled.
|
||||
Go, Rust, .NET and legacy PCRE follow only through their actual named runtimes.
|
||||
No flavour is emulated through JavaScript and relabelled.
|
||||
|
||||
## Licensing
|
||||
|
||||
@@ -308,7 +346,7 @@ notices; see [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and
|
||||
No regex101 application source, branding, generated explanation prose,
|
||||
reference text or visual design is copied. RegexLib and RGXP.RU fixtures are not
|
||||
redistributed because a suitable fixture licence was not established. No
|
||||
RegexHub fixture is shipped in v0.2.0.
|
||||
RegexHub fixture is shipped in v0.3.0.
|
||||
|
||||
## Known limitations
|
||||
|
||||
@@ -332,6 +370,14 @@ RegexHub fixture is shipped in v0.2.0.
|
||||
recognizes common capture forms and PCRE2-only constructs for reference, but
|
||||
the actual PCRE2 compiler is authoritative. Branch-reset capture numbering is
|
||||
omitted from the explanation rather than guessed.
|
||||
- Python and Java structural explanations are deliberately partial lexical
|
||||
models. CPython `re` and TeaVM `Pattern` respectively remain authoritative
|
||||
for compilation, group count and matching.
|
||||
- Python reports native Unicode code-point offsets and converts them at valid
|
||||
boundaries to editor UTF-16 ranges. Java and ECMAScript report native UTF-16
|
||||
offsets; PCRE2 reports native UTF-8 bytes. Native units remain visible.
|
||||
- TeaVM `java.util.regex` is not OpenJDK. Its `U` compatibility request does
|
||||
not add OpenJDK's `UNICODE_CHARACTER_CLASS` semantics.
|
||||
- Capture-table rendering is paged at 200 rows while aggregate collection is
|
||||
bounded separately.
|
||||
- Explanation/extraction trees and editor decorations show bounded 2,000-node
|
||||
@@ -366,6 +412,9 @@ RegexHub fixture is shipped in v0.2.0.
|
||||
- Pattern formatting is intentionally narrow and ECMAScript-only. Its bounded
|
||||
source/candidate and unit-test validation is evidence for the checked
|
||||
snapshots, not a proof of equivalence over all subjects.
|
||||
- Cross-flavour comparison, PCRE2 C generation and subject minimization remain
|
||||
scoped to their explicitly supported ECMAScript/PCRE2 paths; selecting
|
||||
Python or Java does not silently substitute one of those engines.
|
||||
|
||||
Architecture, security, performance, provenance and flavour details live in
|
||||
[`docs/`](docs/).
|
||||
|
||||
Reference in New Issue
Block a user