lotobo/regex-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

feat: add Python and Java regex engines

Browse Source
This commit is contained in:
Albrecht Degering
2026-07-27 02:02:21 +02:00
parent 4951c966d4
commit 7079cde15f
95 changed files with 8866 additions and 251 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/ANALYSIS.md
View File

@@ -3,7 +3,7 @@
Regex Tools provides one deliberately scoped analysis implementation:
ECMAScript 2025 patterns parsed into the application-owned normalized AST and
executed by the browser's native `RegExp` runtime. The analyser does not apply
ECMAScript findings to PCRE2 or any other flavour.
ECMAScript findings to PCRE2, Python, Java or any other flavour.
Analysis is advisory. The UI uses “potential risk”, “possibly” and “observed
under selected limits”. It never turns an empty finding list into “safe”,

49
docs/ARCHITECTURE.md
View File

@@ -7,11 +7,15 @@ boundaries.
React workbench
├─ Pattern SyntaxSupervisor → syntax.worker
│ ├─ Regexpp ECMAScript provider → normalized AST
│ └─ PCRE2 lexical provider → partial normalized structure
│ ├─ PCRE2 lexical provider → partial normalized structure
│ ├─ Python lexical provider → partial normalized structure
│ └─ Java lexical provider → partial normalized structure
├─ Replacement SyntaxSupervisor → syntax.worker → typed replacement tokens
├─ EngineSupervisor
│ ├─ ecmascript.worker → native RegExp → match DTOs
│ └─ pcre2.worker → PCRE2 10.47 WASM ABI → match DTOs
│ ├─ pcre2.worker → PCRE2 10.47 WASM ABI → match DTOs
│ ├─ python.worker → Pyodide 314.0.3 → CPython 3.14.2 re → match DTOs
│ └─ java.worker → TeaVM 0.15.0 java.util.regex module → match DTOs
├─ Pcre2TraceSupervisor → pcre2-trace.worker
│ └─ separate ABI-v3 PCRE2_AUTO_CALLOUT operation → bounded trace DTOs
├─ AnalysisPanel
@@ -42,7 +46,18 @@ provenance. React never imports or traverses regexpp's cyclic AST.
Every engine request carries a protocol version, monotonically increasing
request ID and worker generation. A supervisor allows one active request,
rejects stale responses, terminates on timeout/crash/cancel, and lazily creates
a new generation. Timeout is a distinct error state.
a new generation. Timeout is a distinct error state. A separate idempotent
load operation verifies a worker's runtime identity before execution; its
flavour-specific startup deadline is not charged to the 250 ms live or selected
manual execution deadline. A killed generation must load and verify again.
The Python worker imports only the self-hosted Pyodide loader and pack, then
evaluates the application-owned bounded bridge in CPython. The bridge returns
code-point ranges, which are retained as native values and converted only at
valid boundaries to editor UTF-16. The Java worker imports a verified TeaVM
ES2015 module. That module executes TeaVM's own `java.util.regex.Pattern` and
`Matcher` class-library implementation and returns native UTF-16 ranges; it is
not OpenJDK.
PCRE2 tracing is not an `EngineSupervisor` match or replacement operation. Its
own worker compiles with `PCRE2_AUTO_CALLOUT`, copies only complete fixed
@@ -70,13 +85,14 @@ PCRE2 API cannot express directly. Its advertised C17 target is protected by
deterministic golden identities and an exact PCRE2 10.47 compile/execute gate.
Subject minimization is a main-thread bounded reducer whose expensive predicate
checks stay in independently terminable engine workers. Fixed pattern syntax
is parsed once for capture metadata. The baseline and every accepted candidate
retain the exact target and engine identity. Timeout, crash, cancellation,
worker failure and incomplete output are separate outcomes. Deterministic
ddmin deletion is followed by a fixed-point local sweep over Unicode-scalar
deletion and lower-rank canonical replacement; only completion of that sweep
permits a transform-local minimality claim.
checks stay in independently terminable ECMAScript or PCRE2 workers. Fixed
pattern syntax is parsed once for capture metadata. Python and Java are blocked
at this feature boundary rather than routed to a different engine. The baseline
and every accepted candidate retain the exact target and engine identity.
Timeout, crash, cancellation, worker failure and incomplete output are separate
outcomes. Deterministic ddmin deletion is followed by a fixed-point local sweep
over Unicode-scalar deletion and lower-rank canonical replacement; only
completion of that sweep permits a transform-local minimality claim.
Generated-case synthesis and verification are separate trust boundaries. The
dedicated generation worker receives only a validated, complete ECMAScript
@@ -123,12 +139,13 @@ separator during apply. This makes anchor and cross-line behavior deliberately
different rather than silently rewriting the pattern. Match DTOs are reduced to
bounded per-group participation counts and samples before the next document.
Native ranges are retained in the engine's declared unit. PCRE2 always uses
UTF/UCP and exposes UTF-8 byte ranges; its ABI copies scalar records and names
out of WebAssembly before returning and retains no code pointer. Browser editor ranges
are half-open UTF-16 code-unit ranges. Reusable converters cover UTF-8 byte and
Unicode code-point offsets, including invalid-boundary rejection and
lone-surrogate detection.
Native ranges are retained in the engine's declared unit. ECMAScript and Java
use UTF-16 code units. PCRE2 always uses UTF/UCP and exposes UTF-8 byte ranges;
its ABI copies scalar records and names out of WebAssembly before returning and
retains no code pointer. CPython `re` exposes Unicode code-point indices.
Browser editor ranges are half-open UTF-16 code-unit ranges. Reusable
converters cover UTF-8 byte and Unicode code-point offsets, including
invalid-boundary rejection and lone-surrogate detection.
The release boundary is `dist/` plus checked-in legal/source documents. The
Portal consumes the resulting immutable ZIP and never imports React source.

10
docs/COMPARISON_AND_CODEGEN.md
View File

@@ -6,6 +6,10 @@ The comparison panel sends two explicit requests: one to the native browser
ECMAScript worker and one to the bundled PCRE2 10.47 worker. It does not
translate patterns, flags, options or replacement templates.
This is an explicit two-engine feature, not a generic active-flavour
comparison. Selecting Python or Java for the main workbench does not insert it
into either side or route it through ECMAScript/PCRE2.
Two pattern models are available:
- **Shared pattern** sends the same string unchanged to both engines.
@@ -52,8 +56,10 @@ result. The panel renders at most 250 differences and 100 alignments at once.
## PCRE2 C17 generator
The first reviewed code-generation target is the PCRE2 10.47 8-bit C API.
Other languages remain unavailable until their generators have equivalent
escaping, semantic and toolchain gates.
Python, Java and other targets remain unavailable until their generators have
equivalent escaping, semantic and named-runtime/toolchain gates. The executable
Python and Java browser flavours do not by themselves imply a code-generation
target.
The generator consumes a validated PCRE2 snapshot and:

99
docs/ENGINE_BUILDS.md
View File

@@ -1,9 +1,9 @@
# Engine builds
The production application uses the browser's native `RegExp` for ECMAScript
and bundles a verified PCRE2 10.47 WebAssembly pack. Normal application builds
never download or compile an engine: the reviewed files under
`public/engines/pcre2/` are copied into `dist/` and verified there.
and bundles verified PCRE2, Python and Java runtime packs. Normal application
builds never download or compile an engine: reviewed files under
`public/engines/` are copied into `dist/` and verified there.
## Rebuilding PCRE2
@@ -52,7 +52,77 @@ The immutable signed-tag object is pinned. OpenPGP trust validation remains a
release-operator step because the offline builder does not install or trust a
key.
## Runtime boundary
## Installing the Python pack
The Python pack is derived from the exact `pyodide@314.0.3` npm package locked
with integrity:
```text
sha512-sK40My6m8tmBUYtYH9au9rXUeh9x0wfahtHdOlGmJxZDsKBGKtP6KznyFB2+u/klbQTdDionR0uaVd176zVQzQ==
```
That Pyodide release is tag/commit
`ac57031be7564f864d061cb37c5c152e59f83ad4`. Direct executable identity
verification reports CPython 3.14.2; the shipped `pyodide-lock.json`
independently reports `info.python` as 3.14.0, and the verifier records rather
than conflates those two upstream identities. The pack copies only the reviewed
loader, WebAssembly runtime, Python standard library and lock metadata required
for a local CPython `re` worker. It adds deterministic engine metadata, a
closed-file-set checksum manifest and exact Pyodide/CPython licence material.
The install operation reads from the already installed pinned npm package. It
does not contact a CDN or package index. It verifies source package identity,
file hashes, runtime metadata and licence inputs before atomically replacing
only `public/engines/python/`. The distribution verifier checks the same closed
set under `dist/`, validates WebAssembly magic and confirms that all runtime
references remain relative and self-hosted.
```sh
node scripts/python-engine-pack.mjs \
node_modules/pyodide \
.engine-build/python
node scripts/install-python-engine.mjs .engine-build/python
```
The installed closed set is:
- `pyodide.mjs`, `pyodide.asm.mjs` and `pyodide.asm.wasm`;
- `python_stdlib.zip` and `pyodide-lock.json`;
- `engine-metadata.json` and `SHA256SUMS`; and
- `LICENSE.pyodide.txt` and `LICENSE.cpython.txt`.
## Building the Java pack
The Java bridge under `engines/java/` wraps TeaVM 0.15.0
`java.util.regex.Pattern` and `Matcher`. TeaVM tag/commit
`ee91b03e616c4b45401cd11fb0cd7eb0daf6649b` is pinned through the Maven build.
The build produces a minified ES2015 module without source maps, source files,
remote imports or an OpenJDK runtime.
The pack gate verifies the Maven coordinates, bridge source identity, TeaVM
engine identity and self-test, generated module, deterministic metadata,
checksums, Apache-2.0 licence and TeaVM NOTICE. Only the verified pack is
installed under `public/engines/java/`.
```sh
node scripts/java-engine-pack.mjs
node scripts/install-java-engine.mjs
```
The pack command performs two clean offline Maven builds and requires identical
output. The installed closed set is `java-regex.mjs`,
`engine-metadata.json`, `SHA256SUMS`, `LICENSE.txt` and `NOTICE.txt`.
TeaVM supplies its own Apache Harmony-derived class-library implementation.
The resulting engine is therefore identified as
`TeaVM java.util.regex 0.15.0`, not OpenJDK. TeaVM 0.15.0 does not implement
OpenJDK's `Pattern.UNICODE_CHARACTER_CLASS`; the bridge treats the application
`U` flag as a compatibility request, implies Unicode case folding and reports
that predefined-character classes and word boundaries retain TeaVM behaviour.
## Runtime boundaries
### PCRE2
ABI version 3 has no retained native handles. Each call compiles, obtains
capture names, matches/substitutes or traces and releases its code, match data
@@ -72,3 +142,24 @@ Normal execution lives only in `pcre2.worker`. Instrumented
never call it. Each supervisor terminates its worker on timeout, crash,
cancellation or supersession and lazily creates a clean generation for the
next request.
### Python
The Python worker loads the self-hosted Pyodide pack and verifies Pyodide,
CPython and `re` identities before accepting work. The application bridge is
evaluated inside that interpreter and exposes bounded compile, match and native
replacement operations. It reports CPython's Unicode code-point positions;
the TypeScript adapter retains them and converts only valid boundaries to
half-open editor UTF-16 ranges. Runtime load has a separate startup deadline.
Execution timeout, cancellation or crash terminates the worker and its complete
Python/WebAssembly heap.
### Java
The Java worker imports the self-hosted TeaVM ES module and checks the bridge
ABI, declared engine identity and compiled self-test before accepting work.
Every request compiles a `Pattern`, obtains bounded `Matcher` results or native
replacement output, and returns native half-open UTF-16 ranges. No Java virtual
machine, OpenJDK classes, remote service or ECMAScript regex fallback is
involved. Runtime load has a separate startup deadline; termination discards
the complete worker module state.

4
docs/EXECUTION_TRACES.md
View File

@@ -5,6 +5,10 @@ results, but no V8, SpiderMonkey or JavaScriptCore execution trace. Regex Tools
does not relabel structural explanations, static findings or timing
observations as an ECMAScript engine trace.
The bundled CPython `re` and TeaVM `java.util.regex` bridges likewise expose
compilation, bounded match/capture and replacement results, not internal engine
steps. Regex Tools does not label their partial lexical syntax trees as traces.
PCRE2 10.47 has a separate actual automatic-callout vertical. ABI version 3
compiles the requested pattern with `PCRE2_AUTO_CALLOUT`, registers an
application callback, and copies only:

8
docs/EXPLANATION_MODEL.md
View File

@@ -20,8 +20,16 @@ common capture forms. It does not infer branch-reset numbering or claim full
nesting; authoritative compilation, capture counts, names and result ranges
come from PCRE2 itself.
The Python and Java providers likewise emit deliberately partial lexical trees.
They recognize their documented capture and replacement forms for navigation
and display, but do not claim full grammar acceptance. CPython `re` and TeaVM
`Pattern` remain authoritative for compilation. Java capture names displayed
with native numbered spans come from the bounded lexical metadata; they are not
invented as a TeaVM-reported name table.
The explanation tree is structural. It is never called an actual execution
trace. Selecting a node selects and scrolls its exact editor range; selecting
pattern text chooses the smallest enclosing node. PCRE2’s separate trace viewer
contains actual bounded automatic callouts; its movement labels remain a
distinct derived layer and are never copied into the structural explanation.
Python and Java expose no actual execution trace.

12
docs/EXTRACTION_MODEL.md
View File

@@ -18,10 +18,14 @@ budget. Consumers must inspect the preview status rather than treating a
preview as complete. In particular, list export is disabled if its template
would consume an incomplete value.
Repeated ECMAScript captures expose only the final retained capture. The UI
labels this limitation and does not invent history. Later .NET support may add
actual engine capture-history nodes without degrading them to this model.
Repeated ECMAScript, PCRE2, Python and Java captures expose only the final
retained capture. The UI labels this limitation and does not invent history.
Later .NET support may add actual engine capture-history nodes without
degrading them to this model.
The syntactic hierarchy remains useful even if actual spans overlap or fall
outside a parent's geometry; native and normalized ranges are preserved rather
than “fixed”.
than “fixed”. ECMAScript and Java native offsets are UTF-16 code units, PCRE2
native offsets are UTF-8 bytes and Python native offsets are Unicode code
points. All editor ranges are normalized to UTF-16 without discarding the
native unit.

56
docs/FLAVOUR_SUPPORT.md
View File

@@ -1,15 +1,15 @@
# Flavour support
| Flavour | Syntax | Execution | Replacement | Captures | Trace | Analysis | Generated cases | Native offsets |
| ------------ | ------------------------------------------------------------ | ------------------------------------------------------ | ---------------------------------------- | -------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------ | ------------------------------------------------------- | ------------------- |
| ECMAScript | regexpp 4.12.2; explanations partial/feature-matrix tested | Current browser `RegExp`; conformance tested | Bounded ECMAScript `GetSubstitution` | Named/numbered; final repeated capture; no history | Unavailable | Experimental static heuristics + bounded native probes | Bounded normalized-AST sampling; actual-engine verified | UTF-16 |
| PCRE2 10.47 | Application lexical provider partial; compiler authoritative | Pinned 8-bit WASM; bounded/killable/conformance tested | Native bounded `pcre2_substitute()` loop | Native named/numbered records; no capture history | Bounded reported automatic callouts; derived movement labels | Unavailable; ECMAScript heuristics are never applied | Unavailable; structural syntax provider is partial | UTF-8 bytes |
| PCRE | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| Python `re` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned code points |
| Go `regexp` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned UTF-8 bytes |
| Rust `regex` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned UTF-8 bytes |
| .NET | Unavailable | Unavailable | Unavailable | Planned capture history | Unavailable | Unavailable | Unavailable | Planned UTF-16 |
| Java | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned UTF-16 |
| Flavour | Syntax | Execution | Replacement | Captures | Trace | Analysis | Generated cases | Native offsets |
| ---------------------- | ------------------------------------------------------------ | --------------------------------------------------------- | ---------------------------------------- | -------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------ | ------------------------------------------------------- | -------------- |
| ECMAScript | regexpp 4.12.2; explanations partial/feature-matrix tested | Current browser `RegExp`; conformance tested | Bounded ECMAScript `GetSubstitution` | Named/numbered; final repeated capture; no history | Unavailable | Experimental static heuristics + bounded native probes | Bounded normalized-AST sampling; actual-engine verified | UTF-16 |
| PCRE2 10.47 | Application lexical provider partial; compiler authoritative | Pinned 8-bit WASM; bounded/killable/conformance tested | Native bounded `pcre2_substitute()` loop | Native named/numbered records; no capture history | Bounded reported automatic callouts; derived movement labels | Unavailable; ECMAScript heuristics are never applied | Unavailable; structural syntax provider is partial | UTF-8 bytes |
| Python `re` | Application lexical provider partial; compiler authoritative | CPython 3.14.2 in self-hosted Pyodide 314.0.3 | Native bounded `re` expansion | Native named/numbered spans; no capture history | Unavailable | Unavailable; ECMAScript heuristics are never applied | Unavailable; structural syntax provider is partial | Code points |
| Java `java.util.regex` | Application lexical provider partial; compiler authoritative | TeaVM 0.15.0 class library; bounded/killable; not OpenJDK | TeaVM single-digit `$n`; no `${name}` | Native numbered ranges; lexical names; no history | Unavailable | Unavailable; ECMAScript heuristics are never applied | Unavailable; structural syntax provider is partial | UTF-16 |
| PCRE | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| Go `regexp` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned bytes |
| Rust `regex` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned bytes |
| .NET | Unavailable | Unavailable | Unavailable | Planned capture history | Unavailable | Unavailable | Unavailable | Planned UTF-16 |
ECMAScript tests cover its complete exposed flag matrix, internal indices,
zero-length iteration, captures, lookbehind, backreferences, Unicode properties
@@ -22,6 +22,19 @@ and result/output truncation. Trace tests cover reported automatic callouts,
bounded mark copying, native stop at the complete-event cap, compile ranges,
worker loading and browser rendering.
Python tests cover application-level `g`, CPython flags `a`, `i`, `m`, `s` and
`x`, Python-only named-group/replacement syntax, native `re` substitution,
astral code-point/UTF-16 normalization, zero-length iteration, compile errors,
result limits, runtime identity and worker recovery.
Java tests cover application-level `g`, TeaVM `Pattern` flags, Java named-group
syntax, TeaVM's native single-digit `$n` `Matcher` replacement, explicit
rejection of unsupported `${name}`, astral UTF-16 ranges, zero-length
iteration, compile errors, result limits, module identity and worker recovery.
The runtime and UI explicitly report that TeaVM 0.15.0 is not OpenJDK and does
not implement OpenJDK `Pattern.UNICODE_CHARACTER_CLASS`; `U` is a compatibility
request rather than an exact implementation of that flag.
ECMAScript static findings and generated-input timing observations are
advisory. They describe only the selected ECMAScript 2025 pattern, browser
runtime and bounded inputs; they are not proofs of safety or general
@@ -30,13 +43,14 @@ complexity.
ECMAScript generated-case version 1 uses only a complete accepted normalized
AST, an explicit deterministic seed and bounded sampling strategies. Every
retained positive or negative label is confirmed by the actual selected
browser engine. PCRE2 generation is unavailable because its current lexical
provider is intentionally partial; ECMAScript behavior is never relabelled as
PCRE2.
browser engine. PCRE2, Python and Java generation are unavailable because
their current lexical providers are intentionally partial; ECMAScript behavior
is never relabelled as another flavour.
The PCRE2 explanation provider deliberately does not invent full structural
coverage. In particular, branch-reset numbering is omitted from the syntax tree
while actual capture numbers and names still come from PCRE2 results.
The PCRE2, Python and Java explanation providers deliberately do not invent
full structural coverage. PCRE2 branch-reset numbering is omitted from the
syntax tree, while actual capture numbers, names and ranges still come from the
selected runtime result.
ECMAScript and PCRE2 support exact-request comparison for syntax acceptance,
engine compilation, matches, captures and engine-native replacement. Results
@@ -47,8 +61,14 @@ equivalent results.
Pattern formatting is available only for an exact accepted ECMAScript regexpp
snapshot. It canonicalizes literal slashes and raw control/line-separator
representations, then requires paired actual-engine and applicable exact-test
validation before apply. PCRE2 formatting is unavailable because the current
provider is deliberately partial; its syntax is never treated as ECMAScript.
validation before apply. PCRE2, Python and Java formatting is unavailable
because those providers are deliberately partial; their syntax is never
treated as ECMAScript.
Subject minimization accepts only the verified ECMAScript and PCRE2 paths.
Python and Java selections are disabled rather than being minimized through a
different flavour. Semantic comparison is likewise an explicit
ECMAScript/PCRE2 operation; it is not a generic four-way equivalence checker.
PCRE2 is the only current code-generation target. The reviewed C17 generator is
pinned to the PCRE2 10.47 8-bit API; other language generators remain

5
docs/FORMATTING.md
View File

@@ -23,8 +23,9 @@ pretty-printer. The transform is idempotent.
Formatting requires the exact current, accepted, non-recovered syntax snapshot
from the bundled `@eslint-community/regexpp` provider. Transformations are
derived from the provider's literal-token ranges rather than from a heuristic
text scan. A stale parse, a malformed pattern, a partial provider, PCRE2 or any
other flavour is reported as unavailable and is never silently rewritten.
text scan. A stale parse, a malformed pattern, a partial provider, PCRE2,
Python, Java or any other flavour is reported as unavailable and is never
silently rewritten.
## Mandatory validation before apply

13
docs/GENERATED_CASES.md
View File

@@ -25,10 +25,11 @@ The coverage report says `covered`, `partial`, `unsupported` or
sampling strategy was requested. It does not mean every string in the regular
language was enumerated.
PCRE2 generation is unavailable in version 1. Its actual engine is complete for
the advertised execution operations, but its current structural syntax
provider is intentionally partial. Regex Tools does not feed PCRE2 syntax to
the ECMAScript generator or relabel ECMAScript behavior.
PCRE2, Python and Java generation is unavailable in version 1. Their actual
engines are complete for the advertised execution operations, but their
current structural syntax providers are intentionally partial. Regex Tools
does not feed their syntax to the ECMAScript generator or relabel ECMAScript
behaviour.
## Unsupported and partial constructs
@@ -75,7 +76,9 @@ removes the generation provenance.
Candidate synthesis runs in a dedicated killable worker. The verifier then
sends each exact candidate, pattern, flavour version, flags, engine options and
scan mode through `EngineSupervisor`, which selects the registered real engine
worker.
worker. Version 1's eligibility gate accepts only the exact current ECMAScript
snapshot; the generic supervisor boundary does not make partial-provider
flavours eligible.
- An intended positive is labelled `should-match` only after the engine returns
at least one match.

4
docs/MINIMIZATION.md
View File

@@ -10,6 +10,10 @@ selected real ECMAScript or PCRE2 worker. It can preserve:
- the same one-sided comparison timeout while the other engine completes with
authoritative, untruncated output.
Python and Java are not supported minimization targets in v0.3.0. The workspace
disables reduction for those active flavours; it does not substitute
ECMAScript, PCRE2 or a lexical provider as an execution oracle.
The fixed pattern, ordered flags, options, replacement and engine versions form
the target identity. Syntax is parsed once to obtain capture metadata. Every
accepted subject candidate is then verified through the selected execution

10
docs/PARSER_PROFILES.md
View File

@@ -9,11 +9,17 @@ community
It uses only reviewed open-source dependencies and builds without private
registry credentials. ECMAScript uses regexpp 4.12.2. PCRE2 uses an
application-owned lexical provider for partial explanations and the actual
bundled PCRE2 compiler for authoritative acceptance.
bundled PCRE2 compiler for authoritative acceptance. Python and Java use
application-owned lexical providers for partial explanations and
flavour-native replacement tokens; bundled CPython `re` and TeaVM
`java.util.regex` respectively remain authoritative for compilation.
The Java provider follows the bundled TeaVM 0.15.0 replacement subset:
single-digit `$n` is executable, `$12` is `$1` followed by literal `2`, and
Java SE `${name}` references are surfaced as a compatibility limitation.
The product deliberately does not implement or reference an `@r101/parser`
profile. No commercial package alias, import, tarball, credential, licence key
or private CI path exists. Additional flavour syntax will be implemented
or private CI path exists. Additional flavour syntax is implemented
incrementally as open-source providers behind the same application-owned
interface. Partial coverage is exposed as metadata and never presented as a
complete parser.

26
docs/PERFORMANCE.md
View File

@@ -10,12 +10,22 @@ browser, engine version, pattern and subject. The UI therefore reports observed
elapsed time and exact runtime identity but makes no universal throughput or
safety claim.
PCRE2 cold execution includes loading and instantiating the self-hosted
WebAssembly module inside its worker; warm requests reuse that instance.
PCRE2 cold worker startup includes loading and instantiating the self-hosted
WebAssembly module; warm requests reuse that instance.
Every request still compiles and frees its pattern so no unbounded compiled-code
cache can accumulate. Native match-step, depth and heap limits complement the
supervisor wall clock, and cancellation discards the complete worker heap.
Python has a materially larger cold start because its worker loads the
self-hosted Pyodide JavaScript/WebAssembly runtime and Python standard library
before verifying CPython and `re`. Java imports and self-tests its bundled
TeaVM ES module. Engine loading is an explicit operation with a
flavour-specific startup deadline; it is not charged to the 250 ms live or
selected manual execution timeout. Warm requests reuse the verified worker.
Killing a worker for timeout, crash or cancellation discards its complete
runtime, so the next request pays the cold-load cost and verifies identity
again.
Automatic-callout tracing is deliberately excluded from every benchmark. It
compiles an independently instrumented PCRE2 pattern in a separate worker and
can materially change runtime work. The trace path retains at most 50,000
@@ -44,12 +54,12 @@ pages of 200. Individual result-value previews are capped at 64 Ki UTF-16 units
and share an 8 Mi-unit aggregate engine-result budget. The extraction tree
shows 100 UTF-16 units per value and capture-table cells show 512, with explicit
range lengths and separate engine-prefix labels; copy and eligible JSON export
retain the bounded engine value. List export is disabled when a source value is incomplete, so a bounded
preview cannot be mistaken for the full value. List generation additionally
stops at 250,000 template-token evaluations, a 64 Ki UTF-16 row preview, or an
8 Mi-unit aggregate materialization bound. The UI shows 500 list rows while an
eligible export still contains every generated row; any incomplete result
disables export.
retain the bounded engine value. List export is disabled when a source value is
incomplete, so a bounded preview cannot be mistaken for the full value. List
generation additionally stops at 250,000 template-token evaluations, a 64 Ki
UTF-16 row preview, or an 8 Mi-unit aggregate materialization bound. The UI
shows 500 list rows while an eligible export still contains every generated
row; any incomplete result disables export.
Replacement output has a separate 64 MiB bound. Match/capture-limit
incompleteness and output-byte truncation are tracked separately; either makes
the aggregate replacement result incomplete.

20
docs/PORTAL_REQUIREMENTS.md
View File

@@ -9,17 +9,23 @@ Required delivery behavior:
revalidation/no-cache resources.
- Hashed Vite assets may use immutable long-term caching.
- CSP must allow `worker-src 'self' blob:`.
- `engines/pcre2/pcre2.wasm` must use `application/wasm`.
- `engines/pcre2/pcre2.wasm` and
`engines/python/pyodide.asm.wasm` must use `application/wasm`.
- Engine `.mjs` files must use a JavaScript MIME type;
`engines/python/python_stdlib.zip`, JSON, licence, notice and checksum files
must be delivered without content transformation.
- `script-src` must permit self-hosted WebAssembly compilation, normally with
`'wasm-unsafe-eval'`; broad `'unsafe-eval'` is neither needed nor recommended.
- `engines/pcre2/pcre2.mjs`, metadata, checksums, licence and WASM should use
revalidation or a release-scoped immutable cache. They are not
- `connect-src 'self'` must allow the Pyodide worker to fetch its own
same-origin WebAssembly, standard-library and lock files.
- Every file below `engines/pcre2/`, `engines/python/` and `engines/java/`
should use revalidation or a release-scoped immutable cache. These are not
content-hashed filenames and must never drift under one release identity.
The Portal must consume the immutable release ZIP, verify its SHA-256, verify
manifest ID `de.add-ideas.regex-tools` and version `0.2.0`, and mount it at a
manifest ID `de.add-ideas.regex-tools` and version `0.3.0`, and mount it at a
relative target such as `apps/regex/`.
The package, manifest, tag, artifact URL and checksum must all use the v0.2.0
identity. The historical v0.1.0 artifact coordinates remain immutable and must
not be overwritten or reused.
The package, manifest, tag, artifact URL and checksum must all use the v0.3.0
identity. Historical artifact coordinates remain immutable and must not be
overwritten or reused.

9
docs/REFERENCE_IMPLEMENTATIONS.md
View File

@@ -1,14 +1,17 @@
# Reference implementations and sources
Inspection date: 2026-07-24.
Inspection date: 2026-07-27.
| Reference | Exact revision/version | Licence | Use and copying status |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------- |
| Regex Tools | Unborn repository at initial inspection; release identity `v0.1.0` | GPL-3.0-or-later | Application source written for this project. |
| Regex Tools | Release identity `v0.3.0` | GPL-3.0-or-later | Application source written for this project. |
| Toolbox SDK | `53c40a61ba1581246f65773fcbb1c1cfd31ac98e` / 0.2.2 | Apache-2.0 | Contract, shell and release conventions; package APIs used, no source adapted. |
| Toolbox Portal | `a9c31c8986c40a0097966318e925083302e91e13` / 0.5.0 | AGPL-3.0-only | Assembly/lock conventions inspected; no source copied into the app. |
| regexpp | 4.12.2; npm integrity `sha512-EriSTlt5OC9/7SXkRSCAhfSxxoSUgBm33OH+IkwbdpgoqsSsUg7y3uh+IICI/Qg4BBWr3U2i39RpmycbxMq4ew==` | MIT | Pinned ECMAScript syntax provider. Provider AST is normalized in a worker. |
| PCRE2 | signed tag 10.47; commit `f454e231fe5006dd7ff8f4693fd2b8eb94333429` | BSD-3-Clause WITH PCRE2-exception | Pinned offline build; generated verified WebAssembly pack is shipped, source is not vendored. |
| Pyodide | tag 314.0.3; commit `ac57031be7564f864d061cb37c5c152e59f83ad4`; pinned npm integrity | MPL-2.0 | Reviewed self-hosted runtime pack; no CDN use. |
| CPython | executable runtime identity 3.14.2 in Pyodide 314.0.3 | PSF-2.0 | Actual `re` compiler, matcher and replacement runtime; Python source is not vendored. |
| TeaVM | tag/commit `ee91b03e616c4b45401cd11fb0cd7eb0daf6649b` / 0.15.0 | Apache-2.0 | `java.util.regex` class library compiled into the verified module; not OpenJDK. |
| ECMAScript specification | <https://tc39.es/ecma262/>; living specification inspected 2026-07-24 | ECMA terms | Semantics reference; no prose copied. |
| regex101 | <https://regex101.com/> and public parser documentation | Product/commercial terms | Product reference only. No source, branding, explanations, reference prose or visual design copied. |
| `recheck` | 4.5.0 | MIT | Metadata inspected; not installed or shipped. |
@@ -17,5 +20,5 @@ Inspection date: 2026-07-24.
| RegexLib | Website inspected | Redistribution licence not established | No fixture copied. |
| RGXP.RU | Website inspected | Redistribution licence not established | No fixture copied. |
All v0.1.0 and v0.2.0 conformance cases are project-authored. See
All v0.1.0, v0.2.0 and v0.3.0 conformance cases are project-authored. See
`tests/fixtures/README.md`.

12
docs/RELEASE.md
View File

@@ -18,9 +18,13 @@ on publication failure, replaces an existing exact version only with
`--force`, and produces:
```text
release/regex-tools-0.2.0.zip
release/regex-tools-0.2.0.zip.sha256
release/regex-tools-0.3.0.zip
release/regex-tools-0.3.0.zip.sha256
```
The existing `release/regex-tools-0.1.0.zip` and matching sidecar are historical
immutable artifacts. Creating v0.2.0 must not replace, rename or remove them.
Before packaging, the engine verifier checks the closed PCRE2, Python and Java
file sets in `dist/`, including checksums, metadata, licences/notices, module
and WebAssembly identities and self-hosted relative asset references.
The existing v0.1.0 and v0.2.0 ZIPs and matching sidecars are historical
immutable artifacts. Creating v0.3.0 must not replace, rename or remove them.

30
docs/SECURITY.md
View File

@@ -19,6 +19,18 @@ untrusted.
- PCRE2 runs only in mandatory UTF/UCP mode. Exact UTF-8 byte boundaries are
normalized to editor UTF-16 offsets; lone surrogates are rejected instead of
being silently replaced by browser encoding.
- Python starts only after the worker verifies Pyodide, CPython and `re`
identities. Pattern, subject, flags and replacement remain string data passed
to the fixed application bridge; no user-supplied Python source is evaluated.
Native code-point boundaries are checked before conversion to editor UTF-16.
- Java starts only after the worker verifies the TeaVM bridge ABI, engine
identity and compiled self-test. It exposes only fixed bounded
`Pattern`/`Matcher` entry points; no JVM, Java class loading or user-supplied
JavaScript is available. The runtime is TeaVM, not OpenJDK.
- Every bundled engine directory has a closed file set, deterministic
metadata/checksums and licence material verified both before installation and
in `dist/`. Runtime imports and fetches resolve only against the same-origin
release path.
- Replacement templates are strings; executable callbacks are unavailable.
- Replacement/list token decorations, rows, chips, diagnostics and list
evaluation work have separate presentation/operation caps; incomplete list
@@ -72,10 +84,14 @@ untrusted.
predicate check runs in the selected killable engine worker. Timeout, crash,
cancellation and worker error remain distinct; truncated results fail
closed. Subjects and results are ephemeral and never uploaded or persisted.
- No backend, uploads, telemetry, remote corpus URL, CDN, remote font, `eval`,
`Function`, arbitrary command line or unsafe HTML rendering exists.
- No backend, uploads, telemetry, remote corpus URL, CDN, remote font,
arbitrary command line or unsafe HTML rendering exists. Application code
does not evaluate user-controlled JavaScript or Python. The generic generated
Pyodide loader contains an Emscripten dynamic-link branch that uses
JavaScript `eval`, but the shipped self-contained CPython/stdlib path does not
exercise it and loads under the CSP below without broad `'unsafe-eval'`.
The PCRE2-enabled build needs a CSP equivalent to:
The engine-enabled build needs a CSP equivalent to:
```text
default-src 'self';
@@ -91,9 +107,11 @@ base-uri 'none';
form-action 'none';
```
The Toolbox shell currently requires inline styles. Corpus ZIP generation and
PCRE2 use self-hosted modules and local workers under the existing
`worker-src` policy. Broad `unsafe-eval` is not required.
The Toolbox shell currently requires inline styles. Corpus ZIP generation,
PCRE2, Pyodide and TeaVM use self-hosted modules/assets and local workers under
the existing `worker-src` and `connect-src` policy. Broad `'unsafe-eval'` is not
required; `'wasm-unsafe-eval'` remains required for the two WebAssembly
runtimes.
Report vulnerabilities privately to the repository owner before public issue
details are posted.