lotobo/regex-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

feat: publish Regex Tools 0.1.0

Browse Source
This commit is contained in:
Albrecht Degering
2026-07-24 18:04:21 +02:00
commit 873b9b218d
136 changed files with 24212 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,41 @@
# Architecture
Regex Tools is a static browser application with framework-neutral regex
boundaries.
```text
React workbench
├─ Pattern SyntaxSupervisor → syntax.worker → Regexpp provider → normalized AST
├─ Replacement SyntaxSupervisor → syntax.worker → typed replacement tokens
├─ EngineSupervisor → ecmascript.worker → native RegExp → match DTOs
├─ Test SyntaxSupervisor + EngineSupervisor → isolated cancellable test runs
└─ project validator / serializer / IndexedDB persistence
```
Provider AST objects never cross the worker boundary. The syntax worker emits
application-owned nodes, tokens, diagnostics, capture definitions and explicit
provenance. React never imports or traverses regexpp's cyclic AST.
Every engine request carries a protocol version, monotonically increasing
request ID and worker generation. A supervisor allows one active request,
rejects stale responses, terminates on timeout/crash/cancel, and lazily creates
a new generation. Timeout is a distinct error state.
Pattern and replacement syntax use independent supervisors. A pattern snapshot
is bound to its exact pattern, ordered flags and revision; execution cannot use
acceptance or capture metadata from an earlier input. Replacement output is
built while the native engine produces the same bounded authoritative match
sequence that is returned to the UI.
Per-match replacement ranges and token contributions are deterministic mappings
over those actual matches, the complete replacement-token model and the bounded
output. They execute no replacement callback or user code and have separate
presentation and token-evaluation limits.
Native ranges are retained in the engine's declared unit. Browser editor ranges
are half-open UTF-16 code-unit ranges. Reusable converters cover UTF-8 byte and
Unicode code-point offsets for later flavours, including invalid-boundary
rejection and lone-surrogate detection.
The release boundary is `dist/` plus checked-in legal/source documents. The
Portal consumes the resulting immutable ZIP and never imports React source.

28
docs/ENGINE_BUILDS.md Normal file
View File

@@ -0,0 +1,28 @@
# Engine builds
Version 0.1.0 ships no WebAssembly engine.
`npm run engines:build` verifies that this release has no external pack to
build. After the production build, `npm run engines:verify` rejects any
undeclared engine asset and confirms the packaged ECMAScript-only notice.
Neither command accesses the network.
## PCRE2 feasibility record
The next flavour was spiked outside the application repository against official
PCRE2 10.47:
- signed tag object `cd007b4466798f66d479d1442a407099e7c40050`;
- peeled commit `f454e231fe5006dd7ff8f4693fd2b8eb94333429`;
- licence `BSD-3-Clause WITH PCRE2-exception`;
- Emscripten 6.0.4;
- 8-bit library, Unicode enabled, JIT disabled.
Node and Chromium tests demonstrated version/config reporting, Unicode,
named groups, global substitution, automatic and explicit callouts, and
match/depth/heap errors. This was a technical spike only. No spike source,
binary or PCRE2 source is included in v0.1.0.
Production PCRE2 support remains gated on a reviewed bridge, copied DTOs,
allocation cleanup, exact source build, UTF-8/UTF-16 mapping, trace/output caps,
worker recovery, deterministic assets, licences and browser conformance.

13
docs/EXECUTION_TRACES.md Normal file
View File

@@ -0,0 +1,13 @@
# Execution traces
Version 0.1.0 exposes no execution trace.
Browser ECMAScript APIs return compilation, match, capture and replacement
results but not the runtime's internal backtracking events. Regex Tools does
not label structural explanations, static hints or adjacent-result inference
as an actual engine trace.
The next PCRE2 slice will use actual automatic/explicit callout events from an
application-owned bridge. Event count, serialized bytes, match/depth/heap
limits and wall-clock termination must all be enforced. Classifications such
as “apparent backtrack” will remain visibly derived from adjacent events.

20
docs/EXPLANATION_MODEL.md Normal file
View File

@@ -0,0 +1,20 @@
# Explanation model
The community provider converts regexpp nodes into a stable application AST
inside the syntax worker. Nodes carry deterministic IDs, UTF-16 ranges, raw
source, capture metadata, quantifier bounds, assertion kind, support status,
provider identity and provenance.
Explanations are original deterministic text selected by node type. Nullable
and minimum/maximum consumed-length properties are derived recursively where
the normalized node semantics support them. Backreference lengths and exact
bounds for Unicode properties of strings remain unknown rather than being
guessed.
regexpp 4.12.2 does not expose tolerant recovery. A malformed pattern therefore
gets the provider's exact diagnostic plus an application-owned error node. No
partial regexpp AST is claimed.
The explanation tree is structural. It is never called an actual execution
trace. Selecting a node selects and scrolls its exact editor range; selecting
pattern text chooses the smallest enclosing node.

27
docs/EXTRACTION_MODEL.md Normal file
View File

@@ -0,0 +1,27 @@
# Extraction model
Extraction nodes contain actual engine match/capture values and ranges. The
syntax tree supplies capture names and parent relationships; engine results
supply participation, value and offsets.
Statuses distinguish:
- participated with non-empty text;
- participated and matched empty;
- did not participate;
- unavailable range;
- preview-truncated value.
Exact match and capture ranges are retained even when a displayed value exceeds
the 64 Ki UTF-16 per-value preview limit or the shared 8 Mi-unit preview
budget. Consumers must inspect the preview status rather than treating a
preview as complete. In particular, list export is disabled if its template
would consume an incomplete value.
Repeated ECMAScript captures expose only the final retained capture. The UI
labels this limitation and does not invent history. Later .NET support may add
actual engine capture-history nodes without degrading them to this model.
The syntactic hierarchy remains useful even if actual spans overlap or fall
outside a parent's geometry; native and normalized ranges are preserved rather
than “fixed”.

20
docs/FLAVOUR_SUPPORT.md Normal file
View File

@@ -0,0 +1,20 @@
# Flavour support
| Flavour | Syntax | Execution | Replacement | Captures | Trace | Native offsets |
| ------------ | ------------------------------------------------------------------ | ----------------------------------------------- | ---------------------------------------------------- | -------------------------------------------------- | ----------- | ------------------- |
| ECMAScript | regexpp 4.12.2 grammar; explanations partial/feature-matrix tested | Current browser `RegExp`; feature-matrix tested | Native matches; bounded ECMAScript `GetSubstitution` | Named/numbered; final repeated capture; no history | Unavailable | UTF-16 |
| PCRE2 | Unavailable in release | Technical spike only; not shipped | Unavailable | Unavailable | Unavailable | Planned UTF-8 bytes |
| PCRE | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable |
| Python `re` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned code points |
| Go `regexp` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned UTF-8 bytes |
| Rust `regex` | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned UTF-8 bytes |
| .NET | Unavailable | Unavailable | Unavailable | Planned capture history | Unavailable | Planned UTF-16 |
| Java | Unavailable | Unavailable | Unavailable | Unavailable | Unavailable | Planned UTF-16 |
ECMAScript tests cover `g`, `y`, `u`, internal `d`, zero-length iteration,
named groups, unmatched/empty/repeated groups, lookbehind, backreferences,
Unicode properties, astral input, replacement and result truncation.
Actual browser identity is displayed because ECMAScript behavior can evolve
with the runtime. Syntax acceptance and engine compilation are separate;
neither silently rewrites the pattern.

16
docs/PARSER_PROFILES.md Normal file
View File

@@ -0,0 +1,16 @@
# Syntax provider profile
Regex Tools has one public profile:
```text
community
```
It uses only reviewed open-source dependencies and builds without private
registry credentials. Version 0.1.0 parses ECMAScript with regexpp 4.12.2.
The product deliberately does not implement or reference an `@r101/parser`
profile. No commercial package alias, import, tarball, credential, licence key
or private CI path exists. Additional flavour syntax will be implemented
incrementally as open-source providers behind the same application-owned
interface.

66
docs/PERFORMANCE.md Normal file
View File

@@ -0,0 +1,66 @@
# Performance
Live parsing is debounced by 120 ms and live execution by 220 ms. Live runs use
a 250 ms wall-clock limit; manual runs default to 2 seconds and can be raised to
the configured 10-second maximum.
Tests exercise bounded parsing, execution, zero-length iteration, result
truncation and worker recovery. Browser performance depends on hardware,
browser, engine version, pattern and subject. The UI therefore reports observed
elapsed time and exact runtime identity but makes no universal throughput or
safety claim.
The production bundle separates syntax and execution workers. Interactive
syntax/extraction trees and editor decoration sets render at most 2,000 items
each and report both the rendered and actual totals. These are presentation
limits only: normalized syntax, exact ranges and bounded engine results remain
available to the capture table and eligible exports.
Replacement templates have a 64 Ki UTF-16 input limit. Every token below that
limit remains in the syntax model; the editor decorates at most 2,000 tokens,
the token list renders 500 rows, and at most 200 replacement diagnostics
(including an omission summary) cross into the UI. List templates have a 16 Ki
UTF-16 input limit and render 250 token chips.
The per-match replacement view renders at most 200 matches and 2,000
contributions, with a combined 50,000 token-evaluation budget. Original,
replacement and individual contribution text are previewed separately from the
bounded output.
Capture rows are collected under the central 100,000-row limit and rendered in
pages of 200. Individual result-value previews are capped at 64 Ki UTF-16 units
and share an 8 Mi-unit aggregate engine-result budget. The extraction tree
shows 100 UTF-16 units per value and capture-table cells show 512, with explicit
range lengths and separate engine-prefix labels; copy and eligible JSON export
retain the bounded engine value. List export is disabled when a source value is incomplete, so a bounded
preview cannot be mistaken for the full value. List generation additionally
stops at 250,000 template-token evaluations, a 64 Ki UTF-16 row preview, or an
8 Mi-unit aggregate materialization bound. The UI shows 500 list rows while an
eligible export still contains every generated row; any incomplete result
disables export.
Replacement output has a separate 64 MiB bound. Match/capture-limit
incompleteness and output-byte truncation are tracked separately; either makes
the aggregate replacement result incomplete.
Unit-test cases run through supervisors separate from the interactive workers.
Each case uses its configured timeout, cancellation kills the active worker,
and the suite has a 60-second aggregate wall limit. Suite state is capped at
1,000 cases; add, clone, current-result capture and JSON append-import all
enforce that cap and preflight a 32 MiB aggregate in-memory suite bound. Test
names, patterns, subjects, replacement templates and expected values have
explicit per-field limits; subjects must also fit the execution engine's UTF-8
byte cap. Failure diagnostics preview only bounded portions of expected and
actual values and retain at most 2 KiB UTF-8 per result, keeping 1,000 result
messages below the 2 MiB log budget. A complete replacement output above the
expected-value or aggregate-suite limit is not copied into draft state; a
lightweight `should-match` draft is offered when applicable.
Project import, export and local save have a 32 MiB aggregate UTF-8 JSON limit
in addition to field-specific limits. Standalone test-suite import has the same
32 MiB pre-parse file/document limit. Export and save count the indented JSON
representation against a byte budget before materializing or writing it.
IndexedDB schema version 2 indexes `updatedAt`; latest-project loading opens one
descending cursor and validates only that record instead of calling `getAll()`.
Formal cold/warm benchmarks, p95 statistics and growth charts are deferred to
the analysis milestone and will run in killable workers.

19
docs/PORTAL_REQUIREMENTS.md Normal file
View File

@@ -0,0 +1,19 @@
# Portal requirements
Regex Tools 0.1.0 is a static nested-path-safe application.
Required delivery behavior:
- JavaScript workers use a JavaScript MIME type.
- `toolbox-app.json`, HTML and legal/source documents should remain
revalidation/no-cache resources.
- Hashed Vite assets may use immutable long-term caching.
- CSP must allow `worker-src 'self' blob:`.
- No WebAssembly MIME rule or `wasm-unsafe-eval` is required by v0.1.0.
The Portal must consume the immutable release ZIP, verify its SHA-256, verify
manifest ID `de.add-ideas.regex-tools` and version `0.1.0`, and mount it at a
relative target such as `apps/regex/`.
When PCRE2 ships later, the smallest additional change will be
`application/wasm` delivery and `script-src 'self' 'wasm-unsafe-eval'`.

21
docs/REFERENCE_IMPLEMENTATIONS.md Normal file
View File

@@ -0,0 +1,21 @@
# Reference implementations and sources
Inspection date: 2026-07-24.
| Reference | Exact revision/version | Licence | Use and copying status |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------- |
| Regex Tools | Unborn repository at initial inspection; release identity `v0.1.0` | GPL-3.0-or-later | Application source written for this project. |
| Toolbox SDK | `53c40a61ba1581246f65773fcbb1c1cfd31ac98e` / 0.2.2 | Apache-2.0 | Contract, shell and release conventions; package APIs used, no source adapted. |
| Toolbox Portal | `a9c31c8986c40a0097966318e925083302e91e13` / 0.5.0 | AGPL-3.0-only | Assembly/lock conventions inspected; no source copied into the app. |
| regexpp | 4.12.2; npm integrity `sha512-EriSTlt5OC9/7SXkRSCAhfSxxoSUgBm33OH+IkwbdpgoqsSsUg7y3uh+IICI/Qg4BBWr3U2i39RpmycbxMq4ew==` | MIT | Pinned ECMAScript syntax provider. Provider AST is normalized in a worker. |
| PCRE2 | signed tag 10.47; commit `f454e231fe5006dd7ff8f4693fd2b8eb94333429` | BSD-3-Clause WITH PCRE2-exception | External technical spike only; not shipped or copied. |
| ECMAScript specification | <https://tc39.es/ecma262/>; living specification inspected 2026-07-24 | ECMA terms | Semantics reference; no prose copied. |
| regex101 | <https://regex101.com/> and public parser documentation | Product/commercial terms | Product reference only. No source, branding, explanations, reference prose or visual design copied. |
| `recheck` | 4.5.0 | MIT | Metadata inspected; not installed or shipped. |
| `regexp-ast-analysis` | 0.7.1 | MIT | Metadata inspected; not installed or shipped. |
| RegexHub | repository revision `d1d9f19d259745dfdd21935b9ef3e747b62b9bfb` | MIT | Potential future adversarial cases; no file copied. |
| RegexLib | Website inspected | Redistribution licence not established | No fixture copied. |
| RGXP.RU | Website inspected | Redistribution licence not established | No fixture copied. |
All v0.1.0 conformance cases are project-authored. See
`tests/fixtures/README.md`.

23
docs/RELEASE.md Normal file
View File

@@ -0,0 +1,23 @@
# Release process
1. Ensure the intended commit is clean and tagged.
2. Install the exact lockfile with `npm ci`.
3. Run `npm run release:artifact`.
4. Record the reported SHA-256, then run
`npm run package:release -- --force` twice and verify both reported digests
match it. Inspect an existing exact-version artifact before deliberately
replacing it.
5. Inspect ZIP paths, manifest, legal documents and absence of maps/secrets.
6. Publish ZIP and `.sha256` on the matching Gitea release.
7. Use the public immutable URL and digest in Toolbox Portal.
The package script sorts paths without locale-dependent collation, rejects
symlinks and credential-like paths, normalizes file timestamps and modes,
verifies manifest/source identity, stages both outputs and rolls back the pair
on publication failure, replaces an existing exact version only with
`--force`, and produces:
```text
release/regex-tools-0.1.0.zip
release/regex-tools-0.1.0.zip.sha256
```

51
docs/SECURITY.md Normal file
View File

@@ -0,0 +1,51 @@
# Security and privacy
Patterns, flags, replacement templates, subjects and imported projects are
untrusted.
- Syntax and execution use separate workers.
- Regex execution never occurs on the UI thread.
- Timeout, crash and cancellation terminate the worker.
- A terminated worker is never reused.
- Pattern, subject, replacement/list template, match, capture and output sizes
are bounded.
- Replacement templates are strings; executable callbacks are unavailable.
- Replacement/list token decorations, rows, chips, diagnostics and list
evaluation work have separate presentation/operation caps; incomplete list
results cannot be exported.
- Per-match replacement mapping executes no user code and is capped at 200
matches, 2,000 materialized contributions and 50,000 token evaluations.
- Imported JSON is aggregate-size-bounded before parsing, then field-validated
and never evaluated or auto-run. Export and IndexedDB save apply the same
32 MiB aggregate UTF-8 document bound before materialization or persistence.
- Project and standalone-suite test subjects are not exported by default, and
unit-test subjects require a separate opt-in before IndexedDB persistence.
- Unit-test add, update, clone and append-import paths share 1,000-case and
32 MiB aggregate state caps.
- Unit-test assertion diagnostics never stringify complete large values and
retain at most 2 KiB UTF-8 per case. Oversized replacement output is not
copied into an exact current-result draft.
- No backend, uploads, telemetry, remote corpus URL, CDN, remote font, `eval`,
`Function`, arbitrary command line or unsafe HTML rendering exists.
Version 0.1.0 needs a CSP equivalent to:
```text
default-src 'self';
script-src 'self';
worker-src 'self' blob:;
connect-src 'self';
img-src 'self' data:;
font-src 'self';
style-src 'self' 'unsafe-inline';
object-src 'none';
frame-src 'none';
base-uri 'none';
form-action 'none';
```
The Toolbox shell currently requires inline styles. No `unsafe-eval` or
`wasm-unsafe-eval` is required until an actual WebAssembly flavour ships.
Report vulnerabilities privately to the repository owner before public issue
details are posted.