regex-tools/docs/GENERATED_CASES.md

Generated test cases

Regex Tools generator version 1 creates deterministic candidate subjects from the application-owned normalized AST and then verifies every candidate through the selected actual engine adapter. Generation is local-only and bounded. It is sampling, not proof of language coverage, equivalence or safety.

Supported scope

Version 1 is scoped to an accepted ECMAScript 2025 normalized AST from the bundled regexpp-based provider. It requests:

• a shortest structural candidate using the shortest visible alternative and each quantifier minimum;
• each structurally visible alternative;
• quantifier minimum, adjacent and bounded upper values;
• optional-absent and optional-present values;
• bounded character-class, dot and Unicode-property representatives;
• newline and non-boundary context around anchors;
• deletion, replacement and boundary mutations as likely near misses;
• deterministic random alternative, quantifier and class choices.

The coverage report says covered, partial, unsupported or not applicable for each category. “Covered” means that the documented bounded sampling strategy was requested. It does not mean every string in the regular language was enumerated.

Standalone PCRE2, PHP, Perl, Python, Ruby, Java, C++, Go, Rust, .NET and Scala compatibility generation is unavailable in version 1. Their actual engines are complete for the advertised execution operations, but their current structural syntax providers are intentionally partial. Regex Tools does not feed their syntax to the ECMAScript generator or relabel ECMAScript behaviour.

Unsupported and partial constructs

The report includes exact normalized source ranges and a reason for every recognized gap, capped at 250 rendered entries. Version 1 does not solve:

• capture-dependent backreferences;
• lookaround or word-boundary constraints symbolically;
• recursion, subroutine calls or conditional execution;
• branch-reset and atomic grouping;
• engine control verbs or callouts;
• complex Unicode-set algebra and properties of strings;
• recovered or provider-unsupported syntax.

Surrounding candidates may still happen to satisfy a lookaround or boundary. They are retained only when the actual selected engine confirms the requested match outcome. That confirmation does not upgrade the generator’s structural coverage claim.

Determinism and seed

The seed is explicit text of 1–128 UTF-16 units. Generator version 1 hashes its UTF-16 code units with a documented stable 32-bit FNV-1a step and drives an application-owned 32-bit deterministic sequence. It never calls Math.random().

The same generator version, normalized AST, flags, settings and seed produce the same candidate order and candidate identifiers. Verification timing is environment-dependent, and strict wall-time or engine-timeout bounds can therefore stop two runs at different points. The UI records both the original seed and its eight-digit hash. Generated unit tests retain:

• generator id and version;
• the original seed;
• candidate id;
• intended positive or negative outcome.

That provenance survives project and test-suite JSON export when subjects are included. Editing a generated test turns it into a user-authored test and removes the generation provenance.

Actual-engine verification

Candidate synthesis runs in a dedicated killable worker. The verifier then sends each exact candidate, pattern, flavour version, flags, engine options and scan mode through EngineSupervisor, which selects the registered real engine worker. Version 1's eligibility gate accepts only the exact current ECMAScript snapshot; the generic supervisor boundary does not make partial-provider flavours eligible.

• An intended positive is labelled should-match only after the engine returns at least one match.
• An intended negative is labelled should-not-match only after the engine returns no match.
• A candidate with the opposite result is discarded and retained only as a bounded advanced diagnostic.
• Engine compile rejection, timeout, worker crash, cancellation and ordinary execution error remain distinct outcomes.
• Cancellation terminates both candidate-synthesis and active engine workers.
• Configuration changes invalidate results immediately; old cases are never displayed against a new pattern or engine configuration.

The engine name, version, adapter, runtime and native offset unit shown with the result come from the actual execution result, not static marketing metadata.

Bounds

Defaults:

Bound	Default
Retained verified cases	48
Candidate attempts	192
Seeded random variants	16
Per-case engine timeout	500 ms
Aggregate wall time	15 s
Bytes per candidate subject	64 KiB
Aggregate candidate subject bytes	1 MiB
Synthesized repetitions per quantifier	32
Normalized AST traversal	20,000 nodes / 512 levels

Hard caps:

Bound	Hard cap
Retained verified cases	1,000
Candidate attempts	4,000
Per-case engine timeout	10 s
Aggregate wall time	30 s
Bytes per candidate subject	16 MiB
Aggregate candidate subject bytes	16 MiB
Synthesized repetitions per quantifier	1,024
Advanced discard diagnostics	250

UTF-8 sizes are checked before a candidate is retained for verification. Quantifier expansion is preflighted before String.prototype.repeat allocates the output. Reaching a case, attempt, byte, AST, repetition or wall-time bound is visible in the result; it is never reported as complete coverage.

Privacy and persistence

No pattern, candidate, verified subject, diagnostic or seed is uploaded. Results and settings are ephemeral until the user explicitly adds cases to the unit-test suite or downloads the verified JSON artifact. Test-suite export omits subjects by default because generated subjects may still encode literal pattern content. Enabling subject export is an explicit user choice.