139 lines
6.4 KiB
Markdown
139 lines
6.4 KiB
Markdown
# Generated test cases
|
||
|
||
Regex Tools generator version 1 creates deterministic candidate subjects from
|
||
the application-owned normalized AST and then verifies every candidate through
|
||
the selected actual engine adapter. Generation is local-only and bounded. It is
|
||
sampling, not proof of language coverage, equivalence or safety.
|
||
|
||
## Supported scope
|
||
|
||
Version 1 is scoped to an accepted ECMAScript 2025 normalized AST from the
|
||
bundled regexpp-based provider. It requests:
|
||
|
||
- a shortest structural candidate using the shortest visible alternative and
|
||
each quantifier minimum;
|
||
- each structurally visible alternative;
|
||
- quantifier minimum, adjacent and bounded upper values;
|
||
- optional-absent and optional-present values;
|
||
- bounded character-class, dot and Unicode-property representatives;
|
||
- newline and non-boundary context around anchors;
|
||
- deletion, replacement and boundary mutations as likely near misses;
|
||
- deterministic random alternative, quantifier and class choices.
|
||
|
||
The coverage report says `covered`, `partial`, `unsupported` or
|
||
`not applicable` for each category. “Covered” means that the documented bounded
|
||
sampling strategy was requested. It does not mean every string in the regular
|
||
language was enumerated.
|
||
|
||
Standalone PCRE2, PHP, Perl, Python, Ruby, Java, C++, Go, Rust, .NET and Scala
|
||
compatibility generation is unavailable in version 1. Their actual engines are
|
||
complete for the advertised execution operations, but their current structural
|
||
syntax providers are intentionally partial. Regex Tools does not feed their
|
||
syntax to the ECMAScript generator or relabel ECMAScript behaviour.
|
||
|
||
## Unsupported and partial constructs
|
||
|
||
The report includes exact normalized source ranges and a reason for every
|
||
recognized gap, capped at 250 rendered entries. Version 1 does not solve:
|
||
|
||
- capture-dependent backreferences;
|
||
- lookaround or word-boundary constraints symbolically;
|
||
- recursion, subroutine calls or conditional execution;
|
||
- branch-reset and atomic grouping;
|
||
- engine control verbs or callouts;
|
||
- complex Unicode-set algebra and properties of strings;
|
||
- recovered or provider-unsupported syntax.
|
||
|
||
Surrounding candidates may still happen to satisfy a lookaround or boundary.
|
||
They are retained only when the actual selected engine confirms the requested
|
||
match outcome. That confirmation does not upgrade the generator’s structural
|
||
coverage claim.
|
||
|
||
## Determinism and seed
|
||
|
||
The seed is explicit text of 1–128 UTF-16 units. Generator version 1 hashes its
|
||
UTF-16 code units with a documented stable 32-bit FNV-1a step and drives an
|
||
application-owned 32-bit deterministic sequence. It never calls
|
||
`Math.random()`.
|
||
|
||
The same generator version, normalized AST, flags, settings and seed produce
|
||
the same candidate order and candidate identifiers. Verification timing is
|
||
environment-dependent, and strict wall-time or engine-timeout bounds can
|
||
therefore stop two runs at different points. The UI records both the original
|
||
seed and its eight-digit hash. Generated unit tests retain:
|
||
|
||
- generator id and version;
|
||
- the original seed;
|
||
- candidate id;
|
||
- intended positive or negative outcome.
|
||
|
||
That provenance survives project and test-suite JSON export when subjects are
|
||
included. Editing a generated test turns it into a user-authored test and
|
||
removes the generation provenance.
|
||
|
||
## Actual-engine verification
|
||
|
||
Candidate synthesis runs in a dedicated killable worker. The verifier then
|
||
sends each exact candidate, pattern, flavour version, flags, engine options and
|
||
scan mode through `EngineSupervisor`, which selects the registered real engine
|
||
worker. Version 1's eligibility gate accepts only the exact current ECMAScript
|
||
snapshot; the generic supervisor boundary does not make partial-provider
|
||
flavours eligible.
|
||
|
||
- An intended positive is labelled `should-match` only after the engine returns
|
||
at least one match.
|
||
- An intended negative is labelled `should-not-match` only after the engine
|
||
returns no match.
|
||
- A candidate with the opposite result is discarded and retained only as a
|
||
bounded advanced diagnostic.
|
||
- Engine compile rejection, timeout, worker crash, cancellation and ordinary
|
||
execution error remain distinct outcomes.
|
||
- Cancellation terminates both candidate-synthesis and active engine workers.
|
||
- Configuration changes invalidate results immediately; old cases are never
|
||
displayed against a new pattern or engine configuration.
|
||
|
||
The engine name, version, adapter, runtime and native offset unit shown with the
|
||
result come from the actual execution result, not static marketing metadata.
|
||
|
||
## Bounds
|
||
|
||
Defaults:
|
||
|
||
| Bound | Default |
|
||
| -------------------------------------- | ------------------------: |
|
||
| Retained verified cases | 48 |
|
||
| Candidate attempts | 192 |
|
||
| Seeded random variants | 16 |
|
||
| Per-case engine timeout | 500 ms |
|
||
| Aggregate wall time | 15 s |
|
||
| Bytes per candidate subject | 64 KiB |
|
||
| Aggregate candidate subject bytes | 1 MiB |
|
||
| Synthesized repetitions per quantifier | 32 |
|
||
| Normalized AST traversal | 20,000 nodes / 512 levels |
|
||
|
||
Hard caps:
|
||
|
||
| Bound | Hard cap |
|
||
| -------------------------------------- | -------: |
|
||
| Retained verified cases | 1,000 |
|
||
| Candidate attempts | 4,000 |
|
||
| Per-case engine timeout | 10 s |
|
||
| Aggregate wall time | 30 s |
|
||
| Bytes per candidate subject | 16 MiB |
|
||
| Aggregate candidate subject bytes | 16 MiB |
|
||
| Synthesized repetitions per quantifier | 1,024 |
|
||
| Advanced discard diagnostics | 250 |
|
||
|
||
UTF-8 sizes are checked before a candidate is retained for verification.
|
||
Quantifier expansion is preflighted before `String.prototype.repeat` allocates
|
||
the output. Reaching a case, attempt, byte, AST, repetition or wall-time bound
|
||
is visible in the result; it is never reported as complete coverage.
|
||
|
||
## Privacy and persistence
|
||
|
||
No pattern, candidate, verified subject, diagnostic or seed is uploaded.
|
||
Results and settings are ephemeral until the user explicitly adds cases to the
|
||
unit-test suite or downloads the verified JSON artifact. Test-suite export
|
||
omits subjects by default because generated subjects may still encode literal
|
||
pattern content. Enabling subject export is an explicit user choice.
|