39 lines
2.3 KiB
Markdown
39 lines
2.3 KiB
Markdown
# Architecture
|
|
|
|
Unicode Tools is a relocatable static React application. Pure text operations
|
|
live in `src/unicode/model.ts`, while catalog loading, search, character detail,
|
|
Unihan lookup, and confusable analysis run through a dedicated module worker.
|
|
Reusable encoding, scalar, grapheme, normalization and download primitives come
|
|
from `@add-ideas/toolbox-helpers`.
|
|
|
|
The build contains compressed-by-HTTP JSON shards rather than a runtime
|
|
database. A small manifest and dictionaries load first. The first catalog query
|
|
loads assigned character shards, emoji and named sequences once in the worker;
|
|
the roughly 5 MiB Unihan selection is fetched only after an explicit request.
|
|
Results and displayed mapping lists are capped. The UI retains no database or
|
|
persistent search history.
|
|
|
|
`scripts/generate-unicode-data.mjs` converts checksum-pinned Unicode 17.0
|
|
archives and text files into deterministic checked-in source data. It derives
|
|
names, aliases, general categories, scripts, blocks, ages, bidi and case
|
|
properties, emoji, named sequences, selected Unihan fields, and UTS #39
|
|
confusable mappings. The code chart virtualizes complete Unicode block or
|
|
custom-range traversal into fixed row windows, requesting at most 512 entries
|
|
from the worker. Assigned records come from the pinned catalog; missing entries
|
|
are classified explicitly as unassigned, private-use, surrogate, or
|
|
noncharacter instead of being omitted. Private-use and surrogate ranges are not
|
|
expanded into the checked-in catalog itself.
|
|
|
|
The generated catalog also includes lazy worker shards for the checksum-pinned
|
|
UTS #39 `IdentifierStatus.txt` and `IdentifierType.txt` ranges. Worker analysis
|
|
loads them with the confusable table only when requested, applies those exact
|
|
ranges per scalar, creates NFD skeletons, and emits structured diagnostics plus
|
|
an intentionally conservative restriction-level estimate. `unicode/graphemes.ts` layers bounded structural
|
|
inspection and small sequence-editing operations on the shared
|
|
`Intl.Segmenter` wrapper; those operations construct code-point sequences but
|
|
do not alter or validate the pinned RGI emoji catalog.
|
|
|
|
Rendering, normalization and grapheme segmentation use the current browser and
|
|
operating-system Unicode implementation, which can differ from the pinned
|
|
catalog version. The interface discloses that boundary.
|