2.3 KiB
Architecture
Unicode Tools is a relocatable static React application. Pure text operations
live in src/unicode/model.ts, while catalog loading, search, character detail,
Unihan lookup, and confusable analysis run through a dedicated module worker.
Reusable encoding, scalar, grapheme, normalization and download primitives come
from @add-ideas/toolbox-helpers.
The build contains compressed-by-HTTP JSON shards rather than a runtime database. A small manifest and dictionaries load first. The first catalog query loads assigned character shards, emoji and named sequences once in the worker; the roughly 5 MiB Unihan selection is fetched only after an explicit request. Results and displayed mapping lists are capped. The UI retains no database or persistent search history.
scripts/generate-unicode-data.mjs converts checksum-pinned Unicode 17.0
archives and text files into deterministic checked-in source data. It derives
names, aliases, general categories, scripts, blocks, ages, bidi and case
properties, emoji, named sequences, selected Unihan fields, and UTS #39
confusable mappings. The code chart virtualizes complete Unicode block or
custom-range traversal into fixed row windows, requesting at most 512 entries
from the worker. Assigned records come from the pinned catalog; missing entries
are classified explicitly as unassigned, private-use, surrogate, or
noncharacter instead of being omitted. Private-use and surrogate ranges are not
expanded into the checked-in catalog itself.
The generated catalog also includes lazy worker shards for the checksum-pinned
UTS #39 IdentifierStatus.txt and IdentifierType.txt ranges. Worker analysis
loads them with the confusable table only when requested, applies those exact
ranges per scalar, creates NFD skeletons, and emits structured diagnostics plus
an intentionally conservative restriction-level estimate. unicode/graphemes.ts layers bounded structural
inspection and small sequence-editing operations on the shared
Intl.Segmenter wrapper; those operations construct code-point sequences but
do not alter or validate the pinned RGI emoji catalog.
Rendering, normalization and grapheme segmentation use the current browser and operating-system Unicode implementation, which can differ from the pinned catalog version. The interface discloses that boundary.