feat(translator-tools): add locale review and glossary workflow
This commit is contained in:
+6
-8
@@ -1,13 +1,11 @@
|
||||
# Changelog
|
||||
|
||||
## 0.2.0 - 2026-09-02
|
||||
## Unreleased
|
||||
|
||||
- Added composable literal and line transforms with allocation preflights, plus BOM, UTF-8 validity, zero-byte and byte-newline evidence.
|
||||
- Added paired text/artifact-evidence downloads with source/output hashes and
|
||||
pipeline provenance; automatic Toolbox handoff remains intentionally disabled
|
||||
pending the coordinated Portal consumer rollout.
|
||||
- Added local translator workspace for i18next-like and single-locale JSON review.
|
||||
- Added language pair selection, glossary storage and fuzzy in-context suggestions.
|
||||
- Added local-only import/export pipeline with portal handoff integration.
|
||||
|
||||
## 0.1.0 - 2026-09-01
|
||||
## 0.1.0 - 2026-09-02
|
||||
|
||||
- Added the initial local-first Text Tools workbench.
|
||||
- Added Toolbox integration, bounded input handling, offline shell and deterministic release packaging.
|
||||
- Initial local-first translator review implementation.
|
||||
|
||||
@@ -1,24 +1,25 @@
|
||||
# Text Tools
|
||||
# Translator Tools
|
||||
|
||||
Compose, inspect and evidence plain-text transformations locally in the browser.
|
||||
Translator Tools is a standalone local-first application in the
|
||||
[add·ideas Toolbox](https://git.add-ideas.de/lotobo/toolbox-portal). Inputs are
|
||||
parsed and edited in the browser only; no locale strings are uploaded.
|
||||
|
||||
Text Tools is a standalone local-first application in the [add·ideas Toolbox](https://git.add-ideas.de/lotobo/toolbox-portal). Inputs are processed in the browser and are not uploaded.
|
||||
## Quick scope
|
||||
|
||||
## Version 0.2 scope
|
||||
- Import locale files (`.json`, `.js`, `.txt`) in i18next-like or single-locale forms.
|
||||
- Review authoritative language against one target language at a time.
|
||||
- Edit source and target strings and copy-source quickly.
|
||||
- Add translation glossary entries per language pair and receive fuzzy suggestions.
|
||||
- Add new languages and export a reviewer-updated translation bundle locally.
|
||||
- Send exports through the same-tool transfer channel to the Portal.
|
||||
|
||||
- Ordered pipelines for line endings, trimming, whitespace, stable sorting, deduplication, literal replacement, line prefix/suffix/filter/number/join and line-order reversal
|
||||
- Locale-aware case conversion, Unicode NFC/NFD/NFKC/NFKD and best-effort Latin transliteration
|
||||
- JSON/HTML/URL/Base64/hex escaping plus strict inverse decode/unescape stages; binary decoders accept canonical encodings and require valid UTF-8 text
|
||||
- Bounded RFC-style quoted CSV column selection across embedded delimiters and newlines, alongside the retained literal multi-character delimiter-per-line mode
|
||||
- Explicit UTF-8, UTF-16LE and Latin-1 file decoding plus BOM, malformed UTF-8 offset, zero-byte parity, byte-level newline and replacement-character evidence
|
||||
- Versioned JSON recipe import/export, per-step size/change reports, and a paired text-plus-JSON artifact download with source/output SHA-256 and transformation provenance
|
||||
## Notes
|
||||
|
||||
Pipelines are limited to 2,000,000 UTF-16 input units and an 8×/32 MiB output expansion. File input is limited to 16 MiB before decoding. Encoding is selected explicitly rather than guessed; compatibility normalisation, transliteration, narrow encodings and several transforms can be lossy, so source, result, and warnings remain visible. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and [docs/PRIVACY-SECURITY.md](docs/PRIVACY-SECURITY.md).
|
||||
|
||||
The artifact evidence is a portable download contract, not an automatic Toolbox
|
||||
handoff. SDK 0.3.0 provides the shared transfer contract, but automatic Open With
|
||||
remains disabled in this release until the Portal consumer rollout is
|
||||
coordinated. Local downloads stay explicit and portable in the meantime.
|
||||
- The parser accepts JavaScript-style locale assignment forms such as
|
||||
`const de = { ... }` and tolerant spacing/commas.
|
||||
- Parsing and transfers are bounded to 5 MiB per imported/exported transfer file.
|
||||
- Language detection relies on file name hints (`de_*.json`) and explicit user
|
||||
selection.
|
||||
|
||||
## Development
|
||||
|
||||
@@ -32,7 +33,7 @@ npm run test:browser
|
||||
|
||||
## Release
|
||||
|
||||
`npm run release:artifact` creates a deterministic `release/text-tools-0.2.0.zip` and checksum sidecar.
|
||||
`npm run release:artifact` creates a deterministic `release/translator-tools-0.1.0.zip` and checksum sidecar.
|
||||
|
||||
## Licence
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Third-party notices
|
||||
|
||||
Text Tools 0.2.0 directly depends on these runtime packages:
|
||||
Translator Tools 0.1.0 directly depends on these runtime packages:
|
||||
|
||||
| Package | Pinned version | Declared licence |
|
||||
| -------------------------------- | -------------: | ---------------- |
|
||||
|
||||
+11
-4
@@ -1,9 +1,16 @@
|
||||
# Architecture
|
||||
|
||||
Text Tools is a static React/Vite application wrapped in the shared Toolbox shell. `text/pipeline.ts` defines an ordered, typed list of transformations. `applyPipeline` starts from the exact source, applies enabled steps one at a time, checks expansion after every step, and returns the last successful output with per-step before/after counts and warnings.
|
||||
Translator Tools is a static React/Vite application wrapped in the shared Toolbox shell.
|
||||
|
||||
Text encoding/decoding, line-ending conversion, bounded CSV parsing and Unicode-aware case primitives come from `@add-ideas/toolbox-helpers`. File decoding uses an explicit encoding and fatal-or-replacement choice; it does not guess a charset. Inverse JSON/HTML/URL/Base64/hex stages are strict: malformed syntax and non-UTF-8 byte output fail the step rather than being silently repaired. Column selection offers a quoted CSV mode that preserves embedded delimiters/newlines and a separate literal-delimiter-per-line mode. Recipe JSON carries a schema/app identity, accepts at most 100 supported steps, discards supplied IDs and assigns local monotonic IDs.
|
||||
The workbench parses a single imported locale file in-browser, normalizes either:
|
||||
|
||||
`text/evidence.ts` inspects original bytes before decoding, including BOM, exact UTF-8 validity/first invalid offset, zero-byte parity and CR/LF/CRLF counts. Artifact export emits the selected output bytes and a deterministic evidence document containing canonical-source and exact-output SHA-256 values, inventories, recipe and step reports. The document records that direct Toolbox handoff is unavailable in this release; the SDK 0.3.0 transfer contract is intentionally not activated until the Portal consumer rollout is coordinated.
|
||||
- i18next-like language maps (`{ "en": {...}, "de": {...} }`)
|
||||
- single-locale objects (`{ "key": "value" }`, or JS-style assignment forms like `const de = {...}`).
|
||||
|
||||
Version 0.2 operations are synchronous and bounded, so the app creates no worker and has no persistence or server API. Relative entry and asset URLs keep the build relocatable below a nested portal path.
|
||||
Strings are flattened into stable dot paths for review/editing and can be serialized
|
||||
back with an inverse unflattening step. Suggestions come from local glossary entries
|
||||
and are matched by language pair and fuzzy string distance.
|
||||
|
||||
No background workers are required today. Persistent state is limited to a local
|
||||
Storage glossary cache; everything else remains in process memory until download or
|
||||
transfer.
|
||||
|
||||
@@ -1,9 +1,16 @@
|
||||
# Privacy and security
|
||||
|
||||
Source text, imported files, recipes and results stay in page memory. There is no telemetry, analytics, account, persistence or runtime network path. Imported text is rendered as text/preformatted content, not executable HTML; HTML escaping produces a string and does not preview it as markup.
|
||||
Locale files and generated outputs stay in-browser and are not uploaded.
|
||||
|
||||
Files are rejected above 16 MiB before reading. Decoded/pasted pipeline input is limited to 2,000,000 UTF-16 units, recipe text to 1,000,000 units/100 steps, and output to both 8× the original (with a small-input floor) and 32 MiB. Quoted CSV selection additionally limits rows, columns and field size. Strict inverse stages reject malformed escapes/encodings and invalid UTF-8; they do not attempt permissive repair. These limits reduce accidental expansion but do not prove that output is safe for a downstream interpreter.
|
||||
No accounts, telemetry or runtime network requests are used for the core workflow.
|
||||
The app stores glossary terms in local browser storage so they are available on the same
|
||||
device for repeated reviews.
|
||||
|
||||
Compatibility normalisation, case conversion, transliteration, whitespace/line transforms, column selection and narrow encodings can change or lose information. Locale sorting depends on the browser's `Intl.Collator`. The exact source, output, per-step counts and warnings remain visible so users can review those changes before copying or downloading.
|
||||
Parser/serializer limits are bounded to prevent accidental expansion:
|
||||
|
||||
Byte evidence does not guess an encoding or prove the selected decoder is correct. A valid UTF-8 byte sequence may represent text intended for another encoding, zero-byte parity is only a UTF-16 review signal, and replacement characters may already have existed in the source. Artifact evidence hashes canonical UTF-8 source text and exact exported bytes, clearly labels both, and includes filenames and transformation details that may themselves be sensitive.
|
||||
- Imports and exports are capped at 5 MiB each.
|
||||
- Large objects are flattened and rendered with per-session bounds.
|
||||
|
||||
Suggested translations are purely local matching; they do not call external services.
|
||||
Transfers to the Portal are signed local handoffs only and never stored by a remote
|
||||
processor by default.
|
||||
|
||||
Generated
+486
-71
File diff suppressed because it is too large
Load Diff
+7
-6
@@ -1,16 +1,16 @@
|
||||
{
|
||||
"name": "text-tools",
|
||||
"version": "0.2.0",
|
||||
"description": "Transform and inspect plain text locally in the browser.",
|
||||
"name": "translator-tools",
|
||||
"version": "0.1.0",
|
||||
"description": "Review and translate i18next-like JSON language files locally.",
|
||||
"license": "GPL-3.0-or-later",
|
||||
"author": "Albrecht Degering",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://git.add-ideas.de/lotobo/text-tools.git"
|
||||
"url": "git+https://git.add-ideas.de/lotobo/translator-tools.git"
|
||||
},
|
||||
"homepage": "https://git.add-ideas.de/lotobo/text-tools",
|
||||
"homepage": "https://git.add-ideas.de/lotobo/translator-tools",
|
||||
"bugs": {
|
||||
"url": "https://git.add-ideas.de/lotobo/text-tools/issues"
|
||||
"url": "https://git.add-ideas.de/lotobo/translator-tools/issues"
|
||||
},
|
||||
"private": true,
|
||||
"type": "module",
|
||||
@@ -42,6 +42,7 @@
|
||||
"@add-ideas/toolbox-contract": "0.3.0",
|
||||
"@add-ideas/toolbox-helpers": "0.2.0",
|
||||
"@add-ideas/toolbox-shell-react": "0.3.0",
|
||||
"json5": "^2.2.3",
|
||||
"react": "19.2.8",
|
||||
"react-dom": "19.2.8"
|
||||
},
|
||||
|
||||
+6
-8
@@ -1,13 +1,11 @@
|
||||
# Changelog
|
||||
|
||||
## 0.2.0 - 2026-09-02
|
||||
## Unreleased
|
||||
|
||||
- Added composable literal and line transforms with allocation preflights, plus BOM, UTF-8 validity, zero-byte and byte-newline evidence.
|
||||
- Added paired text/artifact-evidence downloads with source/output hashes and
|
||||
pipeline provenance; automatic Toolbox handoff remains intentionally disabled
|
||||
pending the coordinated Portal consumer rollout.
|
||||
- Added local translator workspace for i18next-like and single-locale JSON review.
|
||||
- Added language pair selection, glossary storage and fuzzy in-context suggestions.
|
||||
- Added local-only import/export pipeline with portal handoff integration.
|
||||
|
||||
## 0.1.0 - 2026-09-01
|
||||
## 0.1.0 - 2026-09-02
|
||||
|
||||
- Added the initial local-first Text Tools workbench.
|
||||
- Added Toolbox integration, bounded input handling, offline shell and deterministic release packaging.
|
||||
- Initial local-first translator review implementation.
|
||||
|
||||
@@ -1077,6 +1077,36 @@ Declared licence: Apache-2.0
|
||||
limitations under the License.
|
||||
|
||||
|
||||
==============================================================================
|
||||
json5@2.2.3
|
||||
Declared licence: MIT
|
||||
==============================================================================
|
||||
--- LICENSE.md ---
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2012-2018 Aseem Kishore, and [others].
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
[others]: https://github.com/json5/json5/contributors
|
||||
|
||||
|
||||
==============================================================================
|
||||
react@19.2.8
|
||||
Declared licence: MIT
|
||||
|
||||
+18
-17
@@ -1,24 +1,25 @@
|
||||
# Text Tools
|
||||
# Translator Tools
|
||||
|
||||
Compose, inspect and evidence plain-text transformations locally in the browser.
|
||||
Translator Tools is a standalone local-first application in the
|
||||
[add·ideas Toolbox](https://git.add-ideas.de/lotobo/toolbox-portal). Inputs are
|
||||
parsed and edited in the browser only; no locale strings are uploaded.
|
||||
|
||||
Text Tools is a standalone local-first application in the [add·ideas Toolbox](https://git.add-ideas.de/lotobo/toolbox-portal). Inputs are processed in the browser and are not uploaded.
|
||||
## Quick scope
|
||||
|
||||
## Version 0.2 scope
|
||||
- Import locale files (`.json`, `.js`, `.txt`) in i18next-like or single-locale forms.
|
||||
- Review authoritative language against one target language at a time.
|
||||
- Edit source and target strings and copy-source quickly.
|
||||
- Add translation glossary entries per language pair and receive fuzzy suggestions.
|
||||
- Add new languages and export a reviewer-updated translation bundle locally.
|
||||
- Send exports through the same-tool transfer channel to the Portal.
|
||||
|
||||
- Ordered pipelines for line endings, trimming, whitespace, stable sorting, deduplication, literal replacement, line prefix/suffix/filter/number/join and line-order reversal
|
||||
- Locale-aware case conversion, Unicode NFC/NFD/NFKC/NFKD and best-effort Latin transliteration
|
||||
- JSON/HTML/URL/Base64/hex escaping plus strict inverse decode/unescape stages; binary decoders accept canonical encodings and require valid UTF-8 text
|
||||
- Bounded RFC-style quoted CSV column selection across embedded delimiters and newlines, alongside the retained literal multi-character delimiter-per-line mode
|
||||
- Explicit UTF-8, UTF-16LE and Latin-1 file decoding plus BOM, malformed UTF-8 offset, zero-byte parity, byte-level newline and replacement-character evidence
|
||||
- Versioned JSON recipe import/export, per-step size/change reports, and a paired text-plus-JSON artifact download with source/output SHA-256 and transformation provenance
|
||||
## Notes
|
||||
|
||||
Pipelines are limited to 2,000,000 UTF-16 input units and an 8×/32 MiB output expansion. File input is limited to 16 MiB before decoding. Encoding is selected explicitly rather than guessed; compatibility normalisation, transliteration, narrow encodings and several transforms can be lossy, so source, result, and warnings remain visible. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and [docs/PRIVACY-SECURITY.md](docs/PRIVACY-SECURITY.md).
|
||||
|
||||
The artifact evidence is a portable download contract, not an automatic Toolbox
|
||||
handoff. SDK 0.3.0 provides the shared transfer contract, but automatic Open With
|
||||
remains disabled in this release until the Portal consumer rollout is
|
||||
coordinated. Local downloads stay explicit and portable in the meantime.
|
||||
- The parser accepts JavaScript-style locale assignment forms such as
|
||||
`const de = { ... }` and tolerant spacing/commas.
|
||||
- Parsing and transfers are bounded to 5 MiB per imported/exported transfer file.
|
||||
- Language detection relies on file name hints (`de_*.json`) and explicit user
|
||||
selection.
|
||||
|
||||
## Development
|
||||
|
||||
@@ -32,7 +33,7 @@ npm run test:browser
|
||||
|
||||
## Release
|
||||
|
||||
`npm run release:artifact` creates a deterministic `release/text-tools-0.2.0.zip` and checksum sidecar.
|
||||
`npm run release:artifact` creates a deterministic `release/translator-tools-0.1.0.zip` and checksum sidecar.
|
||||
|
||||
## Licence
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Third-party notices
|
||||
|
||||
Text Tools 0.2.0 directly depends on these runtime packages:
|
||||
Translator Tools 0.1.0 directly depends on these runtime packages:
|
||||
|
||||
| Package | Pinned version | Declared licence |
|
||||
| -------------------------------- | -------------: | ---------------- |
|
||||
|
||||
@@ -1,9 +1,16 @@
|
||||
# Architecture
|
||||
|
||||
Text Tools is a static React/Vite application wrapped in the shared Toolbox shell. `text/pipeline.ts` defines an ordered, typed list of transformations. `applyPipeline` starts from the exact source, applies enabled steps one at a time, checks expansion after every step, and returns the last successful output with per-step before/after counts and warnings.
|
||||
Translator Tools is a static React/Vite application wrapped in the shared Toolbox shell.
|
||||
|
||||
Text encoding/decoding, line-ending conversion, bounded CSV parsing and Unicode-aware case primitives come from `@add-ideas/toolbox-helpers`. File decoding uses an explicit encoding and fatal-or-replacement choice; it does not guess a charset. Inverse JSON/HTML/URL/Base64/hex stages are strict: malformed syntax and non-UTF-8 byte output fail the step rather than being silently repaired. Column selection offers a quoted CSV mode that preserves embedded delimiters/newlines and a separate literal-delimiter-per-line mode. Recipe JSON carries a schema/app identity, accepts at most 100 supported steps, discards supplied IDs and assigns local monotonic IDs.
|
||||
The workbench parses a single imported locale file in-browser, normalizes either:
|
||||
|
||||
`text/evidence.ts` inspects original bytes before decoding, including BOM, exact UTF-8 validity/first invalid offset, zero-byte parity and CR/LF/CRLF counts. Artifact export emits the selected output bytes and a deterministic evidence document containing canonical-source and exact-output SHA-256 values, inventories, recipe and step reports. The document records that direct Toolbox handoff is unavailable in this release; the SDK 0.3.0 transfer contract is intentionally not activated until the Portal consumer rollout is coordinated.
|
||||
- i18next-like language maps (`{ "en": {...}, "de": {...} }`)
|
||||
- single-locale objects (`{ "key": "value" }`, or JS-style assignment forms like `const de = {...}`).
|
||||
|
||||
Version 0.2 operations are synchronous and bounded, so the app creates no worker and has no persistence or server API. Relative entry and asset URLs keep the build relocatable below a nested portal path.
|
||||
Strings are flattened into stable dot paths for review/editing and can be serialized
|
||||
back with an inverse unflattening step. Suggestions come from local glossary entries
|
||||
and are matched by language pair and fuzzy string distance.
|
||||
|
||||
No background workers are required today. Persistent state is limited to a local
|
||||
Storage glossary cache; everything else remains in process memory until download or
|
||||
transfer.
|
||||
|
||||
@@ -1,9 +1,16 @@
|
||||
# Privacy and security
|
||||
|
||||
Source text, imported files, recipes and results stay in page memory. There is no telemetry, analytics, account, persistence or runtime network path. Imported text is rendered as text/preformatted content, not executable HTML; HTML escaping produces a string and does not preview it as markup.
|
||||
Locale files and generated outputs stay in-browser and are not uploaded.
|
||||
|
||||
Files are rejected above 16 MiB before reading. Decoded/pasted pipeline input is limited to 2,000,000 UTF-16 units, recipe text to 1,000,000 units/100 steps, and output to both 8× the original (with a small-input floor) and 32 MiB. Quoted CSV selection additionally limits rows, columns and field size. Strict inverse stages reject malformed escapes/encodings and invalid UTF-8; they do not attempt permissive repair. These limits reduce accidental expansion but do not prove that output is safe for a downstream interpreter.
|
||||
No accounts, telemetry or runtime network requests are used for the core workflow.
|
||||
The app stores glossary terms in local browser storage so they are available on the same
|
||||
device for repeated reviews.
|
||||
|
||||
Compatibility normalisation, case conversion, transliteration, whitespace/line transforms, column selection and narrow encodings can change or lose information. Locale sorting depends on the browser's `Intl.Collator`. The exact source, output, per-step counts and warnings remain visible so users can review those changes before copying or downloading.
|
||||
Parser/serializer limits are bounded to prevent accidental expansion:
|
||||
|
||||
Byte evidence does not guess an encoding or prove the selected decoder is correct. A valid UTF-8 byte sequence may represent text intended for another encoding, zero-byte parity is only a UTF-16 review signal, and replacement characters may already have existed in the source. Artifact evidence hashes canonical UTF-8 source text and exact exported bytes, clearly labels both, and includes filenames and transformation details that may themselves be sensitive.
|
||||
- Imports and exports are capped at 5 MiB each.
|
||||
- Large objects are flattened and rendered with per-session bounds.
|
||||
|
||||
Suggested translations are purely local matching; they do not call external services.
|
||||
Transfers to the Portal are signed local handoffs only and never stored by a remote
|
||||
processor by default.
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="Text Tools"><path fill="#5b4ec4" d="M11 4h30l12 12v44H11z"/><path fill="#fff" opacity=".28" d="M41 4v14h12"/><text x="32" y="43" fill="#fff" font-family="system-ui,sans-serif" font-size="18" font-weight="800" text-anchor="middle">TE</text></svg>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="Translator Tools"><path fill="#5b4ec4" d="M11 4h30l12 12v44H11z"/><path fill="#fff" opacity=".28" d="M41 4v14h12"/><text x="32" y="43" fill="#fff" font-family="system-ui,sans-serif" font-size="18" font-weight="800" text-anchor="middle">TR</text></svg>
|
||||
|
||||
|
Before Width: | Height: | Size: 329 B After Width: | Height: | Size: 335 B |
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "Text Tools",
|
||||
"short_name": "Text",
|
||||
"description": "Transform and inspect plain text locally in the browser.",
|
||||
"name": "Translator Tools",
|
||||
"short_name": "Translator",
|
||||
"description": "Review and edit locale strings locally in the browser.",
|
||||
"start_url": "./",
|
||||
"scope": "./",
|
||||
"display": "standalone",
|
||||
|
||||
+2
-2
@@ -1,5 +1,5 @@
|
||||
const CACHE_PREFIX = "text-tools-shell-";
|
||||
const CACHE_NAME = CACHE_PREFIX + "0.2.0";
|
||||
const CACHE_PREFIX = "translator-tools-shell-";
|
||||
const CACHE_NAME = CACHE_PREFIX + "0.1.0";
|
||||
const CORE = ["./", "./manifest.webmanifest", "./favicon.svg"];
|
||||
self.addEventListener("install", (event) => {
|
||||
event.waitUntil(
|
||||
|
||||
+23
-26
@@ -1,22 +1,21 @@
|
||||
{
|
||||
"$schema": "https://git.add-ideas.de/lotobo/toolbox-sdk/raw/branch/main/schemas/toolbox-app.v1.schema.json",
|
||||
"schemaVersion": 1,
|
||||
"id": "de.add-ideas.text-tools",
|
||||
"name": "Text Tools",
|
||||
"version": "0.2.0",
|
||||
"description": "Compose text transforms and export encoding evidence locally.",
|
||||
"id": "de.add-ideas.translator-tools",
|
||||
"name": "Translator Tools",
|
||||
"version": "0.1.0",
|
||||
"description": "Review and edit translation bundles in the browser with glossary suggestions.",
|
||||
"entry": "./",
|
||||
"icon": "./favicon.svg",
|
||||
"categories": ["text", "developer", "productivity"],
|
||||
"categories": ["text", "developer", "productivity", "i18n"],
|
||||
"tags": [
|
||||
"text",
|
||||
"unicode",
|
||||
"pipeline",
|
||||
"normalize",
|
||||
"sort",
|
||||
"escape",
|
||||
"encoding",
|
||||
"newline"
|
||||
"translation",
|
||||
"i18n",
|
||||
"locale",
|
||||
"glossary",
|
||||
"review",
|
||||
"json",
|
||||
"localization"
|
||||
],
|
||||
"integration": {
|
||||
"contextVersion": 1,
|
||||
@@ -26,32 +25,30 @@
|
||||
"requirements": {
|
||||
"secureContext": false,
|
||||
"workers": false,
|
||||
"indexedDb": false,
|
||||
"indexedDb": true,
|
||||
"crossOriginIsolated": false,
|
||||
"topLevelContext": false
|
||||
},
|
||||
"io": {
|
||||
"accepts": [
|
||||
{
|
||||
"mediaType": "text/*",
|
||||
"extensions": [".txt", ".csv", ".md", ".log"],
|
||||
"label": "Bounded text files"
|
||||
"mediaType": "application/json",
|
||||
"extensions": [".json", ".js"],
|
||||
"label": "Locale JSON or i18next-like translation files"
|
||||
}
|
||||
],
|
||||
"produces": [
|
||||
{
|
||||
"mediaType": "text/plain",
|
||||
"extensions": [".txt"],
|
||||
"label": "Transformed text"
|
||||
},
|
||||
{
|
||||
"mediaType": "application/json",
|
||||
"extensions": [".json"],
|
||||
"label": "Recipe and artifact evidence"
|
||||
"label": "Reviewed translation bundle"
|
||||
}
|
||||
]
|
||||
},
|
||||
"capabilities": { "required": [], "optional": ["web-crypto"] },
|
||||
"capabilities": {
|
||||
"required": [],
|
||||
"optional": ["indexeddb", "local-artifact-handoff"]
|
||||
},
|
||||
"privacy": {
|
||||
"processing": "local",
|
||||
"fileUploads": true,
|
||||
@@ -59,14 +56,14 @@
|
||||
"label": "Inputs stay in this browser; nothing is uploaded."
|
||||
},
|
||||
"source": {
|
||||
"repository": "https://git.add-ideas.de/lotobo/text-tools",
|
||||
"repository": "https://git.add-ideas.de/lotobo/translator-tools",
|
||||
"license": "GPL-3.0-or-later"
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"id": "source",
|
||||
"label": "Source",
|
||||
"url": "https://git.add-ideas.de/lotobo/text-tools"
|
||||
"url": "https://git.add-ideas.de/lotobo/translator-tools"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
+24
-3
@@ -1,10 +1,14 @@
|
||||
import { lazy, Suspense, useState } from "react";
|
||||
import { lazy, Suspense, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { AppShell } from "@add-ideas/toolbox-shell-react";
|
||||
import "@add-ideas/toolbox-shell-react/styles.css";
|
||||
import "./styles.css";
|
||||
import { ErrorBoundary } from "./components/ErrorBoundary";
|
||||
import { HelpDialog } from "./components/HelpDialog";
|
||||
import { manifest } from "./toolbox/manifest";
|
||||
import {
|
||||
receiveTranslatorTransfer,
|
||||
type IncomingTranslatorTransfer,
|
||||
} from "./toolbox/transfer";
|
||||
|
||||
const Workbench = lazy(async () => ({
|
||||
default: (await import("./components/Workbench")).Workbench,
|
||||
@@ -12,6 +16,23 @@ const Workbench = lazy(async () => ({
|
||||
|
||||
export function App() {
|
||||
const [helpOpen, setHelpOpen] = useState(false);
|
||||
const [incoming, setIncoming] = useState<IncomingTranslatorTransfer>();
|
||||
const incomingStarted = useRef(false);
|
||||
const activeSourceLabel = useMemo(() => "Translator Tools", []);
|
||||
|
||||
useEffect(() => {
|
||||
if (incomingStarted.current) return;
|
||||
incomingStarted.current = true;
|
||||
void receiveTranslatorTransfer()
|
||||
.then((value) => setIncoming(value))
|
||||
.catch((reason: unknown) =>
|
||||
setIncoming({
|
||||
status: "missing",
|
||||
error: reason instanceof Error ? reason.message : String(reason),
|
||||
}),
|
||||
);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<ErrorBoundary>
|
||||
<AppShell
|
||||
@@ -22,11 +43,11 @@ export function App() {
|
||||
<Suspense
|
||||
fallback={
|
||||
<p className="loading" role="status">
|
||||
Preparing Text Tools…
|
||||
Preparing {activeSourceLabel}…
|
||||
</p>
|
||||
}
|
||||
>
|
||||
<Workbench />
|
||||
<Workbench incoming={incoming} />
|
||||
</Suspense>
|
||||
</AppShell>
|
||||
<HelpDialog open={helpOpen} onClose={() => setHelpOpen(false)} />
|
||||
|
||||
@@ -15,7 +15,7 @@ export class ErrorBoundary extends Component<
|
||||
if (this.state.error)
|
||||
return (
|
||||
<main className="fatal">
|
||||
<h1>Text Tools could not continue</h1>
|
||||
<h1>Translator Tools could not continue</h1>
|
||||
<p>{this.state.error.message}</p>
|
||||
<button type="button" onClick={() => location.reload()}>
|
||||
Reload
|
||||
|
||||
@@ -25,16 +25,20 @@ export function HelpDialog({
|
||||
<div className="dialog-heading">
|
||||
<div>
|
||||
<p className="eyebrow">Local-first help</p>
|
||||
<h2 id="help-title">About Text Tools</h2>
|
||||
<h2 id="help-title">About Translator Tools</h2>
|
||||
</div>
|
||||
<button type="button" onClick={onClose} aria-label="Close help">
|
||||
×
|
||||
</button>
|
||||
</div>
|
||||
<p>Transform and inspect plain text locally in the browser.</p>
|
||||
<p>
|
||||
All processing is performed in this browser. Imported data is treated as
|
||||
untrusted and bounded before parsing.
|
||||
Review and translate i18next-style locale JSON files directly in your
|
||||
browser. No network upload is required.
|
||||
</p>
|
||||
<p>
|
||||
Imported data is parsed with bounded input checks and can include
|
||||
JavaScript-style objects (single quotes, trailing commas). Glossary
|
||||
terms are saved to local storage in this browser only.
|
||||
</p>
|
||||
</dialog>
|
||||
);
|
||||
|
||||
+975
-695
File diff suppressed because it is too large
Load Diff
+247
-246
@@ -1,26 +1,28 @@
|
||||
:root {
|
||||
--toolbox-background: #f6f7fb;
|
||||
--toolbox-background: #f4f7fc;
|
||||
--toolbox-surface: #fff;
|
||||
--toolbox-surface-soft: #eff1f7;
|
||||
--toolbox-text: #202332;
|
||||
--toolbox-muted: #656b7d;
|
||||
--toolbox-border: #d9dce7;
|
||||
--toolbox-accent: #5b4ec4;
|
||||
--toolbox-accent-hover: #493caf;
|
||||
--toolbox-accent-soft: #ece9ff;
|
||||
--toolbox-text: #1f2433;
|
||||
--toolbox-muted: #62697c;
|
||||
--toolbox-border: #d9deea;
|
||||
--toolbox-accent: #4f46e5;
|
||||
--toolbox-accent-hover: #4437d6;
|
||||
--toolbox-accent-soft: #eceaff;
|
||||
--toolbox-accent-contrast: #fff;
|
||||
--toolbox-focus: #137d75;
|
||||
--toolbox-focus: #107d74;
|
||||
--toolbox-danger: #b42342;
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
html {
|
||||
min-width: 20rem;
|
||||
min-height: 100%;
|
||||
background: var(--toolbox-background);
|
||||
scrollbar-gutter: stable;
|
||||
}
|
||||
|
||||
body {
|
||||
min-width: 20rem;
|
||||
min-height: 100vh;
|
||||
@@ -29,15 +31,17 @@ body {
|
||||
color: var(--toolbox-text);
|
||||
font-family: Inter, ui-sans-serif, system-ui, sans-serif;
|
||||
}
|
||||
|
||||
button,
|
||||
input,
|
||||
select,
|
||||
textarea {
|
||||
font: inherit;
|
||||
}
|
||||
|
||||
button,
|
||||
.button {
|
||||
min-height: 2.55rem;
|
||||
min-height: 2.45rem;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
@@ -50,40 +54,70 @@ button,
|
||||
font-weight: 720;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
button:hover:not(:disabled),
|
||||
.button:hover {
|
||||
border-color: var(--toolbox-accent);
|
||||
background: var(--toolbox-surface-soft);
|
||||
}
|
||||
|
||||
button:disabled {
|
||||
opacity: 0.62;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.secondary {
|
||||
background: #f7f7ff;
|
||||
}
|
||||
|
||||
button.primary {
|
||||
border-color: var(--toolbox-accent);
|
||||
background: var(--toolbox-accent);
|
||||
color: var(--toolbox-accent-contrast);
|
||||
}
|
||||
|
||||
button.primary:hover {
|
||||
background: var(--toolbox-accent-hover);
|
||||
}
|
||||
|
||||
:where(button, input, select, textarea, a):focus-visible {
|
||||
outline: 3px solid color-mix(in srgb, var(--toolbox-focus) 42%, transparent);
|
||||
outline-offset: 2px;
|
||||
}
|
||||
|
||||
input,
|
||||
select,
|
||||
textarea {
|
||||
width: 100%;
|
||||
min-height: 2.55rem;
|
||||
min-height: 2.45rem;
|
||||
padding: 0.58rem 0.7rem;
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.62rem;
|
||||
background: var(--toolbox-surface);
|
||||
color: var(--toolbox-text);
|
||||
}
|
||||
|
||||
textarea {
|
||||
min-height: 10rem;
|
||||
min-height: 9rem;
|
||||
resize: vertical;
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, monospace;
|
||||
line-height: 1.48;
|
||||
line-height: 1.4;
|
||||
}
|
||||
|
||||
a {
|
||||
color: inherit;
|
||||
}
|
||||
|
||||
.toolbox-shell__main {
|
||||
width: min(100%, 90rem);
|
||||
padding: clamp(0.75rem, 1.8vw, 1.5rem);
|
||||
}
|
||||
|
||||
.workbench {
|
||||
display: grid;
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.hero,
|
||||
.panel {
|
||||
border: 1px solid var(--toolbox-border);
|
||||
@@ -91,296 +125,263 @@ textarea {
|
||||
background: var(--toolbox-surface);
|
||||
box-shadow: 0 8px 28px rgb(30 36 70 / 4%);
|
||||
}
|
||||
|
||||
.hero {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 1rem;
|
||||
align-items: flex-start;
|
||||
padding: clamp(1.1rem, 3vw, 2rem);
|
||||
padding: clamp(1rem, 3vw, 1.5rem);
|
||||
}
|
||||
|
||||
.hero h1,
|
||||
.panel h2,
|
||||
.panel h3,
|
||||
.help-dialog h2,
|
||||
.fatal h1 {
|
||||
.help-dialog h2 {
|
||||
margin: 0;
|
||||
letter-spacing: -0.025em;
|
||||
}
|
||||
|
||||
.hero p:not(.eyebrow) {
|
||||
max-width: 52rem;
|
||||
margin: 0.55rem 0 0;
|
||||
color: var(--toolbox-muted);
|
||||
line-height: 1.55;
|
||||
}
|
||||
.eyebrow {
|
||||
margin: 0 0 0.3rem;
|
||||
color: var(--toolbox-accent);
|
||||
font-size: 0.69rem;
|
||||
font-weight: 820;
|
||||
letter-spacing: 0.115em;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.privacy-pill {
|
||||
flex: 0 0 auto;
|
||||
padding: 0.38rem 0.62rem;
|
||||
padding: 0.4rem 0.7rem;
|
||||
border-radius: 999px;
|
||||
font-size: 0.84rem;
|
||||
border: 1px solid var(--toolbox-border);
|
||||
background: var(--toolbox-accent-soft);
|
||||
color: var(--toolbox-accent);
|
||||
font-size: 0.75rem;
|
||||
font-weight: 760;
|
||||
color: #2f2a6d;
|
||||
align-self: flex-start;
|
||||
}
|
||||
|
||||
.panel {
|
||||
padding: 1rem;
|
||||
}
|
||||
|
||||
.panel-heading {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 1rem;
|
||||
align-items: end;
|
||||
margin-bottom: 0.9rem;
|
||||
align-items: flex-start;
|
||||
margin-bottom: 0.85rem;
|
||||
}
|
||||
.capability-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(min(100%, 13rem), 1fr));
|
||||
gap: 0.75rem;
|
||||
|
||||
.panel-heading h2 {
|
||||
margin: 0.16rem 0 0;
|
||||
}
|
||||
.capability-grid article {
|
||||
padding: 0.9rem;
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.72rem;
|
||||
background: var(--toolbox-surface-soft);
|
||||
|
||||
.eyebrow {
|
||||
margin: 0;
|
||||
color: var(--toolbox-accent);
|
||||
font-size: 0.67rem;
|
||||
font-weight: 820;
|
||||
letter-spacing: 0.115em;
|
||||
}
|
||||
.capability-grid p {
|
||||
margin: 0.4rem 0 0;
|
||||
color: var(--toolbox-muted);
|
||||
line-height: 1.48;
|
||||
}
|
||||
.workspace-tabs {
|
||||
|
||||
.actions {
|
||||
display: flex;
|
||||
gap: 0.4rem;
|
||||
overflow-x: auto;
|
||||
padding-bottom: 0.2rem;
|
||||
}
|
||||
.workspace-tabs button[aria-selected="true"] {
|
||||
border-color: var(--toolbox-accent);
|
||||
background: var(--toolbox-accent);
|
||||
color: var(--toolbox-accent-contrast);
|
||||
}
|
||||
.grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(min(100%, 20rem), 1fr));
|
||||
gap: 0.85rem;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.field {
|
||||
display: grid;
|
||||
gap: 0.35rem;
|
||||
}
|
||||
.field > span {
|
||||
font-size: 0.76rem;
|
||||
font-weight: 750;
|
||||
}
|
||||
.muted {
|
||||
color: var(--toolbox-muted);
|
||||
}
|
||||
.result {
|
||||
padding: 0.8rem;
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.68rem;
|
||||
background: var(--toolbox-surface-soft);
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
.loading,
|
||||
.fatal {
|
||||
width: min(100% - 2rem, 60rem);
|
||||
margin: 2rem auto;
|
||||
padding: 1rem;
|
||||
}
|
||||
.help-dialog {
|
||||
width: min(36rem, calc(100% - 2rem));
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.9rem;
|
||||
background: var(--toolbox-surface);
|
||||
color: var(--toolbox-text);
|
||||
}
|
||||
.help-dialog::backdrop {
|
||||
background: rgb(20 24 45 / 55%);
|
||||
}
|
||||
.dialog-heading {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 1rem;
|
||||
align-items: start;
|
||||
}
|
||||
.workspace {
|
||||
display: grid;
|
||||
gap: 0.85rem;
|
||||
}
|
||||
.editor-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 1rem;
|
||||
}
|
||||
.editor-grid textarea {
|
||||
min-height: 22rem;
|
||||
}
|
||||
.encoding-row {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(10rem, 16rem) minmax(12rem, 1fr);
|
||||
gap: 0.7rem;
|
||||
align-items: end;
|
||||
}
|
||||
.actions {
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.primary {
|
||||
border-color: var(--toolbox-accent);
|
||||
background: var(--toolbox-accent);
|
||||
color: var(--toolbox-accent-contrast);
|
||||
}
|
||||
.file-button {
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.file-button input {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
opacity: 0;
|
||||
cursor: pointer;
|
||||
}
|
||||
.check {
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
|
||||
.field-inline {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
min-height: 2.55rem;
|
||||
}
|
||||
.check input {
|
||||
width: auto;
|
||||
min-height: auto;
|
||||
}
|
||||
.inventory {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(7rem, 1fr));
|
||||
gap: 0.45rem;
|
||||
margin: 0;
|
||||
}
|
||||
.inventory div {
|
||||
padding: 0.55rem;
|
||||
border-radius: 0.55rem;
|
||||
background: var(--toolbox-surface-soft);
|
||||
}
|
||||
.inventory dt {
|
||||
color: var(--toolbox-muted);
|
||||
font-size: 0.66rem;
|
||||
font-weight: 750;
|
||||
}
|
||||
.inventory dd {
|
||||
margin: 0.15rem 0 0;
|
||||
}
|
||||
.steps {
|
||||
display: grid;
|
||||
gap: 0.55rem;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
list-style: none;
|
||||
}
|
||||
.steps li {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(12rem, 0.8fr) minmax(12rem, 1.4fr) auto;
|
||||
gap: 0.6rem;
|
||||
align-items: center;
|
||||
padding: 0.6rem;
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.65rem;
|
||||
}
|
||||
.step-actions {
|
||||
display: flex;
|
||||
gap: 0.25rem;
|
||||
}
|
||||
.column-options {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(8rem, 1fr) minmax(4rem, 0.55fr) minmax(
|
||||
7rem,
|
||||
1fr
|
||||
);
|
||||
gap: 0.4rem;
|
||||
}
|
||||
.step-actions button {
|
||||
min-width: 2.55rem;
|
||||
padding: 0.4rem;
|
||||
|
||||
.field > span {
|
||||
font-size: 0.92rem;
|
||||
color: var(--toolbox-muted);
|
||||
}
|
||||
.warning,
|
||||
.error,
|
||||
.notice {
|
||||
margin: 0;
|
||||
padding: 0.7rem;
|
||||
|
||||
.form-grid {
|
||||
display: grid;
|
||||
gap: 0.8rem;
|
||||
grid-template-columns: repeat(auto-fit, minmax(12rem, 1fr));
|
||||
}
|
||||
|
||||
.field-group {
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.field-group input {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.table-wrap {
|
||||
overflow-x: auto;
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.65rem;
|
||||
line-height: 1.5;
|
||||
border-radius: 0.8rem;
|
||||
}
|
||||
.warning {
|
||||
border-color: #d9a72e;
|
||||
background: #fff8df;
|
||||
color: #725000;
|
||||
}
|
||||
.error {
|
||||
border-color: var(--toolbox-danger);
|
||||
color: var(--toolbox-danger);
|
||||
}
|
||||
.notice {
|
||||
background: var(--toolbox-surface-soft);
|
||||
}
|
||||
table {
|
||||
|
||||
.review-grid {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
}
|
||||
th,
|
||||
td {
|
||||
padding: 0.55rem;
|
||||
|
||||
.review-grid th,
|
||||
.review-grid td {
|
||||
border-bottom: 1px solid var(--toolbox-border);
|
||||
text-align: left;
|
||||
border-right: 1px solid var(--toolbox-border);
|
||||
padding: 0.55rem;
|
||||
vertical-align: top;
|
||||
}
|
||||
th {
|
||||
|
||||
.review-grid th:last-child,
|
||||
.review-grid td:last-child {
|
||||
border-right: none;
|
||||
}
|
||||
|
||||
.review-grid thead {
|
||||
background: #f9fbff;
|
||||
}
|
||||
|
||||
.review-grid textarea {
|
||||
min-height: 6rem;
|
||||
}
|
||||
|
||||
.key-cell {
|
||||
width: 24ch;
|
||||
max-width: 24ch;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
.count {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
color: var(--toolbox-muted);
|
||||
font-size: 0.7rem;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
details {
|
||||
padding: 0.4rem 0.5rem;
|
||||
border-radius: 0.5rem;
|
||||
background: #f8f9fc;
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.65rem;
|
||||
}
|
||||
summary {
|
||||
padding: 0.65rem;
|
||||
cursor: pointer;
|
||||
font-weight: 750;
|
||||
|
||||
.muted {
|
||||
color: var(--toolbox-muted);
|
||||
}
|
||||
.recipe {
|
||||
|
||||
.row-actions {
|
||||
margin-top: 0.35rem;
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.35rem;
|
||||
}
|
||||
|
||||
.row-actions .chip,
|
||||
.chip {
|
||||
border-radius: 999px;
|
||||
min-height: auto;
|
||||
padding: 0.35rem 0.56rem;
|
||||
}
|
||||
|
||||
.chip-list {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.35rem;
|
||||
}
|
||||
|
||||
.notice {
|
||||
margin: 0.75rem 0 0;
|
||||
color: #1f5a75;
|
||||
}
|
||||
|
||||
.error {
|
||||
margin: 0.75rem 0 0;
|
||||
color: var(--toolbox-danger);
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.help-dialog {
|
||||
margin: auto;
|
||||
width: min(50rem, 95vw);
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 1rem;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.help-dialog .dialog-heading {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 1rem;
|
||||
border-bottom: 1px solid var(--toolbox-border);
|
||||
}
|
||||
|
||||
.help-dialog p {
|
||||
margin: 0.7rem 1rem;
|
||||
}
|
||||
|
||||
.glossary,
|
||||
.language-list,
|
||||
.term-list,
|
||||
.workspace ul {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.glossary-form {
|
||||
display: grid;
|
||||
gap: 0.6rem;
|
||||
padding: 0 0.65rem 0.65rem;
|
||||
gap: 0.55rem;
|
||||
margin: 0.5rem 0 0;
|
||||
grid-template-columns: repeat(auto-fit, minmax(12rem, 1fr));
|
||||
}
|
||||
@media (max-width: 62rem) {
|
||||
.editor-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.term-list,
|
||||
.language-list {
|
||||
margin-top: 0.65rem;
|
||||
display: grid;
|
||||
gap: 0.45rem;
|
||||
}
|
||||
@media (max-width: 48rem) {
|
||||
.steps li {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
.column-options {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.term-list li,
|
||||
.language-list li {
|
||||
border: 1px solid var(--toolbox-border);
|
||||
border-radius: 0.6rem;
|
||||
background: #fbfcff;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 0.75rem;
|
||||
align-items: center;
|
||||
padding: 0.55rem;
|
||||
}
|
||||
@media (max-width: 42rem) {
|
||||
.hero {
|
||||
|
||||
.term-list p {
|
||||
margin: 0.15rem 0 0;
|
||||
}
|
||||
|
||||
.sr-only {
|
||||
position: absolute;
|
||||
width: 1px;
|
||||
height: 1px;
|
||||
padding: 0;
|
||||
margin: -1px;
|
||||
overflow: hidden;
|
||||
clip: rect(0, 0, 0, 0);
|
||||
white-space: nowrap;
|
||||
border-width: 0;
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
.panel-heading {
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
}
|
||||
.privacy-pill {
|
||||
order: -1;
|
||||
}
|
||||
.encoding-row {
|
||||
grid-template-columns: minmax(0, 1fr);
|
||||
|
||||
.form-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,251 +0,0 @@
|
||||
import {
|
||||
decodeText,
|
||||
digestHex,
|
||||
encodeText,
|
||||
type TextEncoding,
|
||||
} from "@add-ideas/toolbox-helpers";
|
||||
|
||||
import {
|
||||
textInventory,
|
||||
type PipelineResult,
|
||||
type StepReport,
|
||||
type TransformStep,
|
||||
} from "./pipeline";
|
||||
|
||||
export interface TextByteEvidence {
|
||||
readonly schemaVersion: 1;
|
||||
readonly byteLength: number;
|
||||
readonly selectedEncoding: TextEncoding;
|
||||
readonly fatalDecode: boolean;
|
||||
readonly bom?: { encoding: Exclude<TextEncoding, "latin1">; bytes: number };
|
||||
readonly utf8: { valid: boolean; firstInvalidOffset?: number };
|
||||
readonly zeroBytes: {
|
||||
total: number;
|
||||
evenOffsets: number;
|
||||
oddOffsets: number;
|
||||
};
|
||||
readonly byteNewlines: { crlf: number; bareLf: number; bareCr: number };
|
||||
readonly replacementCharacters: number;
|
||||
readonly warnings: readonly string[];
|
||||
}
|
||||
|
||||
export interface TextArtifactEvidence {
|
||||
readonly schemaVersion: 1;
|
||||
readonly artifactType: "de.add-ideas.toolbox.text/v1";
|
||||
readonly createdBy: { app: "text-tools"; version: "0.2.0" };
|
||||
readonly source: {
|
||||
name: string;
|
||||
canonicalUtf8Sha256: string;
|
||||
inventory: ReturnType<typeof textInventory>;
|
||||
byteEvidence?: TextByteEvidence;
|
||||
};
|
||||
readonly output: {
|
||||
name: string;
|
||||
mediaType: "text/plain";
|
||||
encoding: TextEncoding;
|
||||
bytes: number;
|
||||
sha256: string;
|
||||
inventory: ReturnType<typeof textInventory>;
|
||||
};
|
||||
readonly pipeline: {
|
||||
steps: readonly TransformStep[];
|
||||
reports: readonly StepReport[];
|
||||
warnings: readonly string[];
|
||||
};
|
||||
readonly handoff: {
|
||||
supportedByThisBuild: false;
|
||||
note: string;
|
||||
};
|
||||
}
|
||||
|
||||
function bom(bytes: Uint8Array): TextByteEvidence["bom"] {
|
||||
if (bytes[0] === 0xef && bytes[1] === 0xbb && bytes[2] === 0xbf)
|
||||
return { encoding: "utf-8", bytes: 3 };
|
||||
if (bytes[0] === 0xff && bytes[1] === 0xfe)
|
||||
return { encoding: "utf-16le", bytes: 2 };
|
||||
if (bytes[0] === 0xfe && bytes[1] === 0xff)
|
||||
return { encoding: "utf-16be", bytes: 2 };
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function utf8InvalidOffset(bytes: Uint8Array): number | undefined {
|
||||
const continuation = (index: number) =>
|
||||
index < bytes.length && (bytes[index]! & 0xc0) === 0x80;
|
||||
for (let index = 0; index < bytes.length; index += 1) {
|
||||
const first = bytes[index]!;
|
||||
if (first <= 0x7f) continue;
|
||||
if (first >= 0xc2 && first <= 0xdf) {
|
||||
if (!continuation(index + 1)) return index;
|
||||
index += 1;
|
||||
continue;
|
||||
}
|
||||
if (first >= 0xe0 && first <= 0xef) {
|
||||
const second = bytes[index + 1];
|
||||
if (
|
||||
second === undefined ||
|
||||
(first === 0xe0 && (second < 0xa0 || second > 0xbf)) ||
|
||||
(first === 0xed && (second < 0x80 || second > 0x9f)) ||
|
||||
(first !== 0xe0 &&
|
||||
first !== 0xed &&
|
||||
(second < 0x80 || second > 0xbf)) ||
|
||||
!continuation(index + 2)
|
||||
)
|
||||
return index;
|
||||
index += 2;
|
||||
continue;
|
||||
}
|
||||
if (first >= 0xf0 && first <= 0xf4) {
|
||||
const second = bytes[index + 1];
|
||||
if (
|
||||
second === undefined ||
|
||||
(first === 0xf0 && (second < 0x90 || second > 0xbf)) ||
|
||||
(first === 0xf4 && (second < 0x80 || second > 0x8f)) ||
|
||||
(first !== 0xf0 &&
|
||||
first !== 0xf4 &&
|
||||
(second < 0x80 || second > 0xbf)) ||
|
||||
!continuation(index + 2) ||
|
||||
!continuation(index + 3)
|
||||
)
|
||||
return index;
|
||||
index += 3;
|
||||
continue;
|
||||
}
|
||||
return index;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function inspectByteNewlines(bytes: Uint8Array) {
|
||||
let crlf = 0;
|
||||
let bareLf = 0;
|
||||
let bareCr = 0;
|
||||
for (let index = 0; index < bytes.length; index += 1) {
|
||||
if (bytes[index] === 0x0d && bytes[index + 1] === 0x0a) {
|
||||
crlf += 1;
|
||||
index += 1;
|
||||
} else if (bytes[index] === 0x0a) bareLf += 1;
|
||||
else if (bytes[index] === 0x0d) bareCr += 1;
|
||||
}
|
||||
return { crlf, bareLf, bareCr };
|
||||
}
|
||||
|
||||
export function decodeTextWithEvidence(
|
||||
bytes: Uint8Array,
|
||||
selectedEncoding: TextEncoding,
|
||||
fatalDecode: boolean,
|
||||
): { text: string; evidence: TextByteEvidence } {
|
||||
if (bytes.byteLength > 16 * 1024 * 1024)
|
||||
throw new RangeError("Text byte evidence is limited to 16 MiB.");
|
||||
const detectedBom = bom(bytes);
|
||||
const invalidOffset = utf8InvalidOffset(bytes);
|
||||
let zeroTotal = 0;
|
||||
let zeroEven = 0;
|
||||
let zeroOdd = 0;
|
||||
bytes.forEach((value, index) => {
|
||||
if (value !== 0) return;
|
||||
zeroTotal += 1;
|
||||
if (index % 2) zeroOdd += 1;
|
||||
else zeroEven += 1;
|
||||
});
|
||||
const text = decodeText(bytes, selectedEncoding, fatalDecode);
|
||||
const warnings: string[] = [];
|
||||
if (detectedBom && detectedBom.encoding !== selectedEncoding)
|
||||
warnings.push(
|
||||
`The ${detectedBom.encoding.toUpperCase()} BOM conflicts with the selected ${selectedEncoding.toUpperCase()} decoder.`,
|
||||
);
|
||||
if (!detectedBom)
|
||||
warnings.push(
|
||||
"No byte-order mark is present; the selected encoding is an explicit user choice, not a detection claim.",
|
||||
);
|
||||
if (selectedEncoding.startsWith("utf-16") && bytes.length % 2)
|
||||
warnings.push("UTF-16 input has an odd trailing byte.");
|
||||
if (selectedEncoding !== "utf-8" && invalidOffset === undefined)
|
||||
warnings.push(
|
||||
"The same bytes are also well-formed UTF-8; encoding intent cannot be inferred from validity alone.",
|
||||
);
|
||||
const replacementCharacters = [...text].filter(
|
||||
(character) => character === "\uFFFD",
|
||||
).length;
|
||||
if (replacementCharacters)
|
||||
warnings.push(
|
||||
`${replacementCharacters} replacement character(s) appear in decoded text; they may be source data or decoder substitutions.`,
|
||||
);
|
||||
return {
|
||||
text,
|
||||
evidence: Object.freeze({
|
||||
schemaVersion: 1,
|
||||
byteLength: bytes.byteLength,
|
||||
selectedEncoding,
|
||||
fatalDecode,
|
||||
...(detectedBom ? { bom: detectedBom } : {}),
|
||||
utf8: {
|
||||
valid: invalidOffset === undefined,
|
||||
...(invalidOffset === undefined
|
||||
? {}
|
||||
: { firstInvalidOffset: invalidOffset }),
|
||||
},
|
||||
zeroBytes: {
|
||||
total: zeroTotal,
|
||||
evenOffsets: zeroEven,
|
||||
oddOffsets: zeroOdd,
|
||||
},
|
||||
byteNewlines: inspectByteNewlines(bytes),
|
||||
replacementCharacters,
|
||||
warnings: Object.freeze(warnings),
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
export async function createTextArtifactEvidence(input: {
|
||||
sourceName: string;
|
||||
sourceEvidence?: TextByteEvidence;
|
||||
sourceText: string;
|
||||
outputName: string;
|
||||
outputEncoding: TextEncoding;
|
||||
outputBytes: Uint8Array;
|
||||
pipeline: PipelineResult;
|
||||
steps: readonly TransformStep[];
|
||||
}): Promise<TextArtifactEvidence> {
|
||||
if (input.outputBytes.byteLength > 32 * 1024 * 1024)
|
||||
throw new RangeError("Artifact output exceeds the 32 MiB evidence limit.");
|
||||
const canonicalSource = encodeText(input.sourceText, "utf-8");
|
||||
return Object.freeze({
|
||||
schemaVersion: 1,
|
||||
artifactType: "de.add-ideas.toolbox.text/v1",
|
||||
createdBy: {
|
||||
app: "text-tools" as const,
|
||||
version: "0.2.0" as const,
|
||||
},
|
||||
source: {
|
||||
name: input.sourceName,
|
||||
canonicalUtf8Sha256: await digestHex(
|
||||
canonicalSource,
|
||||
"SHA-256",
|
||||
32 * 1024 * 1024,
|
||||
),
|
||||
inventory: textInventory(input.sourceText),
|
||||
...(input.sourceEvidence ? { byteEvidence: input.sourceEvidence } : {}),
|
||||
},
|
||||
output: {
|
||||
name: input.outputName,
|
||||
mediaType: "text/plain" as const,
|
||||
encoding: input.outputEncoding,
|
||||
bytes: input.outputBytes.byteLength,
|
||||
sha256: await digestHex(input.outputBytes, "SHA-256", 32 * 1024 * 1024),
|
||||
inventory: textInventory(input.pipeline.output),
|
||||
},
|
||||
pipeline: {
|
||||
steps: Object.freeze(
|
||||
input.steps.map((step) => Object.freeze({ ...step })),
|
||||
),
|
||||
reports: Object.freeze(
|
||||
input.pipeline.steps.map((report) => Object.freeze({ ...report })),
|
||||
),
|
||||
warnings: Object.freeze([...input.pipeline.warnings]),
|
||||
},
|
||||
handoff: {
|
||||
supportedByThisBuild: false as const,
|
||||
note: "The portable files are ready for explicit local transfer. SDK 0.3.0 provides the shared contract, but Open With remains disabled until the coordinated Portal consumer rollout.",
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,515 +0,0 @@
|
||||
import {
|
||||
base64ToBytes,
|
||||
bytesToBase64,
|
||||
bytesToHex,
|
||||
convertLineEndings,
|
||||
decodeText,
|
||||
encodeText,
|
||||
hexToBytes,
|
||||
normalizeUnicode,
|
||||
parseCsv,
|
||||
stringifyCsv,
|
||||
transformCase,
|
||||
type CaseTransform,
|
||||
type LineEnding,
|
||||
} from "@add-ideas/toolbox-helpers";
|
||||
|
||||
export type StepType =
|
||||
| "line-endings"
|
||||
| "trim-lines"
|
||||
| "trim-document"
|
||||
| "collapse-whitespace"
|
||||
| "sort-lines"
|
||||
| "dedupe-lines"
|
||||
| "case"
|
||||
| "normalize"
|
||||
| "transliterate"
|
||||
| "escape"
|
||||
| "unescape"
|
||||
| "wrap"
|
||||
| "columns"
|
||||
| "replace-literal"
|
||||
| "prefix-lines"
|
||||
| "suffix-lines"
|
||||
| "filter-lines"
|
||||
| "number-lines"
|
||||
| "join-lines"
|
||||
| "reverse-lines";
|
||||
export interface TransformStep {
|
||||
id: string;
|
||||
type: StepType;
|
||||
enabled: boolean;
|
||||
option: string;
|
||||
}
|
||||
export interface StepReport {
|
||||
id: string;
|
||||
type: StepType;
|
||||
beforeUnits: number;
|
||||
afterUnits: number;
|
||||
changed: boolean;
|
||||
warning?: string;
|
||||
}
|
||||
export interface PipelineResult {
|
||||
output: string;
|
||||
steps: StepReport[];
|
||||
warnings: string[];
|
||||
}
|
||||
|
||||
const MAX_INPUT = 2_000_000;
|
||||
const MAX_OUTPUT = 32 * 1024 * 1024;
|
||||
let nextStepId = 0;
|
||||
|
||||
function lines(value: string): string[] {
|
||||
return value.replaceAll("\r\n", "\n").replaceAll("\r", "\n").split("\n");
|
||||
}
|
||||
function htmlEscape(value: string): string {
|
||||
return value
|
||||
.replaceAll("&", "&")
|
||||
.replaceAll("<", "<")
|
||||
.replaceAll(">", ">")
|
||||
.replaceAll('"', """)
|
||||
.replaceAll("'", "'");
|
||||
}
|
||||
function stableUnique(values: string[]): string[] {
|
||||
const seen = new Set<string>();
|
||||
return values.filter((value) => !seen.has(value) && !!seen.add(value));
|
||||
}
|
||||
|
||||
function wrapText(value: string, width: number): string {
|
||||
if (!Number.isSafeInteger(width) || width < 1 || width > 10_000)
|
||||
throw new Error("Wrap width must be 1–10,000 code points.");
|
||||
return lines(value)
|
||||
.flatMap((line) => {
|
||||
const words = line.split(/\s+/u);
|
||||
const output: string[] = [];
|
||||
let current = "";
|
||||
for (const word of words) {
|
||||
if (!word) continue;
|
||||
if (!current) current = word;
|
||||
else if ([...current, " ", ...word].length <= width)
|
||||
current += ` ${word}`;
|
||||
else {
|
||||
output.push(current);
|
||||
current = word;
|
||||
}
|
||||
}
|
||||
output.push(current);
|
||||
return output;
|
||||
})
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
function transformColumns(
|
||||
value: string,
|
||||
option: string,
|
||||
): {
|
||||
value: string;
|
||||
warning?: string;
|
||||
} {
|
||||
const pieces = option.split("|");
|
||||
const mode = pieces.length >= 3 ? pieces[0] : "literal";
|
||||
const delimiterInput = pieces.length >= 3 ? pieces[1] : pieces[0];
|
||||
const order = pieces.length >= 3 ? pieces.slice(2).join("|") : pieces[1];
|
||||
const delimiter = delimiterInput === "\\t" ? "\t" : delimiterInput || ",";
|
||||
if (!delimiter || delimiter.length > 8)
|
||||
throw new Error("Column delimiter must contain 1–8 characters.");
|
||||
if (mode !== "literal" && mode !== "csv")
|
||||
throw new Error("Column parsing mode must be csv or literal.");
|
||||
if (mode === "csv" && delimiter.length !== 1)
|
||||
throw new Error("Quoted CSV mode requires a one-character delimiter.");
|
||||
const indices = (order || "1")
|
||||
.split(",")
|
||||
.map((entry) => Number(entry.trim()) - 1);
|
||||
if (
|
||||
!indices.length ||
|
||||
indices.some(
|
||||
(index) => !Number.isSafeInteger(index) || index < 0 || index > 999,
|
||||
)
|
||||
)
|
||||
throw new Error("Column order uses 1-based indices such as 3,1,2.");
|
||||
if (mode === "literal")
|
||||
return {
|
||||
value: lines(value)
|
||||
.map((line) => {
|
||||
const cells = line.split(delimiter);
|
||||
return indices.map((index) => cells[index] ?? "").join(delimiter);
|
||||
})
|
||||
.join("\n"),
|
||||
};
|
||||
const rows = parseCsv(value, {
|
||||
delimiter,
|
||||
maxRows: 200_000,
|
||||
maxColumns: 1_000,
|
||||
maxFieldChars: 2_000_000,
|
||||
});
|
||||
return {
|
||||
value: stringifyCsv(
|
||||
rows.map((row) => indices.map((index) => row[index] ?? "")),
|
||||
{
|
||||
delimiter,
|
||||
maxRows: 200_000,
|
||||
maxColumns: 1_000,
|
||||
maxFieldChars: 2_000_000,
|
||||
},
|
||||
),
|
||||
warning:
|
||||
"Quoted CSV rows were parsed across embedded delimiters/newlines and serialized canonically with CRLF row endings.",
|
||||
};
|
||||
}
|
||||
|
||||
function strictHtmlUnescape(value: string): string {
|
||||
const entities: Record<string, string> = {
|
||||
"&": "&",
|
||||
"<": "<",
|
||||
">": ">",
|
||||
""": '"',
|
||||
"'": "'",
|
||||
};
|
||||
let output = "";
|
||||
for (let index = 0; index < value.length; index += 1) {
|
||||
if (value[index] !== "&") {
|
||||
output += value[index];
|
||||
continue;
|
||||
}
|
||||
const end = value.indexOf(";", index + 1);
|
||||
if (end < 0 || end - index > 6)
|
||||
throw new SyntaxError(
|
||||
`HTML entity at UTF-16 index ${index} is malformed.`,
|
||||
);
|
||||
const entity = value.slice(index, end + 1);
|
||||
const decoded = entities[entity];
|
||||
if (decoded === undefined)
|
||||
throw new SyntaxError(
|
||||
`HTML entity ${entity} is not emitted by the matching escape stage.`,
|
||||
);
|
||||
output += decoded;
|
||||
index = end;
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
function strictJsonUnescape(value: string): string {
|
||||
const parsed = JSON.parse(`"${value}"`) as unknown;
|
||||
if (typeof parsed !== "string")
|
||||
throw new SyntaxError("JSON escape input is invalid.");
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function unescapeValue(value: string, option: string): string {
|
||||
if (option === "json") return strictJsonUnescape(value);
|
||||
if (option === "html") return strictHtmlUnescape(value);
|
||||
if (option === "url") return decodeURIComponent(value);
|
||||
if (option === "base64")
|
||||
return decodeText(
|
||||
base64ToBytes(value, { maxOutputBytes: MAX_OUTPUT }),
|
||||
"utf-8",
|
||||
true,
|
||||
);
|
||||
if (option === "hex")
|
||||
return decodeText(
|
||||
hexToBytes(value, {
|
||||
maxOutputBytes: MAX_OUTPUT,
|
||||
allowWhitespace: false,
|
||||
allowPrefix: false,
|
||||
}),
|
||||
"utf-8",
|
||||
true,
|
||||
);
|
||||
throw new Error("Unsupported unescape source.");
|
||||
}
|
||||
|
||||
function transliterate(value: string): string {
|
||||
return value
|
||||
.normalize("NFKD")
|
||||
.replaceAll(/\p{Mark}+/gu, "")
|
||||
.replaceAll("ß", "ss")
|
||||
.replaceAll("Æ", "AE")
|
||||
.replaceAll("æ", "ae")
|
||||
.replaceAll("Ø", "O")
|
||||
.replaceAll("ø", "o")
|
||||
.replaceAll("Ł", "L")
|
||||
.replaceAll("ł", "l");
|
||||
}
|
||||
|
||||
function parseReplacement(option: string): readonly [string, string] {
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(option);
|
||||
} catch {
|
||||
throw new SyntaxError("Literal replacement options are invalid.");
|
||||
}
|
||||
if (
|
||||
!Array.isArray(parsed) ||
|
||||
parsed.length !== 2 ||
|
||||
parsed.some((value) => typeof value !== "string")
|
||||
)
|
||||
throw new SyntaxError(
|
||||
"Literal replacement requires [search, replacement].",
|
||||
);
|
||||
const [search, replacement] = parsed as [string, string];
|
||||
if (!search) throw new SyntaxError("Literal search text must not be empty.");
|
||||
if (search.length > 100_000 || replacement.length > 1_000_000)
|
||||
throw new RangeError(
|
||||
"Literal replacement option exceeds its safety limit.",
|
||||
);
|
||||
return [search, replacement];
|
||||
}
|
||||
|
||||
function decodedDelimiter(option: string): string {
|
||||
if (option.length > 1_000)
|
||||
throw new RangeError("Line join delimiter exceeds 1,000 characters.");
|
||||
return option.replaceAll("\\n", "\n").replaceAll("\\t", "\t");
|
||||
}
|
||||
|
||||
function assertProjectedOutput(units: number): void {
|
||||
if (!Number.isSafeInteger(units) || units > MAX_OUTPUT)
|
||||
throw new RangeError(
|
||||
"Transformation would exceed the 32 MiB output limit.",
|
||||
);
|
||||
}
|
||||
|
||||
function replaceLiteral(
|
||||
value: string,
|
||||
search: string,
|
||||
replacement: string,
|
||||
): string {
|
||||
let matches = 0;
|
||||
let offset = 0;
|
||||
while ((offset = value.indexOf(search, offset)) >= 0) {
|
||||
matches += 1;
|
||||
offset += search.length;
|
||||
assertProjectedOutput(
|
||||
value.length + matches * (replacement.length - search.length),
|
||||
);
|
||||
}
|
||||
return value.replaceAll(search, replacement);
|
||||
}
|
||||
|
||||
function decorateLines(
|
||||
value: string,
|
||||
addition: string,
|
||||
side: "prefix" | "suffix",
|
||||
): string {
|
||||
if (addition.length > 100_000)
|
||||
throw new RangeError("Line decoration exceeds 100,000 UTF-16 units.");
|
||||
const values = lines(value);
|
||||
assertProjectedOutput(value.length + values.length * addition.length);
|
||||
return values
|
||||
.map((line) => (side === "prefix" ? addition + line : line + addition))
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
function applyStep(
|
||||
value: string,
|
||||
step: TransformStep,
|
||||
): { value: string; warning?: string } {
|
||||
switch (step.type) {
|
||||
case "line-endings":
|
||||
return { value: convertLineEndings(value, step.option as LineEnding) };
|
||||
case "trim-lines":
|
||||
return {
|
||||
value: lines(value)
|
||||
.map((line) => line.trim())
|
||||
.join("\n"),
|
||||
};
|
||||
case "trim-document":
|
||||
return { value: value.trim() };
|
||||
case "collapse-whitespace":
|
||||
return {
|
||||
value:
|
||||
step.option === "all"
|
||||
? value.replaceAll(/\s+/gu, " ")
|
||||
: lines(value)
|
||||
.map((line) => line.replaceAll(/[\t ]+/gu, " "))
|
||||
.join("\n"),
|
||||
};
|
||||
case "sort-lines": {
|
||||
const locale = step.option || "en";
|
||||
return {
|
||||
value: lines(value)
|
||||
.map((line, index) => ({ line, index }))
|
||||
.sort(
|
||||
(left, right) =>
|
||||
left.line.localeCompare(right.line, locale, {
|
||||
numeric: true,
|
||||
sensitivity: "variant",
|
||||
}) || left.index - right.index,
|
||||
)
|
||||
.map(({ line }) => line)
|
||||
.join("\n"),
|
||||
warning: `Sort order uses the host Intl.Collator for locale ${locale}.`,
|
||||
};
|
||||
}
|
||||
case "dedupe-lines":
|
||||
return { value: stableUnique(lines(value)).join("\n") };
|
||||
case "case":
|
||||
return { value: transformCase(value, step.option as CaseTransform) };
|
||||
case "normalize":
|
||||
return {
|
||||
value: normalizeUnicode(
|
||||
value,
|
||||
step.option as "NFC" | "NFD" | "NFKC" | "NFKD",
|
||||
),
|
||||
};
|
||||
case "transliterate":
|
||||
return {
|
||||
value: transliterate(value),
|
||||
warning:
|
||||
"Best-effort Latin transliteration is lossy and incomplete; it is not language-aware.",
|
||||
};
|
||||
case "escape": {
|
||||
if (step.option === "json")
|
||||
return { value: JSON.stringify(value).slice(1, -1) };
|
||||
if (step.option === "html") return { value: htmlEscape(value) };
|
||||
if (step.option === "url") return { value: encodeURIComponent(value) };
|
||||
if (step.option === "base64")
|
||||
return { value: bytesToBase64(encodeText(value)) };
|
||||
if (step.option === "hex")
|
||||
return { value: bytesToHex(encodeText(value)) };
|
||||
throw new Error("Unsupported escape target.");
|
||||
}
|
||||
case "unescape":
|
||||
return {
|
||||
value: unescapeValue(value, step.option),
|
||||
warning:
|
||||
step.option === "base64" || step.option === "hex"
|
||||
? "Decoded bytes are required to be well-formed UTF-8 text; arbitrary binary is rejected."
|
||||
: undefined,
|
||||
};
|
||||
case "wrap":
|
||||
return { value: wrapText(value, Number(step.option)) };
|
||||
case "columns":
|
||||
return transformColumns(value, step.option);
|
||||
case "replace-literal": {
|
||||
const [search, replacement] = parseReplacement(step.option);
|
||||
return { value: replaceLiteral(value, search, replacement) };
|
||||
}
|
||||
case "prefix-lines":
|
||||
return { value: decorateLines(value, step.option, "prefix") };
|
||||
case "suffix-lines":
|
||||
return { value: decorateLines(value, step.option, "suffix") };
|
||||
case "filter-lines": {
|
||||
if (!step.option || step.option.length > 100_000)
|
||||
throw new SyntaxError("Line filter text must not be empty.");
|
||||
return {
|
||||
value: lines(value)
|
||||
.filter((line) => line.includes(step.option))
|
||||
.join("\n"),
|
||||
warning: "Line filtering removes every line without the literal text.",
|
||||
};
|
||||
}
|
||||
case "number-lines": {
|
||||
const start = Number(step.option || "1");
|
||||
if (!Number.isSafeInteger(start) || Math.abs(start) > 1_000_000_000)
|
||||
throw new RangeError("Line-number start must be a bounded integer.");
|
||||
return {
|
||||
value: lines(value)
|
||||
.map((line, index) => `${start + index}. ${line}`)
|
||||
.join("\n"),
|
||||
};
|
||||
}
|
||||
case "join-lines": {
|
||||
const values = lines(value);
|
||||
const delimiter = decodedDelimiter(step.option);
|
||||
assertProjectedOutput(
|
||||
values.reduce((total, line) => total + line.length, 0) +
|
||||
Math.max(0, values.length - 1) * delimiter.length,
|
||||
);
|
||||
return { value: values.join(delimiter) };
|
||||
}
|
||||
case "reverse-lines":
|
||||
return { value: lines(value).reverse().join("\n") };
|
||||
}
|
||||
}
|
||||
|
||||
export function applyPipeline(
|
||||
input: string,
|
||||
steps: readonly TransformStep[],
|
||||
): PipelineResult {
|
||||
if (input.length > MAX_INPUT)
|
||||
throw new Error(
|
||||
`Input exceeds ${MAX_INPUT.toLocaleString()} UTF-16 units.`,
|
||||
);
|
||||
let output = input;
|
||||
const reports: StepReport[] = [];
|
||||
const warnings: string[] = [];
|
||||
for (const step of steps) {
|
||||
if (!step.enabled) continue;
|
||||
const before = output;
|
||||
const result = applyStep(before, step);
|
||||
output = result.value;
|
||||
if (
|
||||
output.length > MAX_OUTPUT ||
|
||||
output.length > Math.max(1024, input.length * 8)
|
||||
)
|
||||
throw new Error(
|
||||
"Pipeline output exceeded its 8× / 32 MiB expansion limit.",
|
||||
);
|
||||
reports.push({
|
||||
id: step.id,
|
||||
type: step.type,
|
||||
beforeUnits: before.length,
|
||||
afterUnits: output.length,
|
||||
changed: before !== output,
|
||||
warning: result.warning,
|
||||
});
|
||||
if (result.warning) warnings.push(result.warning);
|
||||
}
|
||||
return { output, steps: reports, warnings: [...new Set(warnings)] };
|
||||
}
|
||||
|
||||
export function createStep(type: StepType): TransformStep {
|
||||
const defaults: Record<StepType, string> = {
|
||||
"line-endings": "lf",
|
||||
"trim-lines": "",
|
||||
"trim-document": "",
|
||||
"collapse-whitespace": "line",
|
||||
"sort-lines": "en",
|
||||
"dedupe-lines": "",
|
||||
case: "lower",
|
||||
normalize: "NFC",
|
||||
transliterate: "",
|
||||
escape: "json",
|
||||
unescape: "json",
|
||||
wrap: "80",
|
||||
columns: "csv|,|1",
|
||||
"replace-literal": JSON.stringify(["old", "new"]),
|
||||
"prefix-lines": "> ",
|
||||
"suffix-lines": "",
|
||||
"filter-lines": "text",
|
||||
"number-lines": "1",
|
||||
"join-lines": " ",
|
||||
"reverse-lines": "",
|
||||
};
|
||||
return {
|
||||
id: `step-${++nextStepId}`,
|
||||
type,
|
||||
enabled: true,
|
||||
option: defaults[type],
|
||||
};
|
||||
}
|
||||
|
||||
export function textInventory(value: string) {
|
||||
let codePoints = 0;
|
||||
for (let index = 0; index < value.length; index += 1) {
|
||||
const unit = value.charCodeAt(index);
|
||||
if (
|
||||
unit >= 0xd800 &&
|
||||
unit <= 0xdbff &&
|
||||
index + 1 < value.length &&
|
||||
value.charCodeAt(index + 1) >= 0xdc00 &&
|
||||
value.charCodeAt(index + 1) <= 0xdfff
|
||||
)
|
||||
index += 1;
|
||||
codePoints += 1;
|
||||
}
|
||||
return {
|
||||
utf16Units: value.length,
|
||||
codePoints,
|
||||
lines: value ? value.split(/\r\n|\r|\n/u).length : 0,
|
||||
crlf: (value.match(/\r\n/gu) ?? []).length,
|
||||
bareLf: (value.match(/(?<!\r)\n/gu) ?? []).length,
|
||||
bareCr: (value.match(/\r(?!\n)/gu) ?? []).length,
|
||||
finalNewline: /(?:\r\n|\r|\n)$/u.test(value),
|
||||
};
|
||||
}
|
||||
@@ -1,22 +1,21 @@
|
||||
{
|
||||
"$schema": "https://git.add-ideas.de/lotobo/toolbox-sdk/raw/branch/main/schemas/toolbox-app.v1.schema.json",
|
||||
"schemaVersion": 1,
|
||||
"id": "de.add-ideas.text-tools",
|
||||
"name": "Text Tools",
|
||||
"version": "0.2.0",
|
||||
"description": "Compose text transforms and export encoding evidence locally.",
|
||||
"id": "de.add-ideas.translator-tools",
|
||||
"name": "Translator Tools",
|
||||
"version": "0.1.0",
|
||||
"description": "Review and edit translation bundles in the browser with glossary suggestions.",
|
||||
"entry": "./",
|
||||
"icon": "./favicon.svg",
|
||||
"categories": ["text", "developer", "productivity"],
|
||||
"categories": ["text", "developer", "productivity", "i18n"],
|
||||
"tags": [
|
||||
"text",
|
||||
"unicode",
|
||||
"pipeline",
|
||||
"normalize",
|
||||
"sort",
|
||||
"escape",
|
||||
"encoding",
|
||||
"newline"
|
||||
"translation",
|
||||
"i18n",
|
||||
"locale",
|
||||
"glossary",
|
||||
"review",
|
||||
"json",
|
||||
"localization"
|
||||
],
|
||||
"integration": {
|
||||
"contextVersion": 1,
|
||||
@@ -26,32 +25,30 @@
|
||||
"requirements": {
|
||||
"secureContext": false,
|
||||
"workers": false,
|
||||
"indexedDb": false,
|
||||
"indexedDb": true,
|
||||
"crossOriginIsolated": false,
|
||||
"topLevelContext": false
|
||||
},
|
||||
"io": {
|
||||
"accepts": [
|
||||
{
|
||||
"mediaType": "text/*",
|
||||
"extensions": [".txt", ".csv", ".md", ".log"],
|
||||
"label": "Bounded text files"
|
||||
"mediaType": "application/json",
|
||||
"extensions": [".json", ".js"],
|
||||
"label": "Locale JSON or i18next-like translation files"
|
||||
}
|
||||
],
|
||||
"produces": [
|
||||
{
|
||||
"mediaType": "text/plain",
|
||||
"extensions": [".txt"],
|
||||
"label": "Transformed text"
|
||||
},
|
||||
{
|
||||
"mediaType": "application/json",
|
||||
"extensions": [".json"],
|
||||
"label": "Recipe and artifact evidence"
|
||||
"label": "Reviewed translation bundle"
|
||||
}
|
||||
]
|
||||
},
|
||||
"capabilities": { "required": [], "optional": ["web-crypto"] },
|
||||
"capabilities": {
|
||||
"required": [],
|
||||
"optional": ["indexeddb", "local-artifact-handoff"]
|
||||
},
|
||||
"privacy": {
|
||||
"processing": "local",
|
||||
"fileUploads": true,
|
||||
@@ -59,14 +56,14 @@
|
||||
"label": "Inputs stay in this browser; nothing is uploaded."
|
||||
},
|
||||
"source": {
|
||||
"repository": "https://git.add-ideas.de/lotobo/text-tools",
|
||||
"repository": "https://git.add-ideas.de/lotobo/translator-tools",
|
||||
"license": "GPL-3.0-or-later"
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"id": "source",
|
||||
"label": "Source",
|
||||
"url": "https://git.add-ideas.de/lotobo/text-tools"
|
||||
"url": "https://git.add-ideas.de/lotobo/translator-tools"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
import {
|
||||
TOOLBOX_TRANSFER_QUERY_PARAMETER,
|
||||
consumeToolboxTransfer,
|
||||
createToolboxTransfer,
|
||||
createToolboxTransferUrl,
|
||||
loadToolboxContext,
|
||||
readToolboxTransferToken,
|
||||
type ToolboxTransfer,
|
||||
} from "@add-ideas/toolbox-contract";
|
||||
import { APP_VERSION } from "../version";
|
||||
|
||||
export const TRANS_TOOL_ID = "de.add-ideas.translator-tools";
|
||||
export const PORTAL_APP_ID = "de.add-ideas.toolbox-portal";
|
||||
export const MAX_TRANSLATOR_TRANSFER_BYTES = 5 * 1024 * 1024;
|
||||
|
||||
interface ReceiveDependencies {
|
||||
consume: typeof consumeToolboxTransfer;
|
||||
readToken: typeof readToolboxTransferToken;
|
||||
replaceUrl: (url: string) => void;
|
||||
}
|
||||
|
||||
const receiveDefaults: ReceiveDependencies = {
|
||||
consume: consumeToolboxTransfer,
|
||||
readToken: readToolboxTransferToken,
|
||||
replaceUrl: (url) =>
|
||||
window.history.replaceState(window.history.state, "", url),
|
||||
};
|
||||
|
||||
export type IncomingTranslatorTransfer =
|
||||
| { status: "none" | "missing"; error?: string }
|
||||
| { status: "ready"; file: File; transfer: ToolboxTransfer };
|
||||
|
||||
export async function receiveTranslatorTransfer(
|
||||
href: string | URL = window.location.href,
|
||||
dependencies: ReceiveDependencies = receiveDefaults,
|
||||
): Promise<IncomingTranslatorTransfer> {
|
||||
const url = new URL(href);
|
||||
if (!url.searchParams.has(TOOLBOX_TRANSFER_QUERY_PARAMETER))
|
||||
return { status: "none" };
|
||||
|
||||
let token: string | undefined;
|
||||
try {
|
||||
token = dependencies.readToken(url);
|
||||
} finally {
|
||||
url.searchParams.delete(TOOLBOX_TRANSFER_QUERY_PARAMETER);
|
||||
dependencies.replaceUrl(`${url.pathname}${url.search}${url.hash}`);
|
||||
}
|
||||
|
||||
if (!token) return { status: "missing" };
|
||||
|
||||
const transfer = await dependencies.consume(token, TRANS_TOOL_ID);
|
||||
if (!transfer) return { status: "missing" };
|
||||
|
||||
if (transfer.files.length !== 1)
|
||||
throw new RangeError(
|
||||
"Translator Tools accepts exactly one file per handoff.",
|
||||
);
|
||||
|
||||
const source = transfer.files[0];
|
||||
if (!source) throw new RangeError("The handoff did not contain a file.");
|
||||
|
||||
if (!source.size || source.size > MAX_TRANSLATOR_TRANSFER_BYTES)
|
||||
throw new RangeError(
|
||||
"Transferred translation files must be between 1 byte and 5 MiB.",
|
||||
);
|
||||
|
||||
if (source.size !== source.blob.size)
|
||||
throw new RangeError("Transferred size evidence does not match the file.");
|
||||
|
||||
const mediaType = source.mediaType.split(";", 1)[0]!.toLowerCase();
|
||||
if (
|
||||
!mediaType.startsWith("application/json") &&
|
||||
!/\.(?:json|js|txt)$/iu.test(source.name)
|
||||
) {
|
||||
throw new TypeError(
|
||||
"The transferred file is not a supported locale/translation document.",
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
status: "ready",
|
||||
file: new File([source.blob], source.name, {
|
||||
type: source.mediaType || source.blob.type,
|
||||
}),
|
||||
transfer,
|
||||
};
|
||||
}
|
||||
|
||||
interface SendDependencies {
|
||||
createTransfer: typeof createToolboxTransfer;
|
||||
loadContext: typeof loadToolboxContext;
|
||||
}
|
||||
|
||||
const sendDefaults: SendDependencies = {
|
||||
createTransfer: createToolboxTransfer,
|
||||
loadContext: loadToolboxContext,
|
||||
};
|
||||
|
||||
export async function prepareTranslatorPortalHandoff(
|
||||
blob: Blob,
|
||||
name: string,
|
||||
dependencies: SendDependencies = sendDefaults,
|
||||
): Promise<URL> {
|
||||
if (!blob.size || blob.size > MAX_TRANSLATOR_TRANSFER_BYTES)
|
||||
throw new RangeError(
|
||||
"Translator exports must be between 1 byte and 5 MiB.",
|
||||
);
|
||||
|
||||
const context = await dependencies.loadContext();
|
||||
if (context.status !== "ready")
|
||||
throw new Error(
|
||||
"Open Translator Tools from the Toolbox Portal to send this output locally.",
|
||||
);
|
||||
|
||||
const transfer = await dependencies.createTransfer({
|
||||
source: { appId: TRANS_TOOL_ID, appVersion: APP_VERSION },
|
||||
targetAppId: PORTAL_APP_ID,
|
||||
files: [
|
||||
{
|
||||
blob,
|
||||
name,
|
||||
mediaType: blob.type || "application/json",
|
||||
size: blob.size,
|
||||
},
|
||||
],
|
||||
evidence: {
|
||||
formatVersion: "1",
|
||||
operation: "translator-workspace-output",
|
||||
settings: {
|
||||
preferredDestinationAppIds: [
|
||||
"de.add-ideas.regex-tools",
|
||||
"de.add-ideas.data-tools",
|
||||
],
|
||||
},
|
||||
warnings: [
|
||||
"Translator output uses your current local edits; review authoritative and target pairs before downstream use.",
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
return createToolboxTransferUrl(
|
||||
context.context.catalog.homeUrl,
|
||||
transfer.token,
|
||||
{ location: context.context.catalog.homeUrl },
|
||||
);
|
||||
}
|
||||
+1
-1
@@ -1 +1 @@
|
||||
export const APP_VERSION = "0.2.0";
|
||||
export const APP_VERSION = "0.1.0";
|
||||
|
||||
@@ -1,91 +0,0 @@
|
||||
import { expect, test, type Page } from "@playwright/test";
|
||||
import { Buffer } from "node:buffer";
|
||||
|
||||
const ORIGIN = "http://127.0.0.1:4173";
|
||||
async function localOnly(page: Page) {
|
||||
const external: string[] = [];
|
||||
await page.route("**/*", async (route) => {
|
||||
const url = new URL(route.request().url());
|
||||
if (url.origin !== ORIGIN) {
|
||||
external.push(url.href);
|
||||
await route.abort();
|
||||
} else await route.continue();
|
||||
});
|
||||
return external;
|
||||
}
|
||||
|
||||
test("runs from a nested path without external requests", async ({ page }) => {
|
||||
const errors: string[] = [];
|
||||
page.on("pageerror", (error) => errors.push(error.message));
|
||||
page.on("console", (message) => {
|
||||
if (message.type() === "error") errors.push(message.text());
|
||||
});
|
||||
const external = await localOnly(page);
|
||||
await page.goto("/deep/nested/text/");
|
||||
await expect(page.getByRole("heading", { name: "Text Tools" })).toBeVisible();
|
||||
expect(external).toEqual([]);
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
test("applies the visible text pipeline and preserves its last output", async ({
|
||||
page,
|
||||
}) => {
|
||||
const external = await localOnly(page);
|
||||
await page.goto("/deep/nested/text/");
|
||||
await page.getByLabel("Text source").fill(" Alpha \r\nAlpha\n");
|
||||
await page.getByRole("button", { name: "Apply pipeline" }).click();
|
||||
await expect(page.getByLabel("Transformed output")).toHaveValue("Alpha\n");
|
||||
await expect(
|
||||
page.getByRole("cell", { name: "Trim every line" }),
|
||||
).toBeVisible();
|
||||
await expect(
|
||||
page.getByRole("cell", { name: "Deduplicate lines" }),
|
||||
).toBeVisible();
|
||||
expect(external).toEqual([]);
|
||||
});
|
||||
|
||||
test("shows byte evidence and composes an added literal transform", async ({
|
||||
page,
|
||||
}) => {
|
||||
await page.goto("/deep/nested/text/");
|
||||
await page.getByLabel("Decode as").selectOption("utf-8");
|
||||
await page.locator('input[type="file"]').setInputFiles({
|
||||
name: "bom.txt",
|
||||
mimeType: "text/plain",
|
||||
buffer: Buffer.from([0xef, 0xbb, 0xbf, ...Buffer.from("old\r\nold")]),
|
||||
});
|
||||
await page.getByText("Byte decoding evidence").click();
|
||||
await expect(page.getByText("utf-8 / utf-8")).toBeVisible();
|
||||
await expect(
|
||||
page
|
||||
.locator("details")
|
||||
.filter({ hasText: "Byte decoding evidence" })
|
||||
.getByText("1 / 0 / 0", { exact: true }),
|
||||
).toBeVisible();
|
||||
await page.getByLabel("New transformation").selectOption("replace-literal");
|
||||
await page.getByRole("button", { name: "Add step" }).click();
|
||||
await page.getByLabel("Literal search text").fill("old");
|
||||
await page.getByLabel("Literal replacement text").fill("new");
|
||||
await page.getByRole("button", { name: "Apply pipeline" }).click();
|
||||
await expect(page.getByLabel("Transformed output")).toHaveValue("new");
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Export artifact + evidence" }),
|
||||
).toBeVisible();
|
||||
});
|
||||
|
||||
test("serves the release identity and hardened headers", async ({
|
||||
request,
|
||||
}) => {
|
||||
const index = await request.get("/deep/nested/text/");
|
||||
expect(index.ok()).toBe(true);
|
||||
expect(index.headers()["content-security-policy"]).toContain(
|
||||
"default-src 'self'",
|
||||
);
|
||||
expect(await index.text()).not.toMatch(/\b(?:src|href)=["']\//u);
|
||||
const manifest = await request.get("/deep/nested/text/toolbox-app.json");
|
||||
await expect(manifest.json()).resolves.toMatchObject({
|
||||
id: "de.add-ideas.text-tools",
|
||||
version: "0.2.0",
|
||||
entry: "./",
|
||||
});
|
||||
});
|
||||
@@ -1,18 +0,0 @@
|
||||
import { expect, test } from "@playwright/test";
|
||||
|
||||
test("keeps the primary workspace inside a narrow viewport", async ({
|
||||
page,
|
||||
}) => {
|
||||
await page.goto("/deep/nested/text/");
|
||||
await expect(page.locator("main").first()).toBeVisible();
|
||||
await expect(
|
||||
page.locator("main .loading, main .workbench-loading"),
|
||||
).toHaveCount(0);
|
||||
|
||||
const widths = await page.evaluate(() => ({
|
||||
content: document.documentElement.scrollWidth,
|
||||
viewport: document.documentElement.clientWidth,
|
||||
}));
|
||||
expect(widths.viewport).toBeLessThanOrEqual(430);
|
||||
expect(widths.content).toBeLessThanOrEqual(widths.viewport + 1);
|
||||
});
|
||||
@@ -1,17 +0,0 @@
|
||||
import { render, screen } from "@testing-library/react";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { App } from "../../src/App";
|
||||
|
||||
describe("Text Tools", () => {
|
||||
it("renders the local workbench and standard shell", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async () => new Response("Not found", { status: 404 })),
|
||||
);
|
||||
render(<App />);
|
||||
expect(
|
||||
await screen.findByRole("heading", { name: "Text Tools" }),
|
||||
).toBeVisible();
|
||||
expect(await screen.findByText("Browser-local")).toBeVisible();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,13 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { manifest } from "../src/toolbox/manifest";
|
||||
import { APP_VERSION } from "../src/version";
|
||||
|
||||
describe("translator-tools smoke", () => {
|
||||
it("exposes a coherent toolbox manifest", () => {
|
||||
expect(manifest.id).toBe("de.add-ideas.translator-tools");
|
||||
expect(manifest.name).toBe("Translator Tools");
|
||||
expect(manifest.version).toBe(APP_VERSION);
|
||||
expect(manifest.categories).toContain("text");
|
||||
expect(manifest.entry).toBe("./");
|
||||
});
|
||||
});
|
||||
@@ -1,147 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
applyPipeline,
|
||||
textInventory,
|
||||
type TransformStep,
|
||||
} from "../../src/text/pipeline";
|
||||
import {
|
||||
createTextArtifactEvidence,
|
||||
decodeTextWithEvidence,
|
||||
} from "../../src/text/evidence";
|
||||
|
||||
const step = (type: TransformStep["type"], option = ""): TransformStep => ({
|
||||
id: type,
|
||||
type,
|
||||
option,
|
||||
enabled: true,
|
||||
});
|
||||
describe("text pipelines", () => {
|
||||
it("applies transformations in the declared order", () =>
|
||||
expect(
|
||||
applyPipeline(" b \nA\na", [
|
||||
step("trim-lines"),
|
||||
step("dedupe-lines"),
|
||||
step("sort-lines", "en"),
|
||||
]).output,
|
||||
).toBe("a\nA\nb"));
|
||||
it("keeps disabled steps out of the report", () =>
|
||||
expect(
|
||||
applyPipeline("A", [{ ...step("case", "lower"), enabled: false }]),
|
||||
).toEqual({ output: "A", steps: [], warnings: [] }));
|
||||
it("escapes HTML as data", () =>
|
||||
expect(
|
||||
applyPipeline("<img src=x onerror=1>", [step("escape", "html")]).output,
|
||||
).toBe("<img src=x onerror=1>"));
|
||||
it("strictly reverses each escape/encoding stage", () => {
|
||||
const source = 'café <x> & "quoted"\n';
|
||||
for (const option of ["json", "html", "url", "base64", "hex"])
|
||||
expect(
|
||||
applyPipeline(applyPipeline(source, [step("escape", option)]).output, [
|
||||
step("unescape", option),
|
||||
]).output,
|
||||
).toBe(source);
|
||||
expect(() => applyPipeline("©", [step("unescape", "html")])).toThrow(
|
||||
/not emitted/u,
|
||||
);
|
||||
expect(() => applyPipeline("w6k", [step("unescape", "base64")])).toThrow(
|
||||
/canonical|padding|UTF-8/u,
|
||||
);
|
||||
expect(() => applyPipeline("%ZZ", [step("unescape", "url")])).toThrow();
|
||||
});
|
||||
it("selects columns from quoted CSV records including embedded newlines", () => {
|
||||
const result = applyPipeline(
|
||||
'name,note,id\r\nAlice,"hello, world",1\r\nBob,"two\nlines",2\r\n',
|
||||
[step("columns", "csv|,|3,1,2")],
|
||||
);
|
||||
expect(result.output).toContain('1,Alice,"hello, world"');
|
||||
expect(result.output).toContain('2,Bob,"two\nlines"');
|
||||
expect(result.warnings[0]).toMatch(/Quoted CSV/u);
|
||||
});
|
||||
it("retains multi-character literal-delimiter mode", () =>
|
||||
expect(
|
||||
applyPipeline("left::right\na::b", [step("columns", "literal|::|2,1")])
|
||||
.output,
|
||||
).toBe("right::left\nb::a"));
|
||||
it("reports lossy transliteration", () =>
|
||||
expect(applyPipeline("Crème", [step("transliterate")]).warnings[0]).toMatch(
|
||||
/lossy/u,
|
||||
));
|
||||
it("tracks exact line endings", () =>
|
||||
expect(textInventory("a\r\nb\nc\r")).toMatchObject({
|
||||
crlf: 1,
|
||||
bareLf: 1,
|
||||
bareCr: 1,
|
||||
finalNewline: true,
|
||||
}));
|
||||
it("composes literal replacement, filtering, numbering and line decoration", () => {
|
||||
const result = applyPipeline("old apple\nold pear\nbanana", [
|
||||
step("replace-literal", JSON.stringify(["old", "fresh"])),
|
||||
step("filter-lines", "fresh"),
|
||||
step("prefix-lines", "- "),
|
||||
step("number-lines", "10"),
|
||||
step("suffix-lines", "!"),
|
||||
step("join-lines", " | "),
|
||||
]);
|
||||
expect(result.output).toBe("10. - fresh apple! | 11. - fresh pear!");
|
||||
expect(result.warnings).toEqual(
|
||||
expect.arrayContaining([expect.stringMatching(/filtering removes/u)]),
|
||||
);
|
||||
});
|
||||
|
||||
it("reverses lines without reversing grapheme contents", () =>
|
||||
expect(applyPipeline("😀a\nb", [step("reverse-lines")]).output).toBe(
|
||||
"b\n😀a",
|
||||
));
|
||||
|
||||
it("preflights expanding transforms before allocating their output", () => {
|
||||
expect(() =>
|
||||
applyPipeline("x".repeat(40), [
|
||||
step("replace-literal", JSON.stringify(["x", "y".repeat(1_000_000)])),
|
||||
]),
|
||||
).toThrow(/32 MiB/u);
|
||||
expect(() =>
|
||||
applyPipeline("a\nb", [step("prefix-lines", "x".repeat(100_001))]),
|
||||
).toThrow(/100,000/u);
|
||||
});
|
||||
});
|
||||
|
||||
describe("byte and portable artifact evidence", () => {
|
||||
it("records BOM conflicts, UTF-8 validity and byte newline forms", () => {
|
||||
const bytes = Uint8Array.of(0xef, 0xbb, 0xbf, 0x61, 0x0d, 0x0a, 0x62);
|
||||
const decoded = decodeTextWithEvidence(bytes, "latin1", true);
|
||||
expect(decoded.evidence).toMatchObject({
|
||||
byteLength: 7,
|
||||
bom: { encoding: "utf-8", bytes: 3 },
|
||||
utf8: { valid: true },
|
||||
byteNewlines: { crlf: 1, bareLf: 0, bareCr: 0 },
|
||||
});
|
||||
expect(decoded.evidence.warnings.join(" ")).toMatch(/conflicts/u);
|
||||
const malformed = decodeTextWithEvidence(
|
||||
Uint8Array.of(0xc3, 0x28),
|
||||
"latin1",
|
||||
true,
|
||||
);
|
||||
expect(malformed.evidence.utf8).toEqual({
|
||||
valid: false,
|
||||
firstInvalidOffset: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("emits exact output hashes and an honest handoff marker", async () => {
|
||||
const steps = [step("case", "upper")];
|
||||
const pipeline = applyPipeline("hello", steps);
|
||||
const bytes = new TextEncoder().encode(pipeline.output);
|
||||
const evidence = await createTextArtifactEvidence({
|
||||
sourceName: "input.txt",
|
||||
sourceText: "hello",
|
||||
outputName: "output.txt",
|
||||
outputEncoding: "utf-8",
|
||||
outputBytes: bytes,
|
||||
pipeline,
|
||||
steps,
|
||||
});
|
||||
expect(evidence.output.sha256).toMatch(/^[a-f0-9]{64}$/u);
|
||||
expect(evidence.handoff.supportedByThisBuild).toBe(false);
|
||||
expect(evidence.pipeline.steps[0]).toMatchObject({ type: "case" });
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user