Summary
Audit and harden the Workbench's numbered inline-tag (<N>) parsing regexes so that slightly-malformed tags from an LLM don't leak into the target as literal text.
Background
Some models (Mistral Large, reported June 2026) have started mishandling inline tags during AI translation — emitting empty pairs (<1></1> with the translated text left outside) and, potentially, malformed forms (< 1>, </ 1 >, trailing spaces, etc.).
The prompt half is already addressed: the batch and Ollama single-segment translation prompts now include a conditional inline-tag rule (preserve <N> markers, no empty pairs, wrap content) — added only when a segment actually contains <N> markers.
The deferred half is parser tolerance. The Trados plugin solved its equivalent with a single SegmentTagHandler placeholder regex made whitespace- and case-tolerant, so a drifted tag is still recognised (and strippable) rather than leaking as literal text. The Workbench is harder: it parses <N> tags with several regexes spread across import/export/display, so blanket-loosening them is riskier and needs a careful, case-by-case audit.
Scope
Audit these <N>-tag regexes (and any siblings) for malformed-tag tolerance, deciding per-site whether tolerance is safe:
Supervertaler.py:1030 — (</?[a-zA-Z]...>|</?\d+>)Supervertaler.py:1156 — memoQ/HTML/Trados numeric combined patternSupervertaler.py:1286 — <tag>, </tag>, numeric <N>/</N>Supervertaler.py:4267 — </?\d+> (Trados numeric)Supervertaler.py:15881 — </?[0-9]+> (SDLXLIFF numeric)
Goal: a model tag that drifts from the canonical <1> / </1> / <1/> form is still recognised on the round-trip (or cleanly stripped), so literal placeholder text never reaches the delivered target — mirroring the Trados SegmentTagHandler tolerance fix.
Notes
- Lower urgency than the prompt fix; do only after confirming the prompt change reduces the failure rate, and ideally with real malformed-output samples to target the right variants.
- Be conservative: these regexes also drive import/export, so over-broad matching could mis-handle legitimate content.
Summary
Audit and harden the Workbench's numbered inline-tag (
<N>) parsing regexes so that slightly-malformed tags from an LLM don't leak into the target as literal text.Background
Some models (Mistral Large, reported June 2026) have started mishandling inline tags during AI translation — emitting empty pairs (
<1></1>with the translated text left outside) and, potentially, malformed forms (< 1>,</ 1 >, trailing spaces, etc.).The prompt half is already addressed: the batch and Ollama single-segment translation prompts now include a conditional inline-tag rule (preserve
<N>markers, no empty pairs, wrap content) — added only when a segment actually contains<N>markers.The deferred half is parser tolerance. The Trados plugin solved its equivalent with a single
SegmentTagHandlerplaceholder regex made whitespace- and case-tolerant, so a drifted tag is still recognised (and strippable) rather than leaking as literal text. The Workbench is harder: it parses<N>tags with several regexes spread across import/export/display, so blanket-loosening them is riskier and needs a careful, case-by-case audit.Scope
Audit these
<N>-tag regexes (and any siblings) for malformed-tag tolerance, deciding per-site whether tolerance is safe:Supervertaler.py:1030—(</?[a-zA-Z]...>|</?\d+>)Supervertaler.py:1156— memoQ/HTML/Trados numeric combined patternSupervertaler.py:1286—<tag>,</tag>, numeric<N>/</N>Supervertaler.py:4267—</?\d+>(Trados numeric)Supervertaler.py:15881—</?[0-9]+>(SDLXLIFF numeric)Goal: a model tag that drifts from the canonical
<1>/</1>/<1/>form is still recognised on the round-trip (or cleanly stripped), so literal placeholder text never reaches the delivered target — mirroring the TradosSegmentTagHandlertolerance fix.Notes