[office/crow-translate] /: Collapse any repeated transcription, without exception

Git commit dc88bce1db47e19cc666861e757842d4decdb2e9 by Mauritius Clemens.
Committed on 18/08/2026 at 13:45.
Pushed by pillowtrucker into branch 'master'.

Collapse any repeated transcription, without exception

collapseRepeatedTranscription tried to tell a decoding loop from text
that legitimately repeats, using copy count and (briefly) line length: a
single repeating line only counted as a runaway at four or more copies,
so a poem's refrain would survive. That let the most common real failure
straight through. glm-ocr transcribes a whole paragraph as ONE line and
re-emits it, so its runaway arrives as period=1 with as few as two or
three complete copies and no truncated tail - meaning whether the user
saw duplicated text came down to how many copies the model happened to
produce.

Drop the exception. Any exact tiling, with an optional incomplete final
tile, collapses to a single unit. A scanned poem whose lines tile exactly
loses its repeats and the reader can restore them; that is the better
trade than a backstop which fires unpredictably. Text that merely repeats
without tiling - a stanza echoed at the end - is untouched, since it
never matched in the first place.

Reproduced against the real local glm-ocr with the request the app
actually sends, on the dense screenshot from the report: three verbatim
copies of one 794-character line. Before, 2384 chars in and 2384 out;
after, 794 - the transcription exactly once.

M  +7    -9    src/ocr/llmocr.cpp
M  +24   -7    tests/test_llmocr.cpp

https://invent.kde.org/office/crow-translate/-/commit/dc88bce1db47e19cc666861e757842d4decdb2e9

diff --git a/src/ocr/llmocr.cpp b/src/ocr/llmocr.cpp
index 33e83479..059846cf 100644
--- a/src/ocr/llmocr.cpp
+++ b/src/ocr/llmocr.cpp
@@ -118,10 +118,12 @@ QString LlmOcr::collapseRepeatedTranscription(const QString &text)
     // transcription, then the same block re-emitted verbatim until the token
     // budget runs out - which typically cuts the FINAL copy mid-block, leaving
     // a partial tail whose last line is a prefix of the block's next line.
-    // Poetry can legitimately repeat lines, so single-line loops stay intact
-    // unless they run very long (>= 4 copies); a multi-line block repeated
-    // twice or more is collapsed even at 2 copies (user decision 2026-08-17),
-    // accepting that a 2-line refrain doubled verbatim would also collapse.
+    // Any such tiling is collapsed to a single unit, with no exception for
+    // text that legitimately repeats: a scanned poem whose lines tile exactly
+    // loses its repeats, and the reader can restore them (user decision
+    // 2026-08-18). Trying to tell a refrain from a decoding loop by copy count
+    // or line length only made the real failures depend on how many copies the
+    // model happened to emit.
     for (int period = 1; period <= n / 2; ++period) {
         const int copies = n / period;
         if (copies < 2) {
@@ -139,11 +141,7 @@ QString LlmOcr::collapseRepeatedTranscription(const QString &text)
         if (!tiled) {
             continue;
         }
-        const bool runAwayLoop = (period == 1 && copies >= 4) || (period >= 2 && copies >= 2);
-        if (runAwayLoop) {
-            QStringList unit = lines.mid(0, period);
-            return unit.join(newline);
-        }
+        return lines.mid(0, period).join(newline);
     }
     return text.trimmed();
 }
diff --git a/tests/test_llmocr.cpp b/tests/test_llmocr.cpp
index 893e2897..33e726d0 100644
--- a/tests/test_llmocr.cpp
+++ b/tests/test_llmocr.cpp
@@ -100,19 +100,19 @@ private slots:
         QCOMPARE(LlmOcr::collapseRepeatedTranscription(QString()), QString());
         QCOMPARE(LlmOcr::collapseRepeatedTranscription(QStringLiteral(" ")), QString());
 
-        // A single line looped to runaway length collapses.
+        // A single line looped collapses.
         QCOMPARE(LlmOcr::collapseRepeatedTranscription(QStringLiteral("no\nno\nno\nno")), QStringLiteral("no"));
 
-        // A multi-line block repeated twice now collapses too (2-copy bar,
-        // user decision): a 2-line refrain poem doubled verbatim is the known,
-        // accepted edge.
+        // Any exact tiling collapses, down to two copies.
         const QString twoCopies = block + QStringLiteral("\n\n") + block;
         QCOMPARE(LlmOcr::collapseRepeatedTranscription(twoCopies), block);
 
-        // A short poem repeating one line three times stays intact (single
-        // lines only read as runaway at four or more copies).
+        // Deliberately accepted loss: a poem whose lines tile exactly is
+        // indistinguishable from a decoding loop, so it collapses too. Telling
+        // the two apart by copy count or line length only made real runaways
+        // depend on how many copies the model happened to emit.
         const QString triple = QStringLiteral("Water\nWater\nWater");
-        QCOMPARE(LlmOcr::collapseRepeatedTranscription(triple), triple);
+        QCOMPARE(LlmOcr::collapseRepeatedTranscription(triple), QStringLiteral("Water"));
 
         // A poem that repeats its first stanza verbatim at the end (not an
         // exact tiling of the whole response) keeps every line.
@@ -126,6 +126,23 @@ private slots:
             + QStringLiteral("\n\nSystem Monitor\nCPU 12%");
         QCOMPARE(LlmOcr::collapseRepeatedTranscription(truncated), block);
 
+        // glm-ocr emits a whole paragraph as ONE very long line and repeats
+        // it, so the runaway arrives as period=1 with only two or three
+        // COMPLETE copies and no truncated tail. Reported 2026-08-18 against a
+        // dense Wikipedia screenshot; reproduced against the real local
+        // glm-ocr with the app's exact request (three verbatim copies of one
+        // 794-character line).
+        const QString paragraph = QStringLiteral(
+            "JoJo's Bizarre Adventure had over 100 million copies in circulation by December 2016,[88] "
+            "and over 120 million copies in circulation by August 2023.[89] making it one of the "
+            "best-selling manga series of all time.[90] The first volume of JoJolion was the second "
+            "best-selling manga for its debut week; its second volume reached third place, and its "
+            "third reached second place.[91][92][93]");
+        const QString longLineThrice = paragraph + QStringLiteral("\n") + paragraph + QStringLiteral("\n") + paragraph;
+        QCOMPARE(LlmOcr::collapseRepeatedTranscription(longLineThrice), paragraph);
+        const QString longLineTwice = paragraph + QStringLiteral("\n") + paragraph;
+        QCOMPARE(LlmOcr::collapseRepeatedTranscription(longLineTwice), paragraph);
+
         // A single line repeated, ending in a line the token budget cut
         // mid-word (period=1 case) collapses to one line too.
         const QString singleLineLoop = QStringLiteral("Try integrating using the Ollama web service to create a locally-run personal code assistant.\n")