Skip to content

fix(infer): add repetition_penalty and CLI ngram parameters to prevent text looping - #73

Open
not-knope wants to merge 2 commits into
baidu:mainfrom
not-knope:fix/repetition-looping-ngram-penalty
Open

fix(infer): add repetition_penalty and CLI ngram parameters to prevent text looping#73
not-knope wants to merge 2 commits into
baidu:mainfrom
not-knope:fix/repetition-looping-ngram-penalty

Conversation

@not-knope

Copy link
Copy Markdown

Description

This PR fixes text repetition/looping issues on dense or structured document pages reported in #55.

Problem:

When processing dense document pages (e.g. forms, government documents), the inference path can enter endless text repetition loops generating 8K–32K redundant tokens until \max_length\ is reached. The root cause is that
o_repeat_ngram_size=35\ only blocks repeating n-grams of exact length 35; shorter repeating phrases (8–15 tokens) fall between 35-gram sliding window boundaries and repeat infinitely.

Solution & Changes Made:

  1. **Added --repetition_penalty\ CLI argument and payload support in \infer.py**: Allows passing a repetition penalty factor (e.g. --repetition_penalty 1.1) in \infer_one\ to penalize repeated tokens at the logit level and break short phrase repetition loops.
  2. **Added --no_repeat_ngram_size\ and --ngram_window\ CLI flags in \infer.py**: Allows users to configure n-gram size (e.g. \15\–\20) and sliding window sizes instead of keeping them hardcoded at module level.
  3. Updated \README.md\ Documentation: Documented --repetition_penalty, --no_repeat_ngram_size, and --ngram_window\ under useful options and added recommendations on mitigating OCR text repetition loops.
  4. Added Unit Tests (\ ests/test_infer_repetition.py): Verified default argument parsing, custom CLI flag parsing, and correct payload construction in \infer_one.

Fixes #55

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Text repetition/looping on certain pages — root cause: no_repeat_ngram_size=35 misses short repeating units

1 participant