Skip to content

feat(documents): visually parse uploads and feed them into notes - #177

Merged
Valtora merged 16 commits into
mainfrom
feat/visual-document-parsing
Jul 30, 2026
Merged

feat(documents): visually parse uploads and feed them into notes#177
Valtora merged 16 commits into
mainfrom
feat/visual-document-parsing

Conversation

@Valtora

@Valtora Valtora commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Pull Request

Description

Uploaded documents were parsed as plain text: PDF and TXT/MD only, sliced into 500-character sliding windows, reaching meeting chat and nothing else. A slide deck or a scanned report lost almost everything that made it worth attaching.

This replaces the pipeline end to end.

Structural extraction covers PPTX, DOCX, XLSX and CSV alongside PDF, TXT, MD and images. Office formats are zipped XML rather than pixels, so this recovers slide titles, table cells, speaker notes, and the exact values behind native charts without any headless-office renderer. That last point decided the design: python-pptx returns the numbers a chart was built from, which a vision model reading a rendered chart could only estimate from pixel heights, and speaker notes are invisible to any visual read at all.

Visual analysis is on by default. Each page's images go to the user's configured model, and generate_text_from_images is implemented for Anthropic, OpenAI, Gemini and Ollama. Both subscription CLIs are wired without loosening their posture: Codex takes first-class --image arguments, and Claude takes inline content blocks through the Agent SDK's streaming-input mode, so allowed_tools stays empty and max_turns stays at one.

Local OCR sits between the two. It runs on the worker, calls no provider, and makes a scanned page searchable on an install with no AI configured at all. It is skipped on any page whose text layer is already substantial, since it is strictly worse at text than a real text layer, and it only ever supplements that layer rather than replacing it.

Parsed pages feed notes generation, not just chat, which was the largest gap: you could attach the agenda and the deck and the generated notes would ignore both.

Parsing runs on a new Celery lane. A parse has no page cap, so one large upload can hold a worker slot for a long time; on the io lane that would sit beside Meeting Edge and meeting chat and degrade a live meeting. The lane reuses the worker-io image, so it adds a container but no build and no image to scan.

Embedding model change (destructive, one time)

The RAG model moves from all-MiniLM-L6-v2 to jina-embeddings-v2-small-en. The old model truncated input at roughly 256 tokens, so the tail of any real document page was never searchable; the new one has an 8192-token window, which lets a whole page be a single retrieval unit.

Vectors from two models are not comparable at any width, so there is no migration path: the revision widens context_chunks.embedding from 384 to 512 dimensions and deletes every existing row. Search and meeting chat return nothing until rebuild_text_embeddings_task has run. Re-indexing is local inference and costs nothing; documents predating this change are re-parsed structurally, never visually, so no provider quota is spent unasked. Full operator instructions are in docs/DEPLOYMENT.md.

New dependencies

  • python-pptx, openpyxl, python-docx, pytesseract (requirements/worker.txt)
  • tesseract-ocr and tesseract-ocr-eng system packages (docker/Dockerfile.worker, ~15 MB)

Tesseract is driven through the system binary rather than a Python OCR library on purpose: every such library depends on onnxruntime, which the worker image deliberately replaces with onnxruntime-gpu, and reintroducing the CPU build alongside it is the exact conflict that replacement prevents.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that changes existing behaviour)
  • Documentation update

Breaking: the embedding cutover empties context_chunks. Search and meeting chat are degraded until the rebuild task completes.

Checks run

  • Backend tests: source .venv/bin/activate && pytest — 1280 passed
  • Python quality: python scripts/check.py — lint, format, whitespace, filesize, heldpins, typecheck, docs, alembic, tests
  • Frontend lint: cd frontend && npm run lint
  • Frontend unit tests: cd frontend && npm run test — 335 passed across 52 files
  • Frontend build: cd frontend && npm run build
  • Docs validation: python3 scripts/validate_docs.py
  • Alembic validation: python3 scripts/validate_alembic.py

Migration impact

  • No database migration in this PR.
  • Adds an Alembic migration. Single checked-in head preserved; no committed revisions deleted or renamed. Upgrade and downgrade behaviour described above.

Three revisions, single head c9e4a25b71f3:

  • a4f1e9c72b58document_pages, parse state on documents, notes staleness on transcripts, and the 384 to 512 dimension embedding cutover. Destructive: empties context_chunks, for the reason above. The downgrade is symmetrically destructive.
  • b7d3f1a02c94file_size_bytes and parse_stage on documents.
  • c9e4a25b71f3 — adds OCR to the pageparsemode enum. Commits before ALTER TYPE ... ADD VALUE, which cannot run inside a transaction block, and is idempotent via IF NOT EXISTS. No downgrade: PostgreSQL cannot drop an enum value, and recreating the type would rewrite the table.

document_pages is registered in the backup ownership map and archived rather than rebuilt, unlike context_chunks: re-embedding is free local inference, but re-parsing can mean a paid vision call per page, so a restore that re-ran the parser would spend provider quota to reproduce text the archive already held.

Documentation impact

  • No documentation change required.

  • Updated the relevant guide(s) in the same PR.

  • docs/USAGE.md — attaching documents, supported formats, the 250 MB ceiling, visual analysis and its opt-out, the Ollama vision-model requirement, and why timing relative to notes generation matters.

  • docs/ARCHITECTURE.md — the fourth Celery lane and a Document Parsing section covering the three tiers, incremental page persistence, and the untrusted-content fencing.

  • docs/DEPLOYMENT.mdworker-parse including why it reuses the worker-io image, and the embedding cutover as a one-time destructive migration with the rebuild command.

Security impact

  • No security-sensitive change.
  • Touches auth, tokens, encryption, capture ownership, or exposure. docs/SECURITY.md boundaries preserved and updated where behaviour changed.

No change to auth, tokens, or encryption, and docs/SECURITY.md needed no edit, but three things are worth review:

  • Untrusted document content now reaches two LLM sinks. Document text is authored outside the meeting, and visual parsing widens that: a model transcribing a page reproduces any instruction printed on it. Both notes generation and meeting chat now fence it in <attached_document> delimiters with an explicit data-not-instructions rule. This matters most for chat, where update_meeting_notes can overwrite the notes document.
  • New file-serving endpoint. GET /documents/{id}/download returns the original upload. Ownership is checked through the recording exactly as delete and re-parse do, and the filename is sanitised before it reaches Content-Disposition, where a quote or newline in a user-supplied title could inject a header.
  • CLI OAuth posture is unchanged. Images ride as --image arguments (Codex) or inline content blocks (Claude). No tool was granted and max_turns stays at one, so the ADR-0002 rationale is intact.

Upload limits: the document cap rises from 20 MB to 250 MB, matching legacy recordings, and gains the free-disk pre-flight the restore path already had. The cap alone never protected the volume, since many uploads below it still fill a disk.

Manual verification

Deployed to a live homelab instance and verified end to end:

  • Visual parsing through a real Claude subscription. A 7-page deck parsed with all pages recorded VISUAL. Stored content includes diagram layout, arrow directions and logo identification, none of which a PDF text layer can produce.
  • Local OCR on a genuinely scanned PDF (text rasterised, so the text layer is empty): all lines recovered verbatim.
  • Granular progress observed live: 0 → 3 → 4 → 6 → 7 across a parse, stepping in clusters of three, which is the vision fan-out.
  • Migration applied on a populated database; pageparsemode confirmed as STRUCTURAL, VISUAL, OCR.
  • Stalled-parse recovery, after a worker died mid-parse: pages persisted, the row was recoverable, and re-parse resumed.
  • Four format parsers exercised against real generated files in tests, including PPTX chart values and speaker notes.

No capture, recording context-menu, or auth paths were touched, so no browser capture smoke testing applies.

Still pending before merge:

  • Browser validation of the Documents table: download button, size column, and the re-parse flow.
  • Notes generation with an attached document, and the stale-notes regenerate banner.
  • Meeting chat retrieval with both a transcript and a document present.
  • PPTX, DOCX, XLSX and CSV uploaded through the UI rather than parsed in tests.
  • Backup and restore round trip confirming documents return without re-parsing.
  • The post-upgrade rebuild_text_embeddings_task sweep on a populated library.

Opened as a draft until that manual pass is complete.

Valtora added 16 commits July 30, 2026 17:32
Lays the foundation for visually parsing uploaded documents. Text-only
extraction misses most of what a deck or a report actually carries, so
parsing gains a per-format structural pass and, behind it, the ability to
send page images to a vision-capable model.

Structural extraction covers PPTX, DOCX, XLSX and CSV alongside the
existing PDF and plain-text paths. Office formats are zipped XML rather
than pixels, so this recovers slide titles, tables, speaker notes and
native chart data without any headless-office renderer: python-pptx
returns the exact values a chart was built from, which a vision model
reading a rendered chart could only estimate. Shapes are walked in
reading order rather than z-order, and Word paragraphs and tables are
read from the body XML so their interleaving survives.

Vision arrives as generate_text_from_images on the LLMBackend contract,
implemented for Anthropic, OpenAI, Gemini and Ollama. Both subscription
CLIs are wired without loosening their posture: Codex takes first-class
--image arguments, and Claude takes inline content blocks through the
Agent SDK's streaming-input mode, so allowed_tools stays empty and
max_turns stays at one. A provider that cannot accept images raises
VisionUnsupportedError, which lets a document degrade to a structural
parse rather than fail outright.

The RAG embedding model moves from all-MiniLM-L6-v2 to
jina-embeddings-v2-small-en. The old model truncated input at roughly
256 tokens, so the tail of any real page was never searchable; the new
one has an 8192-token window, which makes a whole page a single
retrieval unit. Vectors from two models are not comparable at any width,
so the migration widens the column to 512 dimensions and purges the
existing rows, and chunks now carry an embedding_version stamp mirroring
the voiceprint precedent.

document_pages stores parsed content one row per page, written as each
page completes so an interrupted parse resumes instead of repeating paid
vision calls. It is archived rather than rebuilt on restore, unlike
context_chunks, because re-embedding is free but re-parsing is not. The
MCP get_documents tool now reads those pages, which also fixes its text
reconstruction: joining overlapping chunks re-emitted fifty characters
at every boundary.

Adds GET /llm/vision-support so the upload flow can warn before a
document is parsed and silently downgraded. The result is tri-state:
only Ollama can answer definitively, via its capabilities list, and a
probe failure resolves to unknown rather than blocking an upload.

Refs: docs/USAGE.md
Replaces the document pipeline end to end. Parsing produced 500-character
sliding windows of PDF or plain text and reached only meeting chat; it now
produces per-page Markdown for ten formats and reaches notes generation as
well.

The parse task moves to its own Celery lane rather than sharing io. A
parse has no page cap, so one large upload can hold a worker slot for a
long time, and on the io lane that would sit beside Meeting Edge, meeting
chat and notes generation and degrade a live meeting. The lane reuses the
worker-io image, so it adds no build and no image to scan, and visual
parsing can still route through a subscription CLI.

Pages are written as each completes, so a worker restart resumes from the
first missing page instead of repeating vision calls that were already
paid for. Vision requests fan out three at a time, which cuts wall-clock
on a long document without tripping provider rate limits. A provider that
cannot accept images downgrades the whole document to a structural parse
once and records why, rather than failing every page in turn; an image
upload has no text layer to fall back on, so that case is a real error
instead of an empty success.

Documents now reach notes generation through both prompt paths, carrying
the full parsed text with no cap. That matches the transcript, which has
never been truncated either, and a model with too small a context reports
it explicitly. A document finishing after the notes were written marks
them stale so the user is prompted to regenerate; regenerating is never
automatic because it spends their quota and overwrites hand edits.

Meeting chat gains a separate retrieval budget per source. Transcript and
document chunks previously competed for one pool of five, so a strongly
matching passage could take every slot and leave an attached deck
unrepresented. Every retrieval also filters on the embedding version,
since vectors from the previous model would otherwise score as noise
ranked like relevance.

Document text is untrusted input, and visual parsing widens that: a model
transcribing a page reproduces any instruction printed on it. Both sinks
now fence it in explicit delimiters with a data-not-instructions rule,
which matters most for chat, where update_meeting_notes can overwrite the
notes document.

The upload cap rises from 20 MB to 250 MB, matched to legacy recordings,
and gains the free-disk pre-flight the restore path already had. The cap
alone never protected the volume: many uploads below it still fill a
disk. Uploads take a deep-parse flag, defaulting on, and documents can be
parsed again from the API once a vision model is available.

Restore re-indexes rather than re-parses, since document_pages is
archived: re-running the parser would spend a vision call per page to
reproduce text the backup already restored.

Refs: docs/USAGE.md, docs/ARCHITECTURE.md
Brings the document UI in line with the new parsing pipeline.

The upload modal accepts the seven new formats, raises its ceiling to
250 MB, and offers a visual-analysis toggle defaulted on. The toggle is
forced on and locked for image uploads, where turning it off would leave
nothing to index at all. Above 20 MB it warns that every page is sent to
the configured AI provider, so the cost is visible at the moment the
choice is made rather than after.

The documents list gains per-page progress, a page count, a re-parse
action, and the non-fatal warning a downgraded parse records. Progress
matters here because a visual parse runs for minutes on a long document
and a bare spinner reads as a hang.

A documents panel now appears alongside the live transcript and notes
while a meeting is recording or processing. The point is timing rather
than convenience: notes are generated at the end of processing, so a deck
attached during the meeting is normally parsed in time to reach them on
the first pass instead of marking them stale afterwards. It is
read-mostly by design, with delete and re-parse left to the Documents
tab, since a destructive control next to a live recording is the wrong
thing to offer mid-meeting.

Notes show a dismissible banner when a document finished parsing after
they were written, offering to regenerate and stating plainly that doing
so overwrites manual edits. Dismissal is not persisted: the banner is
advice rather than a task, and the notes are still stale after a reload.

The upload modal's accents move from blue to orange, matching the
documents view it opens from and the rest of the product.
Sixty-two tests across three files, built on real files rather than
recorded strings so a library upgrade that changes extraction behaviour
fails here instead of quietly degrading uploads.

test_document_parsing covers every format end to end, including the two
results that justify the design: a deck's chart values come back exact
rather than estimated, and speaker notes survive even though they are
invisible on the rendered slide. It also pins the chunker's boundaries,
since splitting a Markdown table mid-row would make it unreadable, and
the merge rule that a rendered page replaces its text layer while a
figure description supplements it.

test_document_prompt_integration asserts both note-generating paths
render documents through the same builder. They are assembled separately
and would otherwise drift, which is the whole reason the builder is
shared. It also covers the injection fencing, including a title crafted
to break out of its own attribute.

test_document_task covers the decision logic around a parse with stub
backends, so no provider is contacted. The important distinction is
between a per-page failure, which falls back to structural text and
carries on, and VisionUnsupportedError, which condemns every remaining
page and must propagate so the document downgrades once. Provider
refusal messages are matched against transient ones as well, because
confusing the two would permanently downgrade a document over a single
flaky call.
USAGE gains an "Attaching Documents" section covering the new formats,
the 250 MB ceiling, and where documents can be attached. It explains why
timing matters: notes are generated once, at the end of processing, so a
document attached during the meeting reaches them on the first pass while
one attached afterwards raises a regenerate prompt instead. Visual
analysis is described as the default with its opt-out, along with the
Ollama requirement to select a vision-capable model, and the note that
diagram-heavy decks parse better exported to PDF.

ARCHITECTURE describes the fourth Celery lane and adds a Document
Parsing section covering the structural pass, when visual analysis
replaces the text layer rather than supplementing it, incremental page
persistence, and the untrusted-content fencing applied to both prompt
sinks.

DEPLOYMENT documents worker-parse, including why it runs the worker-io
image rather than a new one, and records the RAG embedding cutover as a
destructive one-time migration. That entry is the important one: the
model change deletes every existing vector because vectors from two
models are not comparable, so search and meeting chat return nothing
until the rebuild task has run. It gives the command, notes that
re-indexing is free local inference, and states that pre-release
documents are re-parsed structurally rather than visually so no provider
quota is spent unasked.
The API container failed to start with "expression
'DocumentPage.page_number' failed to locate a name". Document declares its
pages relationship by string, and the class it names was only registered
through backend/models/registry.py, which several real entry points never
import: startup_canonical_cutover reaches the model graph through User
alone, and the API process reaches it through recording_public. Either
one configures its mappers with DocumentPage unknown.

The whole test suite passed because most test modules import the registry
explicitly, which registers everything and makes the gap invisible. The
new test encodes the actual invariant instead: importing a model module
must register every class its own relationships name, checked in a
subprocess per module because the SQLAlchemy class registry is
process-global and one earlier import would mask the result. It fails
without this fix and passes with it.

Also corrects the local compose worker-parse service, which pointed at
ghcr.io/valtora/nojoin-worker-io:latest while this stack builds from
source to nojoin-worker-io:local. Left as it was, the parse lane would
have silently run the last release's code.
Document parsing died with SIGSEGV and Celery reported WorkerLostError.
The cause is not the parser: TextEmbeddingService asked ONNX Runtime for
the CUDA execution provider unconditionally, and onnxruntime-gpu does not
degrade gracefully when no device is present. It loads its CUDA provider
library, finds nothing, and takes the process down. A segfault cannot be
caught, so the existing try/except fallback to CPU never ran.

Confirmed by probe: requesting CUDA exits 139 on both worker-io and
worker-parse, with the old and the new embedding model alike, while
CPUExecutionProvider alone succeeds on both. This was therefore a latent
fault in the io lane rather than something the model swap introduced, but
routing parsing to a second GPU-less lane made it fatal and visible.

The fix uses the gpu_is_present helper that already exists for this
purpose, checking for an NVIDIA device node rather than asking a CUDA
library that may itself be the thing that is broken.
get_available_providers is no help: it reports what onnxruntime was
compiled with, not what is usable. verify_gpu_providers stays, now
covering the separate case of a GPU being present but CUDA still being
dropped for a missing shared library.
The card grid clipped document titles at roughly twenty characters and
reported progress as a page counter that reached "7 of 7" while indexing
was still running, which reads as a hang rather than as progress.

Replaces it with a table: name, format, size, pages, status, and per-row
actions. Each row in flight carries its own progress bar, and the label
beneath it comes from a new parse_stage column rather than from the page
counter, so the phases are distinguishable -- reading pages, analysing
with AI, indexing for search. A document whose page count is not yet
known shows an indeterminate bar rather than a stalled zero. Polling
drops to two seconds, since the bar is now the primary signal that work
is still moving.

Adds file_size_bytes, taken from the byte count the upload already
returned rather than stat()-ed on read: the list is a hot path and the
file may have been removed since. Existing rows stay NULL and render as a
dash, which is honest -- stat() would report the file now, not the upload.

Warnings move below the table rather than into a status cell. They are
sentences, and a row that grows to fit one breaks the scan. The format
column reads the extension from the filename rather than the stored MIME
type, which is client-supplied and unreliable.
A worker that dies mid-parse leaves the document PROCESSING forever, and
the re-parse endpoint refused PROCESSING outright, so the only way out
was editing the database. That is exactly what happened when the
embedding model segfaulted: the pages were stored and resumable, but
nothing in the product could restart the job.

A parse writes to its row on every page batch and every stage change, so
silence is a reliable signal that the task is gone rather than slow.
Re-parse is now permitted once a PROCESSING document has been quiet for
ten minutes, and the table shows it as Stalled with a pointer to the
action rather than an animated bar that never moves. The threshold is
deliberately generous: a false positive only allows a redundant parse,
while a false negative wedges the document permanently.
Every document downgraded to a structural parse with "no AI provider is
configured for this account", regardless of the provider actually set up.
The cause was a NameError: get_llm_backend_with_secondary was reached
through `from .constants import *`, but constants imports that factory
inside a function, so it never enters the star namespace. The broad
except then swallowed the NameError and reported it as a configuration
problem.

Imports it explicitly, and returns a reason alongside the backend so the
message on the document card matches what actually happened. Blaming the
user's settings for an internal fault sends them to fix something that
was never wrong, which is worse than saying nothing.

The regression test asserts the name is both referenced by the function
and importable for real, since appearing in the source was exactly the
condition that held while the code was broken.
SecondaryLLMBackend wraps a primary and a secondary provider and forwards
each LLMBackend method explicitly, so generate_text_from_images fell
through to the base class, which raises VisionUnsupportedError by design.
Any account using the provider chain therefore downgraded every document
to a structural parse regardless of whether its provider could read
images, and the card blamed the model.

Forwarding it makes the fallback meaningful rather than incidental: a
primary that cannot accept images fails, the secondary is tried, and only
when both fail does the primary's VisionUnsupportedError surface and
downgrade the document once. That is the chain users are already shown in
Settings. supports_vision is forwarded as a tri-state: a definite yes
from either provider is a yes, a definite no from both is a no, and
anything else stays unknown and is settled by making the call.
Parsing degraded straight from visual analysis to a format's own text
layer, so a scanned page produced nothing whenever no vision model was
reachable. Local OCR now sits between the two: it runs on the worker,
costs nothing, sends nothing anywhere, and makes a scanned document
searchable on an install with no AI configured at all.

It is deliberately the middle tier rather than a replacement. OCR
transcribes glyphs; it cannot say what a chart shows or describe a
diagram, so visual analysis still wins wherever it produced anything, and
the card says explicitly when OCR carried the document instead. It is
also skipped on any page whose text layer is already substantial, since
it is strictly worse at text than a real text layer, and it only ever
supplements that layer rather than replacing it.

Tesseract is driven through the system binary rather than one of the
Python OCR libraries because all of them depend on onnxruntime, which
docker/Dockerfile.worker deliberately replaces with onnxruntime-gpu.
Reintroducing the CPU build alongside it is the exact conflict that
replacement exists to prevent, and it is the same class of fault that
segfaulted the parse lane. The cost is 15 MB of language data and a pure
Python wrapper. Absence is handled rather than assumed: a deployment
without the binary detects it and falls through to structural text.

Pages record which tier produced them, so a partially OCR'd document is
legible after the fact. An image upload, which has no text layer at all,
is now only an error when OCR also finds nothing, and the message
distinguishes "OCR is not installed" from "OCR found nothing".
…ance

Three fixes to the documents table, plus a correction to what the AI is
told about its own inputs.

Downloads: a new authenticated endpoint returns the original file under
its uploaded name rather than the UUID it is stored as, and the table
gains a per-row button. The client fetches a blob through the API client
instead of using a plain link, because an anchor href cannot carry the
auth header. The filename is sanitised before it reaches
Content-Disposition, where a quote or newline would let a user-supplied
title inject a header.

Sizes: documents uploaded before the column existed showed a dash
forever. They are now backfilled from the stored file when the list is
read. Reading it from disk is accurate rather than a guess, since an
uploaded document is written once and never rewritten, so the file's size
is its upload size. A document whose file is gone stays NULL rather than
recording a misleading zero.

Layout: the name column takes the space freed by tightening the fixed
columns, since a truncated title is the one value in the row that cannot
be guessed from context.

Provenance: chunks now record which tier produced their text, and both
the chat context and the notes prompt state it. Without this a model
reading a vision transcription reports the document as "text extraction
only" -- accurate about what it received, since only text is ever sent to
chat, but wrong about the document and misleading to the user reading the
answer.
…oint makes

Relationships name their targets by string, so SQLAlchemy resolves them
only once every class exists. Which classes are registered was left to
each entry point's own imports, and that failed twice: DocumentPage broke
container start, and ContextChunk broke any process that reached Document
without also importing context_chunk.

Both were invisible to the test suite, because most test modules import
backend/models/registry.py directly. That registers everything and makes
a missing registration unobservable, so the suite passed while production
could not configure its mappers.

backend/core/db.py is the fix. Every entry point imports it -- nothing
touches the database without a session -- and no model imports it back,
so it registers the full graph once, with no cycle and no per-module
cross-imports to keep in step. This replaces the DocumentPage import
added to document.py earlier, which treated the symptom.

The test now asserts the property rather than a list: each real entry
point is imported in its own subprocess, since the class registry is
process-global and one earlier import would mask the result, and then
configure_mappers must succeed. Removing the chokepoint fails all five
cases.
…clock

A running parse showed as Stalled while the re-parse it invited was
refused with "this document is already being parsed". The two disagreed
because they were computing the same thing in different places: the
client subtracted updated_at from Date.now, and the server compared the
same column in UTC.

updated_at is stored and serialised without a timezone, so JavaScript
parses it as local time. Any browser outside UTC therefore misjudges the
age by its whole offset, and at an offset above the ten minute threshold
every in-flight parse reads as stalled and never recovers. The
development host runs on UTC, which is why this never appeared here.

The flag is now computed server-side and sent as is_stalled, so the
browser does no clock arithmetic at all and the label beside the button
cannot contradict what the button will do. The endpoint and the
serializer call one function rather than holding separate copies.

Also makes an active parse look active. Accepting a re-parse resets the
page counters immediately, so the bar starts from zero instead of
rendering the previous run's completed count and appearing finished
before it began; a parse whose page count is not yet known shows an
indeterminate sweep rather than a bar sitting at zero, which reads as
stuck; and the stage line carries a spinner.
The bar jumped straight from 0 to the full page count. Pages are
persisted in batches of eight, and pages_parsed was written once per
batch, so any document of eight pages or fewer showed a single step and
nothing in between.

Progress is now decoupled from persistence. The vision pass calls back as
each page settles, successfully or not, and pages that never reach that
tier report as they are persisted, which matters most for OCR since it
runs in the persist loop and can take seconds per page. The callback
fires from the as_completed loop, which runs in the calling thread, so it
may touch the session; the worker threads never do.

Counting is a set of settled page numbers against a baseline taken before
the batch, not a running tally. A page reports from exactly one of the
two paths, and the first version of this counted persisted pages against
`stored` as well and doubled them, which the tests caught. A repeat
report is a no-op, so the vision pages passing through the persist loop
do not commit the same figure twice.
@Valtora
Valtora marked this pull request as ready for review July 30, 2026 21:13
@Valtora
Valtora merged commit 17480b5 into main Jul 30, 2026
19 of 20 checks passed
@Valtora
Valtora deleted the feat/visual-document-parsing branch July 30, 2026 21:15
Valtora added a commit that referenced this pull request Jul 31, 2026
#185)

The ONNX ASR engine asked onnxruntime for CUDAExecutionProvider
unconditionally. Where no GPU is attached that does not degrade to CPU:
onnxruntime-gpu loads its CUDA provider library, finds no device, and
takes the process down with SIGSEGV. A native fault raises no Python
exception, so nothing downstream can catch it and fall back.

The GPU lane normally has a device, so this stayed latent. The CPU-only
deployment documented in docs/DEPLOYMENT.md is the exposed case: it
tells the operator to drop the compose deploy block while keeping the
same onnxruntime-gpu image, which is the crashing shape exactly.

Gate the provider list on gpu_is_present(), the device-node probe
already used a few lines below to choose quantisation, so both
decisions now come from one call. Text embedding was fixed the same way
in #177 after this failure was reported in the field against the io
lane; this closes the remaining unguarded site.

Cover both with tests that assert the requested provider list rather
than only the observable behaviour, since the failure mode is a crash
that a test process cannot survive.

Refs: #183
@Valtora Valtora mentioned this pull request Jul 31, 2026
17 tasks
Valtora added a commit that referenced this pull request Jul 31, 2026
Bump docs/VERSION to 2.3.0 so the tag validates, and fill in the
release-notes template for this range.

A minor bump rather than a major: #177 is a feat with no breaking
marker, and the destructive part of it is a rebuildable search index
rather than an API or a configuration contract. That destruction leads
the Migration section regardless, because an operator who misses it
finds out when search returns nothing.

Three sections needed real work this time. Migration carries the
embedding cutover with its rebuild command and the new worker-parse
service, which is a manual compose edit rather than something the
upgrade does for you. Rollback is no longer code only, since the
previous images cannot read a 512 dimension embedding column and the
downgrade purges the index a second time. Browser-Capture Compatibility
covers the restyle rather than a support-matrix change, because capture
behaviour is unchanged but the workspace around it was rebuilt.

Highlights also has to carry #184 in full. render_release_notes.py
folds refactor into the collapsed Other changes block by design, so the
flat restyle renders as one line in the generated changelog despite
being the headline of the release.

Refs: #177, #179, #182, #184, #185
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant