feat(documents): visually parse uploads and feed them into notes (#177)

* feat(documents): add structural parsers and a vision model contract

Lays the foundation for visually parsing uploaded documents. Text-only
extraction misses most of what a deck or a report actually carries, so
parsing gains a per-format structural pass and, behind it, the ability to
send page images to a vision-capable model.

Structural extraction covers PPTX, DOCX, XLSX and CSV alongside the
existing PDF and plain-text paths. Office formats are zipped XML rather
than pixels, so this recovers slide titles, tables, speaker notes and
native chart data without any headless-office renderer: python-pptx
returns the exact values a chart was built from, which a vision model
reading a rendered chart could only estimate. Shapes are walked in
reading order rather than z-order, and Word paragraphs and tables are
read from the body XML so their interleaving survives.

Vision arrives as generate_text_from_images on the LLMBackend contract,
implemented for Anthropic, OpenAI, Gemini and Ollama. Both subscription
CLIs are wired without loosening their posture: Codex takes first-class
--image arguments, and Claude takes inline content blocks through the
Agent SDK's streaming-input mode, so allowed_tools stays empty and
max_turns stays at one. A provider that cannot accept images raises
VisionUnsupportedError, which lets a document degrade to a structural
parse rather than fail outright.

The RAG embedding model moves from all-MiniLM-L6-v2 to
jina-embeddings-v2-small-en. The old model truncated input at roughly
256 tokens, so the tail of any real page was never searchable; the new
one has an 8192-token window, which makes a whole page a single
retrieval unit. Vectors from two models are not comparable at any width,
so the migration widens the column to 512 dimensions and purges the
existing rows, and chunks now carry an embedding_version stamp mirroring
the voiceprint precedent.

document_pages stores parsed content one row per page, written as each
page completes so an interrupted parse resumes instead of repeating paid
vision calls. It is archived rather than rebuilt on restore, unlike
context_chunks, because re-embedding is free but re-parsing is not. The
MCP get_documents tool now reads those pages, which also fixes its text
reconstruction: joining overlapping chunks re-emitted fifty characters
at every boundary.

Adds GET /llm/vision-support so the upload flow can warn before a
document is parsed and silently downgraded. The result is tri-state:
only Ollama can answer definitively, via its capabilities list, and a
probe failure resolves to unknown rather than blocking an upload.

Refs: docs/USAGE.md

* feat(documents): parse on a dedicated lane and feed notes and chat

Replaces the document pipeline end to end. Parsing produced 500-character
sliding windows of PDF or plain text and reached only meeting chat; it now
produces per-page Markdown for ten formats and reaches notes generation as
well.

The parse task moves to its own Celery lane rather than sharing io. A
parse has no page cap, so one large upload can hold a worker slot for a
long time, and on the io lane that would sit beside Meeting Edge, meeting
chat and notes generation and degrade a live meeting. The lane reuses the
worker-io image, so it adds no build and no image to scan, and visual
parsing can still route through a subscription CLI.

Pages are written as each completes, so a worker restart resumes from the
first missing page instead of repeating vision calls that were already
paid for. Vision requests fan out three at a time, which cuts wall-clock
on a long document without tripping provider rate limits. A provider that
cannot accept images downgrades the whole document to a structural parse
once and records why, rather than failing every page in turn; an image
upload has no text layer to fall back on, so that case is a real error
instead of an empty success.

Documents now reach notes generation through both prompt paths, carrying
the full parsed text with no cap. That matches the transcript, which has
never been truncated either, and a model with too small a context reports
it explicitly. A document finishing after the notes were written marks
them stale so the user is prompted to regenerate; regenerating is never
automatic because it spends their quota and overwrites hand edits.

Meeting chat gains a separate retrieval budget per source. Transcript and
document chunks previously competed for one pool of five, so a strongly
matching passage could take every slot and leave an attached deck
unrepresented. Every retrieval also filters on the embedding version,
since vectors from the previous model would otherwise score as noise
ranked like relevance.

Document text is untrusted input, and visual parsing widens that: a model
transcribing a page reproduces any instruction printed on it. Both sinks
now fence it in explicit delimiters with a data-not-instructions rule,
which matters most for chat, where update_meeting_notes can overwrite the
notes document.

The upload cap rises from 20 MB to 250 MB, matched to legacy recordings,
and gains the free-disk pre-flight the restore path already had. The cap
alone never protected the volume: many uploads below it still fill a
disk. Uploads take a deep-parse flag, defaulting on, and documents can be
parsed again from the API once a vision model is available.

Restore re-indexes rather than re-parses, since document_pages is
archived: re-running the parser would spend a vision call per page to
reproduce text the backup already restored.

Refs: docs/USAGE.md, docs/ARCHITECTURE.md

* feat(frontend): surface visual parsing, live uploads and stale notes

Brings the document UI in line with the new parsing pipeline.

The upload modal accepts the seven new formats, raises its ceiling to
250 MB, and offers a visual-analysis toggle defaulted on. The toggle is
forced on and locked for image uploads, where turning it off would leave
nothing to index at all. Above 20 MB it warns that every page is sent to
the configured AI provider, so the cost is visible at the moment the
choice is made rather than after.

The documents list gains per-page progress, a page count, a re-parse
action, and the non-fatal warning a downgraded parse records. Progress
matters here because a visual parse runs for minutes on a long document
and a bare spinner reads as a hang.

A documents panel now appears alongside the live transcript and notes
while a meeting is recording or processing. The point is timing rather
than convenience: notes are generated at the end of processing, so a deck
attached during the meeting is normally parsed in time to reach them on
the first pass instead of marking them stale afterwards. It is
read-mostly by design, with delete and re-parse left to the Documents
tab, since a destructive control next to a live recording is the wrong
thing to offer mid-meeting.

Notes show a dismissible banner when a document finished parsing after
they were written, offering to regenerate and stating plainly that doing
so overwrites manual edits. Dismissal is not persisted: the banner is
advice rather than a task, and the notes are still stale after a reload.

The upload modal's accents move from blue to orange, matching the
documents view it opens from and the rest of the product.

* test(documents): cover parsing, chunking, prompts and downgrade paths

Sixty-two tests across three files, built on real files rather than
recorded strings so a library upgrade that changes extraction behaviour
fails here instead of quietly degrading uploads.

test_document_parsing covers every format end to end, including the two
results that justify the design: a deck's chart values come back exact
rather than estimated, and speaker notes survive even though they are
invisible on the rendered slide. It also pins the chunker's boundaries,
since splitting a Markdown table mid-row would make it unreadable, and
the merge rule that a rendered page replaces its text layer while a
figure description supplements it.

test_document_prompt_integration asserts both note-generating paths
render documents through the same builder. They are assembled separately
and would otherwise drift, which is the whole reason the builder is
shared. It also covers the injection fencing, including a title crafted
to break out of its own attribute.

test_document_task covers the decision logic around a parse with stub
backends, so no provider is contacted. The important distinction is
between a per-page failure, which falls back to structural text and
carries on, and VisionUnsupportedError, which condemns every remaining
page and must propagate so the document downgrades once. Provider
refusal messages are matched against transient ones as well, because
confusing the two would permanently downgrade a document over a single
flaky call.

* docs: document visual parsing, the parse lane and the embedding cutover

USAGE gains an "Attaching Documents" section covering the new formats,
the 250 MB ceiling, and where documents can be attached. It explains why
timing matters: notes are generated once, at the end of processing, so a
document attached during the meeting reaches them on the first pass while
one attached afterwards raises a regenerate prompt instead. Visual
analysis is described as the default with its opt-out, along with the
Ollama requirement to select a vision-capable model, and the note that
diagram-heavy decks parse better exported to PDF.

ARCHITECTURE describes the fourth Celery lane and adds a Document
Parsing section covering the structural pass, when visual analysis
replaces the text layer rather than supplementing it, incremental page
persistence, and the untrusted-content fencing applied to both prompt
sinks.

DEPLOYMENT documents worker-parse, including why it runs the worker-io
image rather than a new one, and records the RAG embedding cutover as a
destructive one-time migration. That entry is the important one: the
model change deletes every existing vector because vectors from two
models are not comparable, so search and meeting chat return nothing
until the rebuild task has run. It gives the command, notes that
re-indexing is free local inference, and states that pre-release
documents are re-parsed structurally rather than visually so no provider
quota is spent unasked.

* fix(models): register DocumentPage from the module that references it

The API container failed to start with "expression
'DocumentPage.page_number' failed to locate a name". Document declares its
pages relationship by string, and the class it names was only registered
through backend/models/registry.py, which several real entry points never
import: startup_canonical_cutover reaches the model graph through User
alone, and the API process reaches it through recording_public. Either
one configures its mappers with DocumentPage unknown.

The whole test suite passed because most test modules import the registry
explicitly, which registers everything and makes the gap invisible. The
new test encodes the actual invariant instead: importing a model module
must register every class its own relationships name, checked in a
subprocess per module because the SQLAlchemy class registry is
process-global and one earlier import would mask the result. It fails
without this fix and passes with it.

Also corrects the local compose worker-parse service, which pointed at
ghcr.io/valtora/nojoin-worker-io:latest while this stack builds from
source to nojoin-worker-io:local. Left as it was, the parse lane would
have silently run the last release's code.

* fix(documents): stop the embedding model segfaulting on CPU-only lanes

Document parsing died with SIGSEGV and Celery reported WorkerLostError.
The cause is not the parser: TextEmbeddingService asked ONNX Runtime for
the CUDA execution provider unconditionally, and onnxruntime-gpu does not
degrade gracefully when no device is present. It loads its CUDA provider
library, finds nothing, and takes the process down. A segfault cannot be
caught, so the existing try/except fallback to CPU never ran.

Confirmed by probe: requesting CUDA exits 139 on both worker-io and
worker-parse, with the old and the new embedding model alike, while
CPUExecutionProvider alone succeeds on both. This was therefore a latent
fault in the io lane rather than something the model swap introduced, but
routing parsing to a second GPU-less lane made it fatal and visible.

The fix uses the gpu_is_present helper that already exists for this
purpose, checking for an NVIDIA device node rather than asking a CUDA
library that may itself be the thing that is broken.
get_available_providers is no help: it reports what onnxruntime was
compiled with, not what is usable. verify_gpu_providers stays, now
covering the separate case of a GPU being present but CUDA still being
dropped for a missing shared library.

* feat(documents): rebuild the Documents tab as a table with real progress

The card grid clipped document titles at roughly twenty characters and
reported progress as a page counter that reached "7 of 7" while indexing
was still running, which reads as a hang rather than as progress.

Replaces it with a table: name, format, size, pages, status, and per-row
actions. Each row in flight carries its own progress bar, and the label
beneath it comes from a new parse_stage column rather than from the page
counter, so the phases are distinguishable -- reading pages, analysing
with AI, indexing for search. A document whose page count is not yet
known shows an indeterminate bar rather than a stalled zero. Polling
drops to two seconds, since the bar is now the primary signal that work
is still moving.

Adds file_size_bytes, taken from the byte count the upload already
returned rather than stat()-ed on read: the list is a hot path and the
file may have been removed since. Existing rows stay NULL and render as a
dash, which is honest -- stat() would report the file now, not the upload.

Warnings move below the table rather than into a status cell. They are
sentences, and a row that grows to fit one breaks the scan. The format
column reads the extension from the filename rather than the stored MIME
type, which is client-supplied and unreliable.

* fix(documents): let a stalled parse be recovered from the UI

A worker that dies mid-parse leaves the document PROCESSING forever, and
the re-parse endpoint refused PROCESSING outright, so the only way out
was editing the database. That is exactly what happened when the
embedding model segfaulted: the pages were stored and resumable, but
nothing in the product could restart the job.

A parse writes to its row on every page batch and every stage change, so
silence is a reliable signal that the task is gone rather than slow.
Re-parse is now permitted once a PROCESSING document has been quiet for
ten minutes, and the table shows it as Stalled with a pointer to the
action rather than an animated bar that never moves. The threshold is
deliberately generous: a false positive only allows a redundant parse,
while a false negative wedges the document permanently.

* fix(documents): resolve the LLM factory that the wildcard import missed

Every document downgraded to a structural parse with "no AI provider is
configured for this account", regardless of the provider actually set up.
The cause was a NameError: get_llm_backend_with_secondary was reached
through `from .constants import *`, but constants imports that factory
inside a function, so it never enters the star namespace. The broad
except then swallowed the NameError and reported it as a configuration
problem.

Imports it explicitly, and returns a reason alongside the backend so the
message on the document card matches what actually happened. Blaming the
user's settings for an internal fault sends them to fix something that
was never wrong, which is worse than saying nothing.

The regression test asserts the name is both referenced by the function
and importable for real, since appearing in the source was exactly the
condition that held while the code was broken.

* fix(llm): forward image calls through the primary/secondary chain

SecondaryLLMBackend wraps a primary and a secondary provider and forwards
each LLMBackend method explicitly, so generate_text_from_images fell
through to the base class, which raises VisionUnsupportedError by design.
Any account using the provider chain therefore downgraded every document
to a structural parse regardless of whether its provider could read
images, and the card blamed the model.

Forwarding it makes the fallback meaningful rather than incidental: a
primary that cannot accept images fails, the secondary is tried, and only
when both fail does the primary's VisionUnsupportedError surface and
downgrade the document once. That is the chain users are already shown in
Settings. supports_vision is forwarded as a tri-state: a definite yes
from either provider is a yes, a definite no from both is a no, and
anything else stays unknown and is settled by making the call.

* feat(documents): add a local OCR tier below visual analysis

Parsing degraded straight from visual analysis to a format's own text
layer, so a scanned page produced nothing whenever no vision model was
reachable. Local OCR now sits between the two: it runs on the worker,
costs nothing, sends nothing anywhere, and makes a scanned document
searchable on an install with no AI configured at all.

It is deliberately the middle tier rather than a replacement. OCR
transcribes glyphs; it cannot say what a chart shows or describe a
diagram, so visual analysis still wins wherever it produced anything, and
the card says explicitly when OCR carried the document instead. It is
also skipped on any page whose text layer is already substantial, since
it is strictly worse at text than a real text layer, and it only ever
supplements that layer rather than replacing it.

Tesseract is driven through the system binary rather than one of the
Python OCR libraries because all of them depend on onnxruntime, which
docker/Dockerfile.worker deliberately replaces with onnxruntime-gpu.
Reintroducing the CPU build alongside it is the exact conflict that
replacement exists to prevent, and it is the same class of fault that
segfaulted the parse lane. The cost is 15 MB of language data and a pure
Python wrapper. Absence is handled rather than assumed: a deployment
without the binary detects it and falls through to structural text.

Pages record which tier produced them, so a partially OCR'd document is
legible after the fact. An image upload, which has no text layer at all,
is now only an error when OCR also finds nothing, and the message
distinguishes "OCR is not installed" from "OCR found nothing".

* feat(documents): add download, backfill sizes, and state parse provenance

Three fixes to the documents table, plus a correction to what the AI is
told about its own inputs.

Downloads: a new authenticated endpoint returns the original file under
its uploaded name rather than the UUID it is stored as, and the table
gains a per-row button. The client fetches a blob through the API client
instead of using a plain link, because an anchor href cannot carry the
auth header. The filename is sanitised before it reaches
Content-Disposition, where a quote or newline would let a user-supplied
title inject a header.

Sizes: documents uploaded before the column existed showed a dash
forever. They are now backfilled from the stored file when the list is
read. Reading it from disk is accurate rather than a guess, since an
uploaded document is written once and never rewritten, so the file's size
is its upload size. A document whose file is gone stays NULL rather than
recording a misleading zero.

Layout: the name column takes the space freed by tightening the fixed
columns, since a truncated title is the one value in the row that cannot
be guessed from context.

Provenance: chunks now record which tier produced their text, and both
the chat context and the notes prompt state it. Without this a model
reading a vision transcription reports the document as "text extraction
only" -- accurate about what it received, since only text is ever sent to
chat, but wrong about the document and misleading to the user reading the
answer.

* fix(models): register the ORM graph from the one import every entry point makes

Relationships name their targets by string, so SQLAlchemy resolves them
only once every class exists. Which classes are registered was left to
each entry point's own imports, and that failed twice: DocumentPage broke
container start, and ContextChunk broke any process that reached Document
without also importing context_chunk.

Both were invisible to the test suite, because most test modules import
backend/models/registry.py directly. That registers everything and makes
a missing registration unobservable, so the suite passed while production
could not configure its mappers.

backend/core/db.py is the fix. Every entry point imports it -- nothing
touches the database without a session -- and no model imports it back,
so it registers the full graph once, with no cycle and no per-module
cross-imports to keep in step. This replaces the DocumentPage import
added to document.py earlier, which treated the symptom.

The test now asserts the property rather than a list: each real entry
point is imported in its own subprocess, since the class registry is
process-global and one earlier import would mask the result, and then
configure_mappers must succeed. Removing the chokepoint fails all five
cases.

* fix(documents): report parse stalls from the server, not the browser clock

A running parse showed as Stalled while the re-parse it invited was
refused with "this document is already being parsed". The two disagreed
because they were computing the same thing in different places: the
client subtracted updated_at from Date.now, and the server compared the
same column in UTC.

updated_at is stored and serialised without a timezone, so JavaScript
parses it as local time. Any browser outside UTC therefore misjudges the
age by its whole offset, and at an offset above the ten minute threshold
every in-flight parse reads as stalled and never recovers. The
development host runs on UTC, which is why this never appeared here.

The flag is now computed server-side and sent as is_stalled, so the
browser does no clock arithmetic at all and the label beside the button
cannot contradict what the button will do. The endpoint and the
serializer call one function rather than holding separate copies.

Also makes an active parse look active. Accepting a re-parse resets the
page counters immediately, so the bar starts from zero instead of
rendering the previous run's completed count and appearing finished
before it began; a parse whose page count is not yet known shows an
indeterminate sweep rather than a bar sitting at zero, which reads as
stuck; and the stage line carries a spinner.

* feat(documents): report parse progress per page rather than per batch

The bar jumped straight from 0 to the full page count. Pages are
persisted in batches of eight, and pages_parsed was written once per
batch, so any document of eight pages or fewer showed a single step and
nothing in between.

Progress is now decoupled from persistence. The vision pass calls back as
each page settles, successfully or not, and pages that never reach that
tier report as they are persisted, which matters most for OCR since it
runs in the persist loop and can take seconds per page. The callback
fires from the as_completed loop, which runs in the calling thread, so it
may touch the session; the worker threads never do.

Counting is a set of settled page numbers against a baseline taken before
the batch, not a running tally. A page reports from exactly one of the
two paths, and the first version of this counted persisted pages against
`stored` as well and doubled them, which the tests caught. A repeat
report is a no-op, so the vision pages passing through the persist loop
do not commit the same figure twice.

Author: Valtora

backend/alembic/versions/a4f1e9c72b58_visual_document_parsing.py

@@ -0,0 +1,198 @@
+"""Visual document parsing: pages, parse state, and the 512-dim embedding cutover
+
+Revision ID: a4f1e9c72b58
+Revises: e1a7c93b8d24
+Create Date: 2026-07-30 00:00:00.000000
+
+Three related changes that have to land together:
+
+1. ``document_pages`` -- parsed content stored per page/slide/sheet, written
+   incrementally so an interrupted parse resumes instead of repeating vision
+   calls. Also ends the practice of reconstructing a document's text by
+   concatenating its overlapping chunks, which re-emitted every overlap.
+
+2. Parse state on ``documents`` -- requested mode, non-fatal warning, and
+   progress counters.
+
+3. The embedding cutover. ``context_chunks.embedding`` moves from 384 to 512
+   dimensions because the RAG model changes from all-MiniLM-L6-v2 (roughly a
+   256-token window, so the tail of any real page was never searchable) to
+   jina-embeddings-v2-small-en (8192 tokens, so a whole page embeds intact).
+
+   THIS IS A DESTRUCTIVE CUTOVER. Vectors from two different models are not
+   comparable at any width, so every existing row is deleted rather than
+   migrated -- there is no arithmetic that converts one to the other. Search
+   and meeting chat return nothing until the post-upgrade rebuild sweep
+   re-indexes each recording. The sweep is dispatched by
+   ``backend.worker.tasks.rebuild_text_embeddings_task``.
+"""
+
+from typing import Sequence, Union
+
+import pgvector
+import sqlalchemy as sa
+import sqlmodel
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "a4f1e9c72b58"
+down_revision: Union[str, Sequence[str], None] = "e1a7c93b8d24"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Upgrade schema."""
+    op.create_table(
+        "document_pages",
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), nullable=False),
+        sa.Column("id", sa.BigInteger(), nullable=False),
+        sa.Column("document_id", sa.BigInteger(), nullable=True),
+        sa.Column("page_number", sa.Integer(), nullable=False),
+        sa.Column("title", sqlmodel.sql.sqltypes.AutoString(), nullable=True),
+        sa.Column("content", sa.Text(), nullable=False),
+        sa.Column(
+            "parse_mode",
+            sa.Enum("STRUCTURAL", "VISUAL", name="pageparsemode"),
+            nullable=False,
+        ),
+        sa.Column("error_message", sa.Text(), nullable=True),
+        sa.ForeignKeyConstraint(["document_id"], ["documents.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint(
+            "document_id", "page_number", name="uq_document_page_number"
+        ),
+    )
+    op.create_index(
+        op.f("ix_document_pages_document_id"),
+        "document_pages",
+        ["document_id"],
+        unique=False,
+    )
+    op.create_index(
+        op.f("ix_document_pages_page_number"),
+        "document_pages",
+        ["page_number"],
+        unique=False,
+    )
+
+    # --- parse state on documents ---
+    parse_mode = sa.Enum("VISUAL", "STRUCTURAL", name="documentparsemode")
+    parse_mode.create(op.get_bind(), checkfirst=True)
+    op.add_column(
+        "documents",
+        sa.Column(
+            "parse_mode",
+            parse_mode,
+            nullable=False,
+            server_default="VISUAL",
+        ),
+    )
+    op.add_column("documents", sa.Column("parse_warning", sa.Text(), nullable=True))
+    op.add_column("documents", sa.Column("page_count", sa.Integer(), nullable=True))
+    op.add_column(
+        "documents",
+        sa.Column("pages_parsed", sa.Integer(), nullable=False, server_default="0"),
+    )
+
+    # Notes staleness. Documents now feed notes generation, so a document that
+    # finishes parsing after the notes were written leaves them incomplete.
+    op.add_column(
+        "transcripts",
+        sa.Column(
+            "notes_stale_documents",
+            sa.Boolean(),
+            nullable=False,
+            server_default=sa.false(),
+        ),
+    )
+
+    # --- embedding cutover ---
+    # Purge before altering: a 384-dimension value cannot be cast to a
+    # 512-dimension one, so the ALTER only succeeds on an empty column, and
+    # keeping the rows would be worse than losing them (they would score as
+    # noise against every query).
+    op.execute("DELETE FROM context_chunks")
+
+    op.add_column(
+        "context_chunks",
+        sa.Column("document_page_id", sa.BigInteger(), nullable=True),
+    )
+    op.create_foreign_key(
+        "fk_context_chunks_document_page_id",
+        "context_chunks",
+        "document_pages",
+        ["document_page_id"],
+        ["id"],
+        ondelete="CASCADE",
+    )
+    op.create_index(
+        op.f("ix_context_chunks_document_page_id"),
+        "context_chunks",
+        ["document_page_id"],
+        unique=False,
+    )
+
+    op.add_column(
+        "context_chunks",
+        sa.Column(
+            "embedding_version", sa.Integer(), nullable=False, server_default="2"
+        ),
+    )
+    op.create_index(
+        op.f("ix_context_chunks_embedding_version"),
+        "context_chunks",
+        ["embedding_version"],
+        unique=False,
+    )
+
+    # Drop and re-add rather than ALTER TYPE: there is no cast between vector
+    # widths, and with the table emptied above there is nothing to preserve.
+    # Safe here only because no ivfflat/hnsw index exists on this column.
+    op.drop_column("context_chunks", "embedding")
+    op.add_column(
+        "context_chunks",
+        sa.Column(
+            "embedding", pgvector.sqlalchemy.vector.VECTOR(dim=512), nullable=True
+        ),
+    )
+
+
+def downgrade() -> None:
+    """Downgrade schema.
+
+    Symmetrically destructive: going back to a 384-dimension column discards
+    every vector the new model produced, for the same incomparability reason.
+    """
+    op.execute("DELETE FROM context_chunks")
+    op.drop_column("context_chunks", "embedding")
+    op.add_column(
+        "context_chunks",
+        sa.Column(
+            "embedding", pgvector.sqlalchemy.vector.VECTOR(dim=384), nullable=True
+        ),
+    )
+    op.drop_index(
+        op.f("ix_context_chunks_embedding_version"), table_name="context_chunks"
+    )
+    op.drop_column("context_chunks", "embedding_version")
+    op.drop_index(
+        op.f("ix_context_chunks_document_page_id"), table_name="context_chunks"
+    )
+    op.drop_constraint(
+        "fk_context_chunks_document_page_id", "context_chunks", type_="foreignkey"
+    )
+    op.drop_column("context_chunks", "document_page_id")
+
+    op.drop_column("transcripts", "notes_stale_documents")
+    op.drop_column("documents", "pages_parsed")
+    op.drop_column("documents", "page_count")
+    op.drop_column("documents", "parse_warning")
+    op.drop_column("documents", "parse_mode")
+    sa.Enum(name="documentparsemode").drop(op.get_bind(), checkfirst=True)
+
+    op.drop_index(op.f("ix_document_pages_page_number"), table_name="document_pages")
+    op.drop_index(op.f("ix_document_pages_document_id"), table_name="document_pages")
+    op.drop_table("document_pages")
+    sa.Enum(name="pageparsemode").drop(op.get_bind(), checkfirst=True)

backend/alembic/versions/b7d3f1a02c94_add_document_size_and_parse_stage.py

@@ -0,0 +1,46 @@
+"""Add document file size and parse stage
+
+Revision ID: b7d3f1a02c94
+Revises: a4f1e9c72b58
+Create Date: 2026-07-30 00:00:00.000000
+
+Two columns behind the documents table view.
+
+``file_size_bytes`` is recorded at upload rather than stat()-ed on read: the
+documents list is a hot path, and the file may have been removed from disk
+since. Existing rows are left NULL and the UI omits the size for them, which is
+honest -- backfilling by stat() would report the current file, not the upload.
+
+``parse_stage`` names the phase a running parse is in. Page counters alone are
+ambiguous: they reach "7 of 7" while indexing is still running, which reads as a
+hang rather than as progress.
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+import sqlmodel
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "b7d3f1a02c94"
+down_revision: Union[str, Sequence[str], None] = "a4f1e9c72b58"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Upgrade schema."""
+    op.add_column(
+        "documents", sa.Column("file_size_bytes", sa.BigInteger(), nullable=True)
+    )
+    op.add_column(
+        "documents",
+        sa.Column("parse_stage", sqlmodel.sql.sqltypes.AutoString(), nullable=True),
+    )
+
+
+def downgrade() -> None:
+    """Downgrade schema."""
+    op.drop_column("documents", "parse_stage")
+    op.drop_column("documents", "file_size_bytes")

backend/alembic/versions/c9e4a25b71f3_add_ocr_page_parse_mode.py

@@ -0,0 +1,43 @@
+"""Add the OCR value to the page parse mode
+
+Revision ID: c9e4a25b71f3
+Revises: b7d3f1a02c94
+Create Date: 2026-07-30 00:00:00.000000
+
+Local OCR becomes the middle tier between visual analysis and a format's own
+text layer, so a page needs to be able to record that it came from OCR.
+
+``ALTER TYPE ... ADD VALUE`` cannot run inside a transaction block on older
+PostgreSQL, and Alembic wraps migrations in one, so this commits first. The
+statement is idempotent via IF NOT EXISTS, which matters because a partially
+applied migration would otherwise be unrepeatable.
+
+There is no downgrade. PostgreSQL cannot drop a value from an enum type, and
+rebuilding the type would mean rewriting every row that references it -- far
+more destructive than the forward change. Rows written as OCR simply become
+unreachable through the Python enum on an older build.
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "c9e4a25b71f3"
+down_revision: Union[str, Sequence[str], None] = "b7d3f1a02c94"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Upgrade schema."""
+    op.execute("COMMIT")
+    op.execute("ALTER TYPE pageparsemode ADD VALUE IF NOT EXISTS 'OCR'")
+
+
+def downgrade() -> None:
+    """Downgrade schema.
+
+    Intentionally a no-op: PostgreSQL offers no way to remove an enum value, and
+    recreating the type to drop one would rewrite the table.
+    """