diff --git a/CHANGELOG.md b/CHANGELOG.md index 7398be4..48e9367 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,85 @@ Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] ### Added +- **Streaming narrative generation via Server-Sent Events** for the web + builder. `POST /generate` now runs the deterministic prep phase + (parse GPX + load_photos + persist `pending.json`), wipes the raw + upload via the existing `BackgroundTask`, and returns a + `generating.html.j2` page that opens an `EventSource` to the new + `GET /generate/{slug}/stream` endpoint. The SSE endpoint runs the + LLM call with `AnthropicClient.complete_stream`, emits one `chunk` + event per text delta, a `status` event when phases change + (`writing` → `regenerating` → `rendering`), and a terminal `done` + event with the redirect URL once the HTML has been rendered. If the + first response does not parse as JSON, the orchestrator retries once + (emitting a `regenerating` status); two failures land an `error` + event without consuming further LLM calls. Pending state is unlinked + on success so a refresh of the generating page cannot re-trigger the + paid call (and a follow-up `GET /generate/{slug}/stream` returns 404). + - New `AnthropicClient.complete_stream` mirrors `complete` but uses + `messages.stream(...)` and yields each `text_stream` delta. Same + rate-limit retry policy as `complete`, but only before the first + chunk lands — a mid-stream error surfaces as `LLMResponseError` + rather than retrying (re-yielding chunks the consumer already + received would corrupt the SSE stream). + - New `generate_narrative_stream` in `trailstory.llm.narrative` + yields `NarrativeStreamChunk` / `NarrativeStreamRetry` / + `NarrativeStreamComplete` events. The validated `NarrativeOutput` + rides on the terminal event, so the SSE endpoint can build the + `Memory`, render the HTML, and persist final `state.json` in one + pass. Cache is intentionally bypassed for streaming runs — the + user is watching tokens land, an instant cached return would be + jarring; CLI / `generate_narrative` keeps the cache. + - New `web.pipeline.prepare_pipeline` and + `web.pipeline.stream_pipeline` split the previous `run_pipeline` + into the prep + stream halves. `pending.json` (alongside + `state.json` in `output/`) holds the parsed inputs the SSE + endpoint resumes from. +- **"Save for Instagram" button on every rendered memory page** + (`templates/styles/{editorial,log,encyclopedia}.html.j2`). The button + POSTs to `/memory/{slug}/carousel`, fetches each slide URL as a + `Blob`, and either calls `navigator.share({files: ...})` (iOS + Safari path → "Save N Images" share sheet → two-tap to Instagram) + or renders desktop fallback download links with the + `download` attribute. The carousel slide route now sets + `Content-Disposition: attachment; filename="-NN_role.jpg"` so + desktop clicks save to `~/Downloads` with a meaningful, slug- + namespaced name. The button is wired in JavaScript only — no new + Python routes — so existing carousel infrastructure + (`render_instagram_carousel`, `POST /memory/{slug}/carousel`) is + reused. +- **Privacy page lifecycle polish** (`web/templates/privacy.html.j2`). + The "what we do with your upload" section is now an explicit + six-step lifecycle — upload → parse/resize → narrative call → HTML + render → raw upload deletion → workspace expiry — with a + "Verify it yourself" block that links to specific line ranges in + `web/storage.py` (`cleanup_inputs`, `sweep_expired`) and `web/app.py` + (`_run_sweeper`) on GitHub. The privacy link in the form + (`web/templates/landing.html.j2`) opens in a new tab and reads + "How we handle your photos →" so the user can audit without losing + their upload state. + +### Changed +- `POST /generate` no longer 303-redirects to the memory page. It now + returns the new generating page (HTTP 200) with a `data-slug` + attribute and an inline `EventSource` listener on + `/generate/{slug}/stream`. The browser navigates to `/memory/{slug}` + itself once the SSE `done` event arrives. Existing 4xx behaviour for + missing / oversized / unsupported uploads is unchanged — those still + surface synchronously from the prep phase. +- `web/dev.py::make_fake_client_factory` now stubs `complete_stream` + alongside `complete` so `make web-dev` exercises the SSE flow + end-to-end without paying for API calls. The stream is the same + fixture narrative split into ~16 chunks. +- `web/storage.py` adds `Workspace.pending_state_path` for the + intermediate JSON that `prepare_pipeline` writes and + `stream_pipeline` consumes. The retention sweeper continues to + garbage-collect everything older than `RETENTION_SECONDS` (30 + minutes by default). +- The unused synchronous `web.pipeline.run_pipeline` was removed — + it was superseded by `prepare_pipeline + stream_pipeline` and only + the streaming flow is now wired into the routes. + - **FastAPI web builder** (`web/`). Mobile-first, privacy-first, no accounts, no DB. Six endpoints: `GET /` (landing + builder form), `POST /generate` (multipart `gpx` + `photos[]` + `description` + diff --git a/templates/styles/editorial.html.j2 b/templates/styles/editorial.html.j2 index 1f2ac7a..e91c3e7 100644 --- a/templates/styles/editorial.html.j2 +++ b/templates/styles/editorial.html.j2 @@ -222,11 +222,28 @@
WhatsApp + +

+ + diff --git a/templates/styles/encyclopedia.html.j2 b/templates/styles/encyclopedia.html.j2 index 2ba7e4a..298df12 100644 --- a/templates/styles/encyclopedia.html.j2 +++ b/templates/styles/encyclopedia.html.j2 @@ -315,12 +315,44 @@
WhatsApp + +

+ + diff --git a/templates/styles/log.html.j2 b/templates/styles/log.html.j2 index 3b01001..e783492 100644 --- a/templates/styles/log.html.j2 +++ b/templates/styles/log.html.j2 @@ -266,11 +266,37 @@
WhatsApp + +

+ + diff --git a/tests/golden/test-render-editorial.html b/tests/golden/test-render-editorial.html index 4253a7e..d38932c 100644 --- a/tests/golden/test-render-editorial.html +++ b/tests/golden/test-render-editorial.html @@ -221,11 +221,28 @@

test-render-editorial
+ + diff --git a/tests/golden/test-render-encyclopedia.html b/tests/golden/test-render-encyclopedia.html index f634a22..7f7a489 100644 --- a/tests/golden/test-render-encyclopedia.html +++ b/tests/golden/test-render-encyclopedia.html @@ -347,12 +347,44 @@

test-render-encyclopedia
+ + diff --git a/tests/golden/test-render-log.html b/tests/golden/test-render-log.html index 22f15ab..0e31415 100644 --- a/tests/golden/test-render-log.html +++ b/tests/golden/test-render-log.html @@ -301,11 +301,37 @@

test-render-log
+ + diff --git a/tests/test_client.py b/tests/test_client.py index c3beec9..b109bfa 100644 --- a/tests/test_client.py +++ b/tests/test_client.py @@ -11,6 +11,7 @@ from __future__ import annotations +from collections.abc import Iterator from typing import Any from unittest.mock import MagicMock @@ -298,3 +299,118 @@ def test_constructor_does_not_store_plaintext_secret() -> None: client, _ = _build_client() # The unwrapped key must not appear anywhere on the instance. assert not any(API_KEY.get_secret_value() == v for v in vars(client).values()) + + +# ── streaming ──────────────────────────────────────────────────────────────── + + +def _make_stream_manager(chunks: list[str]) -> MagicMock: + """Build a fake context manager mirroring ``messages.stream(...)``. + + The real SDK returns a ``MessageStreamManager`` whose ``__enter__`` + yields a ``MessageStream`` exposing ``text_stream``. The mock keeps + that shape so the client code under test treats it like the real + thing. + """ + stream = MagicMock() + stream.text_stream = iter(chunks) + manager = MagicMock() + manager.__enter__ = MagicMock(return_value=stream) + manager.__exit__ = MagicMock(return_value=None) + return manager + + +def test_complete_stream_yields_chunks_in_order() -> None: + chunks = ["hello ", "world ", "from ", "claude"] + sdk = MagicMock() + sdk.messages.stream.return_value = _make_stream_manager(chunks) + client, _ = _build_client(sdk_mock=sdk) + + out = list(client.complete_stream("p", "s")) + assert out == chunks + # Same model / system / message shape as the non-streaming path. + kwargs = sdk.messages.stream.call_args.kwargs + assert kwargs["model"] == DEFAULT_MODEL + assert kwargs["system"] == "s" + assert kwargs["messages"] == [{"role": "user", "content": "p"}] + + +def test_complete_stream_skips_empty_chunks() -> None: + """The SDK occasionally yields empty strings between deltas; they + should not propagate to the consumer.""" + sdk = MagicMock() + sdk.messages.stream.return_value = _make_stream_manager(["a", "", "b", "", "c"]) + client, _ = _build_client(sdk_mock=sdk) + + assert list(client.complete_stream("p", "s")) == ["a", "b", "c"] + + +def test_complete_stream_raises_on_empty_stream() -> None: + sdk = MagicMock() + sdk.messages.stream.return_value = _make_stream_manager([]) + client, _ = _build_client(sdk_mock=sdk) + + with pytest.raises(LLMResponseError, match="empty"): + list(client.complete_stream("p", "s")) + + +def test_complete_stream_retries_rate_limit_before_first_chunk() -> None: + """A RateLimitError raised before any chunk lands triggers a retry.""" + chunks = ["ok " * 30] + sdk = MagicMock() + sdk.messages.stream.side_effect = [ + _make_rate_limit_error(), + _make_rate_limit_error(), + _make_stream_manager(chunks), + ] + client, _ = _build_client(sdk_mock=sdk) + + out = list(client.complete_stream("p", "s")) + assert out == chunks + assert sdk.messages.stream.call_count == 3 + + +def test_complete_stream_raises_retry_exhausted_after_max_attempts() -> None: + sdk = MagicMock() + sdk.messages.stream.side_effect = _make_rate_limit_error() + client, _ = _build_client(sdk_mock=sdk) + + with pytest.raises(LLMRetryExhaustedError): + list(client.complete_stream("p", "s")) + assert sdk.messages.stream.call_count == DEFAULT_MAX_RETRIES + + +def test_complete_stream_translates_status_error() -> None: + sdk = MagicMock() + sdk.messages.stream.side_effect = _make_status_error(500, "boom") + client, _ = _build_client(sdk_mock=sdk) + + with pytest.raises(LLMResponseError, match="500"): + list(client.complete_stream("p", "s")) + assert sdk.messages.stream.call_count == 1 + + +def test_complete_stream_does_not_retry_after_yielding_chunks() -> None: + """If chunks have been yielded and the SDK then raises, we surface + the error rather than retrying — the consumer has already received + the bytes and a re-attempt would corrupt the stream.""" + + def _raising_chunks() -> Iterator[str]: + yield "first chunk" + raise _make_rate_limit_error() + + stream = MagicMock() + stream.text_stream = _raising_chunks() + manager = MagicMock() + manager.__enter__ = MagicMock(return_value=stream) + manager.__exit__ = MagicMock(return_value=None) + sdk = MagicMock() + sdk.messages.stream.return_value = manager + + client, _ = _build_client(sdk_mock=sdk) + gen = client.complete_stream("p", "s") + assert next(gen) == "first chunk" + with pytest.raises(LLMResponseError, match="mid-stream"): + list(gen) + # Exactly one stream call — no retry after a yield. + assert sdk.messages.stream.call_count == 1 diff --git a/tests/test_instagram_button.py b/tests/test_instagram_button.py new file mode 100644 index 0000000..8f19580 --- /dev/null +++ b/tests/test_instagram_button.py @@ -0,0 +1,286 @@ +"""Tests for the "Save for Instagram" button on the rendered memory page. + +The button is part of every style template (editorial / log / +encyclopedia). On click, the page POSTs to ``/memory/{slug}/carousel``, +fetches each slide URL as a Blob, and either calls ``navigator.share`` +with the files (iOS Safari path) or renders fallback download links +(desktop). The Python-side contract these tests exercise: + +* The button renders into all three style templates. +* The button carries the slug as ``data-slug`` so the JS can build the + POST URL without templating it inline. +* ``POST /memory/{slug}/carousel`` returns exactly N+2 slides for N + selected photos (title + N + closing quote). +* Each slide is fetchable, has a JPEG content type, and a download- + friendly Content-Disposition header for the desktop fallback. + +The tests mock the LLM to avoid paying for narrative calls. +""" + +from __future__ import annotations + +import json +import re +from collections.abc import Iterator +from pathlib import Path +from unittest.mock import MagicMock + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient +from pydantic import SecretStr + +from trailstory.config import Settings +from trailstory.llm.client import AnthropicClient +from web.app import create_app +from web.storage import RETENTION_SECONDS, Storage + +FIXTURES = Path(__file__).parent / "fixtures" +SAMPLE_GPX = FIXTURES / "sample.gpx" +SAMPLE_PHOTOS = FIXTURES / "sample_photos" + +_SLUG_RE = re.compile(r'data-slug="([0-9a-f]{12})"') + + +# ── fixtures ───────────────────────────────────────────────────────────────── + + +def _settings() -> Settings: + return Settings( # type: ignore[call-arg] + anthropic_api_key=SecretStr("sk-test-fake"), + model="claude-opus-4-7-test", + ) + + +def _valid_response_json(n_photos: int = 5) -> str: + return json.dumps( + { + "schema_version": 2, + "title": { + "en": "Above the fog line", + "ru": "Над линией тумана", + "de": "Über der Nebelgrenze", + }, + "subtitle": { + "en": "A morning above the cloud sea", + "ru": "Утро над морем облаков", + "de": "Ein Morgen über dem Wolkenmeer", + }, + "paragraphs": { + "en": ["First light.", "Saddle. Cloud thinning."], + "ru": ["Первые лучи.", "Седловина. Облака редеют."], + "de": ["Erstes Licht.", "Sattel. Wolke lichtet sich."], + }, + "pull_quote": { + "en": "The fog cleared just as we reached the ridge.", + "ru": "Туман рассеялся как раз когда мы вышли на хребет.", + "de": "Der Nebel lichtete sich, gerade als wir den Grat erreichten.", + }, + "milestone": { + "en": "First mountain hike", + "ru": "Первый горный поход", + "de": "Erste Bergwanderung", + }, + "selected_photo_indices": list(range(n_photos)), + } + ) + + +def _stream_chunks(payload: str, *, parts: int = 6) -> list[str]: + if not payload: + return [""] + step = max(1, len(payload) // parts) + return [payload[i : i + step] for i in range(0, len(payload), step)] + + +def _make_client(*, response: str | None = None) -> MagicMock: + body = response if response is not None else _valid_response_json() + fake = MagicMock(spec=AnthropicClient) + fake.model = "claude-opus-4-7-test" + fake.complete.return_value = body + fake.complete_stream.side_effect = lambda *_a, **_kw: iter(_stream_chunks(body)) + return fake + + +def _read_sample_photos(limit: int) -> list[tuple[str, bytes, str]]: + out: list[tuple[str, bytes, str]] = [] + for path in sorted(SAMPLE_PHOTOS.iterdir()): + if path.suffix.lower() in (".jpg", ".jpeg"): + out.append((path.name, path.read_bytes(), "image/jpeg")) + if len(out) >= limit: + break + return out + + +def _generate_files(n_photos: int) -> list[tuple[str, tuple[str, bytes, str]]]: + files: list[tuple[str, tuple[str, bytes, str]]] = [ + ("gpx", ("track.gpx", SAMPLE_GPX.read_bytes(), "application/gpx+xml")), + ] + for name, data, ctype in _read_sample_photos(n_photos): + files.append(("photos", (name, data, ctype))) + return files + + +def _slug_from_generating_page(html: str) -> str: + match = _SLUG_RE.search(html) + assert match is not None, "generating page is missing the data-slug attribute" + return match.group(1) + + +def _drain_stream(client: TestClient, slug: str) -> list[str]: + """Drain the SSE stream and return the list of event names.""" + with client.stream("GET", f"/generate/{slug}/stream") as resp: + assert resp.status_code == 200 + text = b"".join(resp.iter_bytes()).decode("utf-8") + names: list[str] = [] + for block in text.split("\n\n"): + for line in block.splitlines(): + if line.startswith("event:"): + names.append(line[len("event:") :].strip()) + return names + + +@pytest.fixture +def app_factory( + tmp_path: Path, +) -> Iterator[FastAPI]: + storage = Storage(root=tmp_path / "trailstory-web", retention_seconds=RETENTION_SECONDS) + fake = _make_client() + app = create_app( + settings=_settings(), + storage=storage, + client_factory=lambda: fake, + enable_sweeper=False, + ) + yield app + + +@pytest.fixture +def app_factory_4_photos(tmp_path: Path) -> Iterator[FastAPI]: + storage = Storage(root=tmp_path / "trailstory-web", retention_seconds=RETENTION_SECONDS) + fake = _make_client(response=_valid_response_json(n_photos=4)) + app = create_app( + settings=_settings(), + storage=storage, + client_factory=lambda: fake, + enable_sweeper=False, + ) + yield app + + +def _generate_and_render( + app: FastAPI, n_photos: int = 5, style: str = "editorial" +) -> tuple[TestClient, str]: + """Run the prep + SSE + render flow, return the (client, slug).""" + client = TestClient(app) + response = client.post( + "/generate", + data={"description": "x", "style": style}, + files=_generate_files(n_photos), + ) + assert response.status_code == 200, response.text + slug = _slug_from_generating_page(response.text) + names = _drain_stream(client, slug) + assert names[-1] == "done", names + return client, slug + + +# ── button rendering ───────────────────────────────────────────────────────── + + +@pytest.mark.parametrize("style", ["editorial", "log", "encyclopedia"]) +def test_save_for_instagram_button_renders_in_every_style(tmp_path: Path, style: str) -> None: + """All three style templates ship the button so the user gets it + whichever visual treatment they pick.""" + storage = Storage(root=tmp_path / "trailstory-web", retention_seconds=RETENTION_SECONDS) + fake = _make_client() + app = create_app( + settings=_settings(), + storage=storage, + client_factory=lambda: fake, + enable_sweeper=False, + ) + client, slug = _generate_and_render(app, style=style) + page = client.get(f"/memory/{slug}").text + + assert "Save for Instagram" in page + assert f'data-slug="{slug}"' in page + # The carousel POST URL is built client-side from data-slug + a + # constant prefix. Lock the prefix so a refactor that breaks the + # button doesn't pass tests silently. + assert "/memory/" in page and "/carousel" in page + + +def test_button_targets_navigator_share_with_files(app_factory: FastAPI) -> None: + """The two-tap iOS flow relies on ``navigator.canShare({files})``; + the desktop fallback uses download links. Both code paths must be + present in the rendered page.""" + client, slug = _generate_and_render(app_factory) + page = client.get(f"/memory/{slug}").text + + # iOS Safari path. + assert "navigator.canShare" in page + assert "navigator.share" in page + # Desktop fallback. + assert "showFallbackLinks" in page + # Each download link sets the ``download`` attribute so clicks save + # rather than navigate. + assert "setAttribute('download'" in page + + +# ── carousel POST shape ────────────────────────────────────────────────────── + + +def test_carousel_post_returns_n_plus_two_slides(app_factory: FastAPI) -> None: + """For 5 selected photos the carousel is title + 5 + quote = 7 slides.""" + client, slug = _generate_and_render(app_factory, n_photos=5) + + res = client.post(f"/memory/{slug}/carousel") + assert res.status_code == 200 + payload = res.json() + assert "slides" in payload + slides = payload["slides"] + assert len(slides) == 7 + # Slug-namespaced URLs only. + for url in slides: + assert url.startswith(f"/memory/{slug}/carousel/") + + +def test_carousel_post_returns_six_slides_for_four_photos( + app_factory_4_photos: FastAPI, +) -> None: + """Smaller hike, smaller carousel — the count tracks selected photos.""" + client, slug = _generate_and_render(app_factory_4_photos, n_photos=4) + + res = client.post(f"/memory/{slug}/carousel") + assert res.status_code == 200 + slides = res.json()["slides"] + assert len(slides) == 6 # title + 4 + quote + + +def test_carousel_slide_has_jpeg_content_type_and_attachment_disposition( + app_factory: FastAPI, +) -> None: + """Every slide must serve as a JPEG with a download-friendly + Content-Disposition so the desktop fallback link saves with a + meaningful filename instead of opening inline.""" + client, slug = _generate_and_render(app_factory) + slides = client.post(f"/memory/{slug}/carousel").json()["slides"] + + for slide_url in slides: + res = client.get(slide_url) + assert res.status_code == 200 + assert res.headers["content-type"].startswith("image/jpeg") + disposition = res.headers.get("content-disposition", "") + assert "attachment" in disposition + assert slug in disposition # slug-namespaced filename + + +def test_carousel_post_idempotent(app_factory: FastAPI) -> None: + """Two POSTs in a row produce the same slide count — the renderer + overwrites in place rather than appending or 4xxing.""" + client, slug = _generate_and_render(app_factory) + + first = client.post(f"/memory/{slug}/carousel").json() + second = client.post(f"/memory/{slug}/carousel").json() + assert first == second diff --git a/tests/test_narrative.py b/tests/test_narrative.py index 20ff01d..58ea926 100644 --- a/tests/test_narrative.py +++ b/tests/test_narrative.py @@ -19,7 +19,14 @@ LLMResponseError, LLMRetryExhaustedError, ) -from trailstory.llm.narrative import NarrativeGenerationError, generate_narrative +from trailstory.llm.narrative import ( + NarrativeGenerationError, + NarrativeStreamChunk, + NarrativeStreamComplete, + NarrativeStreamRetry, + generate_narrative, + generate_narrative_stream, +) from trailstory.llm.prompts import ( USER_NARRATIVE_RETRY_SUFFIX, USER_NARRATIVE_TEMPLATE, @@ -367,3 +374,106 @@ def test_generate_narrative_supplies_every_template_placeholder() -> None: leftover = re.findall(r"\{[a-zA-Z_][a-zA-Z0-9_]*\}", sent) assert leftover == [], f"unfilled placeholders {leftover}; expected none of {expected}" + + +# ── streaming variant ──────────────────────────────────────────────────────── + + +def _stream_client(*responses: list[str] | Exception) -> MagicMock: + """Build a mocked client whose ``.complete_stream`` yields each list.""" + mock = MagicMock(spec=AnthropicClient) + + def _side_effect(*_a: object, **_kw: object) -> object: + item = mock._stream_responses.pop(0) + if isinstance(item, Exception): + raise item + return iter(item) + + mock._stream_responses = list(responses) + mock.complete_stream.side_effect = _side_effect + mock.model = "claude-opus-4-7-test" + return mock + + +def _split(payload: str, parts: int = 6) -> list[str]: + step = max(1, len(payload) // parts) + return [payload[i : i + step] for i in range(0, len(payload), step)] + + +def test_generate_narrative_stream_yields_chunks_and_terminal_event() -> None: + chunks = _split(_valid_response_json()) + client = _stream_client(chunks) + + events = list(generate_narrative_stream(_hike_input(), _gpx_stats(), _photos(), client=client)) + + chunk_events = [e for e in events if isinstance(e, NarrativeStreamChunk)] + complete_events = [e for e in events if isinstance(e, NarrativeStreamComplete)] + assert len(chunk_events) == len(chunks) + assert "".join(c.text for c in chunk_events) == "".join(chunks) + assert len(complete_events) == 1 + narrative = complete_events[0].narrative + assert isinstance(narrative, NarrativeOutput) + assert narrative.title.en == "Above the fog line" + + +def test_generate_narrative_stream_retries_on_unparseable_first_attempt() -> None: + chunks_bad = _split("this is not json", parts=3) + chunks_good = _split(_valid_response_json()) + client = _stream_client(chunks_bad, chunks_good) + + events = list(generate_narrative_stream(_hike_input(), _gpx_stats(), _photos(), client=client)) + + retries = [e for e in events if isinstance(e, NarrativeStreamRetry)] + completes = [e for e in events if isinstance(e, NarrativeStreamComplete)] + assert len(retries) == 1 + assert len(completes) == 1 + # Both attempts streamed — chunks from each appear. + chunk_text = "".join(e.text for e in events if isinstance(e, NarrativeStreamChunk)) + assert "this is not json" in chunk_text + assert "Above the fog line" in chunk_text + + +def test_generate_narrative_stream_raises_on_double_failure() -> None: + bad = _split("still not json", parts=3) + client = _stream_client(bad, bad) + + with pytest.raises(NarrativeGenerationError, match="non-JSON"): + list(generate_narrative_stream(_hike_input(), _gpx_stats(), _photos(), client=client)) + + +def test_generate_narrative_stream_strips_markdown_fences() -> None: + fenced = "```json\n" + _valid_response_json() + "\n```" + client = _stream_client(_split(fenced)) + + events = list(generate_narrative_stream(_hike_input(), _gpx_stats(), _photos(), client=client)) + # Single attempt — no retry — and the narrative parses cleanly. + completes = [e for e in events if isinstance(e, NarrativeStreamComplete)] + assert len(completes) == 1 + assert completes[0].narrative.title.en == "Above the fog line" + + +def test_generate_narrative_stream_rejects_empty_photo_list() -> None: + client = _stream_client() + with pytest.raises(NarrativeGenerationError, match="at least one photo"): + list(generate_narrative_stream(_hike_input(), _gpx_stats(), [], client=client)) + client.complete_stream.assert_not_called() + + +def test_generate_narrative_stream_translates_llm_error() -> None: + client = _stream_client(LLMResponseError("boom")) + with pytest.raises(NarrativeGenerationError, match="LLM call failed"): + list(generate_narrative_stream(_hike_input(), _gpx_stats(), _photos(), client=client)) + + +def test_generate_narrative_stream_translates_retry_exhausted_error() -> None: + client = _stream_client(LLMRetryExhaustedError("rate limit")) + with pytest.raises(NarrativeGenerationError, match="LLM call failed"): + list(generate_narrative_stream(_hike_input(), _gpx_stats(), _photos(), client=client)) + + +def test_generate_narrative_stream_validates_schema() -> None: + """JSON parses but is missing required keys → NarrativeGenerationError.""" + bogus = json.dumps({"title": {"en": "x"}}) + client = _stream_client(_split(bogus, parts=2)) + with pytest.raises(NarrativeGenerationError, match="schema"): + list(generate_narrative_stream(_hike_input(), _gpx_stats(), _photos(), client=client)) diff --git a/tests/test_web.py b/tests/test_web.py index 59e2562..e3adc42 100644 --- a/tests/test_web.py +++ b/tests/test_web.py @@ -4,21 +4,26 @@ mocks the Anthropic client at the ``client_factory`` injection point. The real LLM SDK is never called. -Tests share three things: +Tests share four things: * ``_make_client`` — a ``MagicMock`` shaped like ``AnthropicClient`` whose - ``.complete`` returns a valid narrative JSON. + ``.complete_stream`` yields a valid narrative JSON in chunks (the SSE + flow's primary path). * ``_settings`` — a ``Settings`` instance with a fake API key, so import paths that call ``load_settings`` still work even though the factory override means the key is never used. * ``_app_with_storage`` — builds the FastAPI app with a ``Storage`` rooted under ``tmp_path`` and the sweeper disabled, so workspaces are scoped to the test run and we can call ``sweep_expired`` deterministically. +* ``_complete_generation`` — runs the full POST /generate → SSE stream + → memory ready handshake and returns the slug. Most happy-path tests + use it instead of asserting the SSE wire format directly. """ from __future__ import annotations import json +import re import time from collections.abc import Iterator from pathlib import Path @@ -45,6 +50,12 @@ SAMPLE_GPX = FIXTURES / "sample.gpx" SAMPLE_PHOTOS = FIXTURES / "sample_photos" +# The generating page renders the slug as a data attribute on the +# Alpine root; the SSE flow reads it from the URL embedded in +# ``EventSource('/generate//stream')``. We pull it from the +# data attribute to keep the assertion stable if the JS shape changes. +_SLUG_RE = re.compile(r'data-slug="([0-9a-f]{12})"') + # ── fixtures ───────────────────────────────────────────────────────────────── @@ -94,11 +105,24 @@ def _valid_response_json(n_photos: int = 5) -> str: ) -def _make_client() -> MagicMock: - """Mocked Anthropic client whose ``.model`` is a real string.""" +def _stream_chunks(payload: str, *, parts: int = 8) -> list[str]: + """Slice ``payload`` into roughly equal SSE chunks for the fake stream.""" + if not payload: + return [""] + step = max(1, len(payload) // parts) + return [payload[i : i + step] for i in range(0, len(payload), step)] + + +def _make_client(*, response: str | None = None) -> MagicMock: + """Mocked Anthropic client supporting both complete and complete_stream.""" + body = response if response is not None else _valid_response_json() fake = MagicMock(spec=AnthropicClient) fake.model = "claude-opus-4-7-test" - fake.complete.return_value = _valid_response_json() + fake.complete.return_value = body + # ``side_effect`` is a callable that returns a fresh iterator on + # every invocation — important because the SSE retry path may call + # complete_stream twice. + fake.complete_stream.side_effect = lambda *_a, **_kw: iter(_stream_chunks(body)) return fake @@ -151,6 +175,70 @@ def _generate_files(n_photos: int = 5) -> list[tuple[str, tuple[str, bytes, str] return files +def _slug_from_generating_page(html: str) -> str: + match = _SLUG_RE.search(html) + assert match is not None, "generating page is missing the data-slug attribute" + return match.group(1) + + +def _parse_sse(text: str) -> list[tuple[str, dict[str, object]]]: + """Parse a captured SSE stream into a list of (event, data) tuples. + + The stream is a sequence of ``event: NAME\\ndata: JSON\\n\\n`` blocks. + Tests assert against the resulting list rather than walking the + bytes, which keeps the assertion focused on the contract instead of + the wire format. + """ + events: list[tuple[str, dict[str, object]]] = [] + for block in text.split("\n\n"): + block = block.strip() + if not block: + continue + name = "" + data = "" + for line in block.split("\n"): + if line.startswith("event:"): + name = line[len("event:") :].strip() + elif line.startswith("data:"): + data = line[len("data:") :].strip() + try: + payload = json.loads(data) if data else {} + except json.JSONDecodeError: + payload = {"raw": data} + events.append((name, payload)) + return events + + +def _drain_stream(client: TestClient, slug: str) -> list[tuple[str, dict[str, object]]]: + """GET the SSE endpoint for ``slug`` and return the parsed events.""" + with client.stream("GET", f"/generate/{slug}/stream") as resp: + assert resp.status_code == 200, resp.read().decode() + body = b"".join(resp.iter_bytes()).decode("utf-8") + return _parse_sse(body) + + +def _complete_generation( + client: TestClient, + *, + description: str = "x", + style: str = "editorial", + location: str | None = None, + n_photos: int = 5, +) -> str: + """Submit the form and drain the SSE stream. Returns the slug.""" + data: dict[str, str] = {"description": description, "style": style} + if location is not None: + data["location"] = location + response = client.post("/generate", data=data, files=_generate_files(n_photos)) + assert response.status_code == 200, response.text + slug = _slug_from_generating_page(response.text) + events = _drain_stream(client, slug) + # The terminal event must be ``done`` on the happy path. + names = [e[0] for e in events] + assert "done" in names, f"stream ended without 'done' event: {names}" + return slug + + # ── pages ──────────────────────────────────────────────────────────────────── @@ -168,6 +256,16 @@ def test_landing_page_returns_form(client: TestClient) -> None: assert 'value="encyclopedia"' in body +def test_landing_page_links_to_privacy_in_new_tab(client: TestClient) -> None: + """Privacy link should open in a new tab so the upload form is preserved.""" + response = client.get("/") + body = response.text + # The "How we handle your photos" link should target _blank. + assert "How we handle your photos" in body + # The link to /privacy near the form should have target="_blank". + assert 'href="/privacy" target="_blank"' in body + + def test_privacy_page_mentions_retention_and_repo(client: TestClient) -> None: response = client.get("/privacy") assert response.status_code == 200 @@ -178,6 +276,20 @@ def test_privacy_page_mentions_retention_and_repo(client: TestClient) -> None: assert "deleted" in body.lower() +def test_privacy_page_links_to_lifecycle_code_with_line_numbers( + client: TestClient, +) -> None: + """The privacy page links to the actual deletion logic on GitHub. + + Locks in the "verify it yourself" promise: the user can click + through to the line ranges that implement the deletion. + """ + body = client.get("/privacy").text + assert "web/storage.py#L" in body + assert "cleanup_inputs" in body + assert "sweep_expired" in body + + def test_healthz_returns_status_ok(client: TestClient) -> None: response = client.get("/healthz") assert response.status_code == 200 @@ -187,7 +299,13 @@ def test_healthz_returns_status_ok(client: TestClient) -> None: # ── happy path ─────────────────────────────────────────────────────────────── -def test_generate_renders_memory_and_redirects(client: TestClient, storage: Storage) -> None: +def test_generate_returns_generating_page(client: TestClient, storage: Storage) -> None: + """POST /generate returns a 200 generating page, not a redirect. + + The page embeds the slug for the SSE listener to pick up. Pending + state is on disk so the SSE endpoint can resume — the workspace + must exist after this call returns. + """ response = client.post( "/generate", data={ @@ -196,42 +314,171 @@ def test_generate_renders_memory_and_redirects(client: TestClient, storage: Stor "location": "Bavarian Alps", }, files=_generate_files(), - follow_redirects=False, ) - - assert response.status_code == 303 - redirect = response.headers["location"] - assert redirect.startswith("/memory/") - slug = redirect.rsplit("/", 1)[-1] + assert response.status_code == 200 + body = response.text + slug = _slug_from_generating_page(body) assert len(slug) == 12 + # Generating page connects to the SSE endpoint by URL. + assert f"/generate/{slug}/stream" in body + # User-facing copy. + assert "Writing your memory" in body or "writing your memory" in body.lower() + + # Pending state persisted; raw uploads scheduled for cleanup but not + # yet wiped (BackgroundTask runs after response — we can't assert + # cleanup directly without flushing it). + workspace = storage.get_workspace(slug) + assert workspace is not None + assert workspace.pending_state_path.is_file() + # The final state.json does not exist yet — that's the SSE phase. + assert not workspace.state_path.is_file() + + +def test_stream_emits_chunks_then_done_then_renders_html( + client: TestClient, storage: Storage +) -> None: + """SSE stream pushes chunks, finishes with 'done', the HTML is on disk.""" + response = client.post( + "/generate", + data={ + "description": "The fog cleared just as we reached the ridge.", + "style": "editorial", + "location": "Bavarian Alps", + }, + files=_generate_files(), + ) + slug = _slug_from_generating_page(response.text) + events = _drain_stream(client, slug) + names = [name for name, _ in events] + + # The wire shape: at least one status, several chunks, a done event. + assert "status" in names + assert names.count("chunk") >= 2, names + assert names[-1] == "done" + + done_payload = events[-1][1] + assert done_payload.get("redirect") == f"/memory/{slug}" + + # State persisted, pending cleared, HTML rendered. + workspace = storage.get_workspace(slug) + assert workspace is not None + assert workspace.state_path.is_file() + assert not workspace.pending_state_path.is_file() + assert (workspace.output_dir / f"{slug}.html").is_file() + # The follow-up GET serves the rendered HTML. - page = client.get(redirect) + page = client.get(f"/memory/{slug}") assert page.status_code == 200 body = page.text assert "Above the fog line" in body assert "Над линией тумана" in body assert "Über der Nebelgrenze" in body assert "data:image/jpeg;base64," in body + # Save for Instagram button is wired into the rendered page. + assert "Save for Instagram" in body + assert f'data-slug="{slug}"' in body - # Counter advanced exactly once. + # Counter advanced exactly once on the prep phase. assert request_counter_value() == 1 - # State persisted for carousel use. - workspace = storage.get_workspace(slug) - assert workspace is not None - assert workspace.state_path.is_file() +def test_stream_retries_once_on_unparseable_first_attempt( + storage: Storage, +) -> None: + """First completed stream returns prose; the second returns valid JSON. -def test_generate_runs_background_cleanup_of_inputs(client: TestClient, storage: Storage) -> None: + Asserts: a 'status' event with phase=='regenerating' appears between + the two attempts, and the final 'done' event still fires. + """ + fake = _make_client() + valid = _valid_response_json() + # First call: prose chunks. Second call: valid JSON chunks. + call_count = {"n": 0} + + def _stream(*_a: object, **_kw: object) -> Iterator[str]: + call_count["n"] += 1 + if call_count["n"] == 1: + return iter(_stream_chunks("this is not json at all", parts=4)) + return iter(_stream_chunks(valid)) + + fake.complete_stream.side_effect = _stream + + app, _ = _app_with_storage(storage, client=fake) + with TestClient(app) as c: + response = c.post( + "/generate", + data={"description": "x", "style": "editorial"}, + files=_generate_files(), + ) + slug = _slug_from_generating_page(response.text) + events = _drain_stream(c, slug) + + phases = [e[1].get("phase") for e in events if e[0] == "status"] + assert "regenerating" in phases + assert events[-1][0] == "done" + assert call_count["n"] == 2 + + +def test_stream_emits_error_on_double_failure(storage: Storage) -> None: + """Two unparseable attempts in a row land an 'error' event, not 'done'.""" + fake = _make_client() + fake.complete_stream.side_effect = lambda *_a, **_kw: iter( + _stream_chunks("still not json", parts=3) + ) + + app, _ = _app_with_storage(storage, client=fake) + with TestClient(app) as c: + response = c.post( + "/generate", + data={"description": "x", "style": "editorial"}, + files=_generate_files(), + ) + slug = _slug_from_generating_page(response.text) + events = _drain_stream(c, slug) + + names = [n for n, _ in events] + assert "done" not in names + assert names[-1] == "error" + + +def test_stream_returns_404_after_workspace_consumed(client: TestClient, storage: Storage) -> None: + """Hitting the SSE endpoint twice cleanly 404s on the second call. + + The first run unlinks ``pending.json`` once rendering succeeds, so a + refresh of the generating page (or an attacker probing the slug) + cannot rerun the LLM call — and cannot pay the bill again. + """ + slug = _complete_generation(client) + second = client.get(f"/generate/{slug}/stream") + assert second.status_code == 404 + + +def test_stream_returns_404_for_unknown_slug(client: TestClient) -> None: + response = client.get("/generate/abcdef012345/stream") + assert response.status_code == 404 + + +def test_stream_endpoint_sets_event_stream_content_type( + client: TestClient, +) -> None: + """SSE clients reject anything that isn't ``text/event-stream``.""" response = client.post( "/generate", data={"description": "x", "style": "editorial"}, files=_generate_files(), - follow_redirects=False, ) - assert response.status_code == 303 - slug = response.headers["location"].rsplit("/", 1)[-1] + slug = _slug_from_generating_page(response.text) + with client.stream("GET", f"/generate/{slug}/stream") as resp: + assert resp.status_code == 200 + assert resp.headers["content-type"].startswith("text/event-stream") + assert resp.headers.get("cache-control") == "no-cache" + # Drain to release the connection. + b"".join(resp.iter_bytes()) + + +def test_generate_runs_background_cleanup_of_inputs(client: TestClient, storage: Storage) -> None: + slug = _complete_generation(client) workspace = storage.get_workspace(slug) assert workspace is not None # BackgroundTask wiped raw uploads, but resized + output remain. @@ -243,13 +490,7 @@ def test_generate_runs_background_cleanup_of_inputs(client: TestClient, storage: def test_carousel_route_renders_slides_after_generate(client: TestClient, storage: Storage) -> None: - response = client.post( - "/generate", - data={"description": "x", "style": "editorial"}, - files=_generate_files(), - follow_redirects=False, - ) - slug = response.headers["location"].rsplit("/", 1)[-1] + slug = _complete_generation(client) car = client.post(f"/memory/{slug}/carousel") assert car.status_code == 200 @@ -261,12 +502,58 @@ def test_carousel_route_renders_slides_after_generate(client: TestClient, storag for slide_url in slide_paths: assert slide_url.startswith(f"/memory/{slug}/carousel/") - # Each slide is fetchable. + # Each slide is fetchable and downloadable. first = client.get(slide_paths[0]) assert first.status_code == 200 assert first.headers["content-type"].startswith("image/jpeg") +def test_carousel_slide_has_attachment_disposition( + client: TestClient, +) -> None: + """Desktop fallback download links rely on Content-Disposition. + + iOS Safari's ``navigator.share({files})`` flow does not need this + header, but the desktop fallback links served by the rendered + memory page do — clicking them should land the file in Downloads + with a meaningful name. + """ + slug = _complete_generation(client) + car = client.post(f"/memory/{slug}/carousel").json() + first = client.get(car["slides"][0]) + disposition = first.headers.get("content-disposition", "") + assert "attachment" in disposition + # Slug is in the suggested filename so multiple downloads stay + # distinct in the user's folder. + assert slug in disposition + + +def test_carousel_returns_n_slides_for_n_photos( + client: TestClient, +) -> None: + """Title + N photos + quote slides — locks in the carousel shape.""" + fake = _make_client(response=_valid_response_json(n_photos=4)) + app = create_app( + settings=_settings(), + storage=Storage(retention_seconds=RETENTION_SECONDS), + client_factory=lambda: fake, + enable_sweeper=False, + ) + with TestClient(app) as c: + response = c.post( + "/generate", + data={"description": "x", "style": "editorial"}, + files=_generate_files(n_photos=4), + ) + slug = _slug_from_generating_page(response.text) + events = _drain_stream(c, slug) + assert events[-1][0] == "done" + + car = c.post(f"/memory/{slug}/carousel").json() + # Title + 4 photos + quote. + assert len(car["slides"]) == 6 + + # ── validation ─────────────────────────────────────────────────────────────── @@ -276,7 +563,6 @@ def test_generate_rejects_missing_gpx(client: TestClient) -> None: "/generate", data={"description": "x", "style": "editorial"}, files=files, - follow_redirects=False, ) # FastAPI returns 422 for missing form/file fields by default; our # validation is one layer below that and only fires once the upload @@ -292,7 +578,6 @@ def test_generate_rejects_missing_photos(client: TestClient) -> None: "/generate", data={"description": "x", "style": "editorial"}, files=files, - follow_redirects=False, ) assert response.status_code in (400, 422) @@ -302,7 +587,6 @@ def test_generate_rejects_unknown_style(client: TestClient) -> None: "/generate", data={"description": "x", "style": "polaroid"}, files=_generate_files(), - follow_redirects=False, ) assert response.status_code == 400 assert "Unknown style" in response.json()["detail"] @@ -317,7 +601,6 @@ def test_generate_rejects_unsupported_photo_format( "/generate", data={"description": "x", "style": "editorial"}, files=files, - follow_redirects=False, ) assert response.status_code == 400 assert "Unsupported" in response.json()["detail"] @@ -334,30 +617,11 @@ def test_generate_rejects_oversized_photo( "/generate", data={"description": "x", "style": "editorial"}, files=_generate_files(n_photos=1), - follow_redirects=False, ) assert response.status_code == 413 assert "Photo exceeds" in response.json()["detail"] -def test_generate_surfaces_pipeline_error_as_400( - storage: Storage, -) -> None: - fake = _make_client() - fake.complete.return_value = "this is not json" - app, _ = _app_with_storage(storage, client=fake) - with TestClient(app) as c: - response = c.post( - "/generate", - data={"description": "x", "style": "editorial"}, - files=_generate_files(), - follow_redirects=False, - ) - assert response.status_code == 400 - # Workspace was deleted to keep tmp clean after the failure. - assert not list(storage.root.iterdir()) - - # ── memory page / carousel 404s ────────────────────────────────────────────── @@ -379,15 +643,7 @@ def test_carousel_returns_404_for_unknown_slug(client: TestClient) -> None: def test_carousel_slide_rejects_traversal(client: TestClient, storage: Storage) -> None: - # First, generate a real workspace so the slug is valid. - gen = client.post( - "/generate", - data={"description": "x", "style": "editorial"}, - files=_generate_files(), - follow_redirects=False, - ) - slug = gen.headers["location"].rsplit("/", 1)[-1] - # Then try to read outside carousel/. + slug = _complete_generation(client) bad = client.get(f"/memory/{slug}/carousel/..%2F..%2Fstate.json") assert bad.status_code in (400, 404) @@ -417,6 +673,31 @@ def test_sweep_keeps_fresh_workspaces(storage: Storage) -> None: assert ws.root.is_dir() +def test_sweep_clears_workspace_after_retention_window( + client: TestClient, storage: Storage +) -> None: + """End-to-end privacy test: upload, generate, age past retention, sweep. + + Confirms ``/privacy``'s "after that window the entire workspace is + deleted" promise is enforced by code, not just docs. + """ + slug = _complete_generation(client) + workspace = storage.get_workspace(slug) + assert workspace is not None and workspace.root.is_dir() + + # Backdate to simulate the 30-minute retention window having passed. + old = time.time() - (storage.retention_seconds + 60) + import os + + os.utime(workspace.root, (old, old)) + + deleted = storage.sweep_expired() + assert deleted >= 1 + assert not workspace.root.exists() + # Memory page now 404s — the page is gone. + assert client.get(f"/memory/{slug}").status_code == 404 + + # ── upload limit constants exist and are sane ─────────────────────────────── @@ -476,10 +757,16 @@ def test_fake_client_factory_drives_full_pipeline(storage: Storage) -> None: "/generate", data={"description": "x", "style": "editorial"}, files=_generate_files(), - follow_redirects=False, ) - assert response.status_code == 303 - slug = response.headers["location"].rsplit("/", 1)[-1] + assert response.status_code == 200 + slug = _slug_from_generating_page(response.text) workspace = storage.get_workspace(slug) assert workspace is not None + assert workspace.pending_state_path.is_file() + + # Drain the SSE stream — the fake stream factory must yield real + # chunks that round-trip through generate_narrative_stream cleanly. + with TestClient(app) as c: + events = _drain_stream(c, slug) + assert events[-1][0] == "done" assert (workspace.output_dir / f"{slug}.html").is_file() diff --git a/trailstory/llm/client.py b/trailstory/llm/client.py index b075700..652a255 100644 --- a/trailstory/llm/client.py +++ b/trailstory/llm/client.py @@ -14,6 +14,7 @@ import logging import random import time +from collections.abc import Iterator from typing import Any, Final import anthropic @@ -167,6 +168,75 @@ def complete(self, prompt: str, system: str) -> str: f"Rate-limit retries exhausted after {self._max_retries} attempts" ) from last_rate_limit + def complete_stream(self, prompt: str, system: str) -> Iterator[str]: + """Stream the assistant response as text chunks. + + Mirrors :meth:`complete` but yields each text delta as it arrives so + callers (the FastAPI SSE endpoint) can push tokens to the browser + while the model is still writing. Rate-limit errors that fire + *before* any chunk is yielded are retried with the same exponential + backoff as :meth:`complete`; any error that fires *after* yielding + begins is surfaced as :class:`LLMResponseError` (re-yielding chunks + the caller has already consumed would corrupt the SSE stream, so we + don't try). + + Args: + prompt: User-role message content. + system: System prompt. + + Yields: + Each text delta in arrival order. The full response is the + concatenation of the yielded chunks. + + Raises: + LLMRetryExhaustedError: All rate-limit retries were used up + before a single chunk was emitted. + LLMResponseError: Empty stream, mid-stream error, or + non-retryable API error. + """ + last_rate_limit: RateLimitError | None = None + + for attempt in range(1, self._max_retries + 1): + chunks_emitted = False + try: + with self._client.messages.stream( + model=self._model, + max_tokens=self._max_tokens, + system=system, + messages=[{"role": "user", "content": prompt}], + ) as stream: + for chunk in stream.text_stream: + if chunk: + chunks_emitted = True + yield chunk + if not chunks_emitted: + raise LLMResponseError("Anthropic API returned an empty stream.") + return + except RateLimitError as exc: + if chunks_emitted: + # Cannot retry once chunks are out the door. + raise LLMResponseError(f"Anthropic API rate-limited mid-stream: {exc}") from exc + last_rate_limit = exc + if attempt >= self._max_retries: + break + delay = self._compute_backoff(attempt) + logger.warning( + "anthropic stream rate-limited (attempt %d/%d); sleeping %.2fs", + attempt, + self._max_retries, + delay, + ) + time.sleep(delay) + continue + except APIStatusError as exc: + raise LLMResponseError(f"Anthropic API error {exc.status_code}: {exc}") from exc + except APIError as exc: + raise LLMResponseError(f"Anthropic API error: {exc}") from exc + + raise LLMRetryExhaustedError( + f"Rate-limit retries exhausted after {self._max_retries} attempts" + ) from last_rate_limit + # -- internal helpers ---------------------------------------------------- def _compute_backoff(self, attempt: int) -> float: diff --git a/trailstory/llm/narrative.py b/trailstory/llm/narrative.py index 35b87fc..57acfdb 100644 --- a/trailstory/llm/narrative.py +++ b/trailstory/llm/narrative.py @@ -30,6 +30,8 @@ import json import logging +from collections.abc import Iterator +from dataclasses import dataclass from json import JSONDecodeError from typing import Any @@ -51,6 +53,38 @@ logger = logging.getLogger(__name__) +@dataclass(frozen=True) +class NarrativeStreamChunk: + """A text delta yielded by the streaming narrative pipeline. + + The SSE endpoint forwards each chunk to the browser as an event so the + user sees the model writing in real time. + """ + + text: str + + +@dataclass(frozen=True) +class NarrativeStreamRetry: + """The first attempt produced unparseable output; we are retrying once. + + Emitted between the failed attempt and the second one so the SSE + consumer can flip to a "regenerating" UI state. + """ + + reason: str + + +@dataclass(frozen=True) +class NarrativeStreamComplete: + """Final event: the streamed output validated cleanly into a narrative.""" + + narrative: NarrativeOutput + + +NarrativeStreamEvent = NarrativeStreamChunk | NarrativeStreamRetry | NarrativeStreamComplete + + class NarrativeGenerationError(Exception): """Terminal failure of the narrative pipeline. @@ -138,9 +172,115 @@ def generate_narrative( return narrative +def generate_narrative_stream( + hike_input: HikeInput, + gpx_stats: GpxStats, + photos: list[PhotoMeta], + *, + client: AnthropicClient, + location: str = "the trail", +) -> Iterator[NarrativeStreamEvent]: + """Streaming variant of :func:`generate_narrative`. + + Yields a :class:`NarrativeStreamChunk` for every text delta from the + LLM, optionally a :class:`NarrativeStreamRetry` between attempts when + the first response did not parse, and finally a + :class:`NarrativeStreamComplete` with the validated + :class:`NarrativeOutput`. + + The retry policy mirrors :func:`generate_narrative`: one extra attempt + with :data:`USER_NARRATIVE_RETRY_SUFFIX` when the first response was + unparseable. Schema-validation failures are not retried — they raise + :class:`NarrativeGenerationError` immediately, the same as the + non-streaming pipeline. + + The cache is intentionally bypassed for streaming runs: the user is + looking at a "writing your story" page and expects to see the words + appear, so a cached instant return would be jarring. The + non-streaming :func:`generate_narrative` keeps the cache for CLI use. + + Args: + hike_input: Hiker's seed text and source paths. + gpx_stats: Parsed GPX stats. + photos: Loaded photos. The model selects 6-8 indices into this list. + client: Anthropic client wrapper. Must implement ``complete_stream``. + location: Fallback place name when ``hike_input.location_name`` is + unset. + + Yields: + :class:`NarrativeStreamEvent` instances. The terminal event is + always :class:`NarrativeStreamComplete` on success. + + Raises: + NarrativeGenerationError: photos list empty, LLM call failed, the + model returned non-JSON twice in a row, or the parsed JSON did + not validate against the schema. + """ + if not photos: + raise NarrativeGenerationError("at least one photo is required to build a narrative") + + place = hike_input.location_name or location + base_prompt = USER_NARRATIVE_TEMPLATE.format( + location=place, + distance_km=gpx_stats.distance_km, + elevation_gain_m=gpx_stats.elevation_gain_m, + duration_min=gpx_stats.duration_min, + summit_elev_m=gpx_stats.summit_elev_m, + n_photos=len(photos), + n_photos_minus_1=len(photos) - 1, + seed_text=hike_input.seed_text, + ) + + parsed, chunks_first = _stream_and_parse(client, base_prompt) + yield from (NarrativeStreamChunk(text=c) for c in chunks_first) + + if parsed is None: + logger.warning( + "first streamed response did not parse as JSON; retrying with explicit directive" + ) + yield NarrativeStreamRetry(reason="model output was not valid JSON; retrying once") + retry_prompt = base_prompt + USER_NARRATIVE_RETRY_SUFFIX + parsed, chunks_retry = _stream_and_parse(client, retry_prompt) + yield from (NarrativeStreamChunk(text=c) for c in chunks_retry) + if parsed is None: + raise NarrativeGenerationError("Model returned non-JSON output on both attempts.") + + try: + narrative = NarrativeOutput.model_validate(parsed) + except ValidationError as exc: + raise NarrativeGenerationError( + f"LLM JSON did not match NarrativeOutput schema: {exc}" + ) from exc + + yield NarrativeStreamComplete(narrative=narrative) + + # -- internal helpers --------------------------------------------------------- +def _stream_and_parse( + client: AnthropicClient, prompt: str +) -> tuple[dict[str, Any] | None, list[str]]: + """Stream a single attempt, accumulate text, try to parse as JSON. + + Returns ``(parsed_dict_or_None, chunks)`` so the caller can yield + each chunk to the SSE consumer in arrival order. + """ + chunks: list[str] = [] + try: + for chunk in client.complete_stream(prompt=prompt, system=SYSTEM_NARRATIVE): + chunks.append(chunk) + except (LLMResponseError, LLMRetryExhaustedError) as exc: + raise NarrativeGenerationError(f"LLM call failed: {exc}") from exc + + cleaned = _strip_code_fences("".join(chunks)) + try: + result = json.loads(cleaned) + except JSONDecodeError: + return None, chunks + return (result if isinstance(result, dict) else None), chunks + + def _call_and_parse(client: AnthropicClient, prompt: str) -> dict[str, Any] | None: """Call the LLM once and try to parse the response as a JSON object. diff --git a/web/dev.py b/web/dev.py index 635d326..e002b73 100644 --- a/web/dev.py +++ b/web/dev.py @@ -83,7 +83,11 @@ def make_fake_client_factory() -> Callable[[], AnthropicClient]: """Return a factory that produces a deterministic fake LLM client. The factory shape matches the real one in ``web.app`` so swapping - is a one-line change at app construction. + is a one-line change at app construction. The mock supports both + the synchronous ``complete`` path (CLI / non-streaming generation) + and the ``complete_stream`` path used by the SSE flow — the stream + method yields the same constant narrative split across a handful + of chunks so the dev page shows the streaming animation. """ def _factory() -> AnthropicClient: @@ -94,6 +98,14 @@ def _factory() -> AnthropicClient: # in by mistake. fake.model = "trailstory-dev-fake" fake.complete.return_value = _FAKE_NARRATIVE + + def _stream(*_args: object, **_kwargs: object) -> object: + # Slice the fixture into evenly-sized chunks so the SSE + # flow has multiple frames to render. + step = max(1, len(_FAKE_NARRATIVE) // 16) + return iter(_FAKE_NARRATIVE[i : i + step] for i in range(0, len(_FAKE_NARRATIVE), step)) + + fake.complete_stream.side_effect = _stream return fake return _factory diff --git a/web/pipeline.py b/web/pipeline.py index 3447a4e..4052bbb 100644 --- a/web/pipeline.py +++ b/web/pipeline.py @@ -11,10 +11,12 @@ * :class:`Style` — three-value enum mirroring the form's radio buttons (editorial / log / encyclopedia). Per ADR-006 the style chooses the visual template only — the narrative text is identical across styles. - Today only the editorial template is wired up; ``log`` and - ``encyclopedia`` accept the same template until their layouts ship. -* :func:`run_pipeline` — takes a ``Workspace`` and form-derived inputs, - returns a populated ``Memory`` and the path of the rendered HTML. +* :func:`prepare_pipeline` — runs the deterministic prep phase (parse + + load_photos) and persists the inputs as ``pending.json`` so the SSE + endpoint can resume with a streaming LLM call. +* :func:`stream_pipeline` — yields :class:`PipelineStreamEvent` + instances as the LLM writes the narrative; on success renders the + HTML and persists final ``state.json``. * :func:`render_carousel` — re-builds the ``Memory`` from ``workspace.state_path`` and renders an Instagram carousel. @@ -26,6 +28,8 @@ import json import logging +from collections.abc import Iterator +from dataclasses import dataclass from datetime import date, datetime from enum import StrEnum from pathlib import Path @@ -33,8 +37,15 @@ from trailstory.gpx import GpxParseError, parse_gpx from trailstory.llm.client import AnthropicClient -from trailstory.llm.narrative import NarrativeGenerationError, generate_narrative -from trailstory.models import HikeInput, Memory, NarrativeOutput, PhotoMeta +from trailstory.llm.narrative import ( + NarrativeGenerationError, + NarrativeStreamChunk, + NarrativeStreamComplete, + NarrativeStreamRetry, + generate_narrative_stream, +) +from trailstory.models import GpxStats, HikeInput, Memory, NarrativeOutput, PhotoMeta +from trailstory.models import Style as _ModelStyle from trailstory.photos import PhotoLoadError, load_photos from trailstory.renderers.html import HtmlRenderError, render_html from trailstory.renderers.instagram import InstagramRenderError, render_instagram_carousel @@ -74,31 +85,63 @@ class PipelineError(Exception): """ -def run_pipeline( +@dataclass(frozen=True) +class PipelineStreamChunk: + """A text delta from the streaming narrative call. + + The SSE endpoint forwards each chunk to the browser so the user sees + the model writing in real time. + """ + + text: str + + +@dataclass(frozen=True) +class PipelineStreamRetry: + """First attempt failed validation; retrying once. + + The SSE endpoint flips the page UI to a "regenerating" state when + this event lands. + """ + + reason: str + + +@dataclass(frozen=True) +class PipelineStreamRendered: + """Final event: HTML written, ``state.json`` persisted, slug ready to redirect to.""" + + slug: str + output_path: Path + + +PipelineStreamEvent = PipelineStreamChunk | PipelineStreamRetry | PipelineStreamRendered + + +def prepare_pipeline( workspace: Workspace, *, description: str, style: Style, - client: AnthropicClient, photo_max_edge: int, photo_quality: int, location: str | None = None, -) -> tuple[Memory, Path]: - """Run parse → load_photos → narrative → render_html for a workspace. +) -> None: + """Run parse + load_photos and persist a pending state for streaming. + + This is the first half of the pipeline split that the SSE flow + requires: ``POST /generate`` does the cheap, deterministic input + parsing here so any 4xx surfaces immediately, then returns the + "generating" page. ``GET /generate/{slug}/stream`` picks up by + reading the persisted pending state and runs the streaming LLM + call. Inputs (raw GPX + photos) must already exist under - ``workspace.gpx_dir`` and ``workspace.photos_dir``. The resized - photos go to ``workspace.resized_dir`` and the rendered HTML goes - to ``workspace.output_dir``. - - The fully-populated ``Memory`` is also persisted as JSON to - ``workspace.state_path`` so :func:`render_carousel` can rebuild it - later — the raw uploads are about to be wiped by the route's - BackgroundTask. - - Style is recorded on the ``Memory`` for forward compatibility with - ADR-006 (per-style templates), but only the editorial template is - wired up today; ``log`` and ``encyclopedia`` render the same way. + ``workspace.gpx_dir`` and ``workspace.photos_dir``; resized photos + land in ``workspace.resized_dir`` and survive the + ``BackgroundTask`` cleanup that wipes the raw uploads. + + Raises :class:`PipelineError` for parse/load failures. """ gpx_path = _single_gpx_file(workspace.gpx_dir) try: @@ -122,29 +165,81 @@ def run_pipeline( seed_text=description, location_name=location, ) + hike_date = _derive_hike_date(stats, photos) + + _persist_pending_state( + workspace, + hike_input=hike_input, + gpx_stats=stats, + photos=photos, + hike_date=hike_date, + location=location, + style=style, + ) + logger.info( + "prepared workspace %s for streaming (style=%s, photos=%d)", + workspace.slug, + style.value, + len(photos), + ) + +def stream_pipeline( + workspace: Workspace, + *, + client: AnthropicClient, +) -> Iterator[PipelineStreamEvent]: + """Resume a prepared pipeline run and stream the narrative. + + Reads the pending state written by :func:`prepare_pipeline`, calls + the streaming LLM, yields :class:`PipelineStreamEvent` instances as + chunks arrive, and on success renders the HTML and persists the + final ``state.json``. The terminal event is always + :class:`PipelineStreamRendered`. + + Raises :class:`PipelineError` if the workspace has no pending state, + the LLM call fails, validation fails, or HTML rendering fails. + """ + pending = _load_pending_state(workspace) try: - narrative = generate_narrative( - hike_input, - stats, - photos, + narrative: NarrativeOutput | None = None + for event in generate_narrative_stream( + pending.hike_input, + pending.gpx_stats, + pending.photos, client=client, - use_cache=False, - ) + ): + if isinstance(event, NarrativeStreamChunk): + yield PipelineStreamChunk(text=event.text) + elif isinstance(event, NarrativeStreamRetry): + yield PipelineStreamRetry(reason=event.reason) + elif isinstance(event, NarrativeStreamComplete): + narrative = event.narrative except NarrativeGenerationError as exc: raise PipelineError(str(exc)) from exc - selected = [photos[i] for i in narrative.selected_photo_indices if 0 <= i < len(photos)] + if narrative is None: + # Defensive: generate_narrative_stream always ends with a + # NarrativeStreamComplete on success, so reaching this branch + # means the generator returned without yielding the terminal + # event — treat as a hard failure. + raise PipelineError("narrative stream ended without a final event") + + selected = [ + pending.photos[i] for i in narrative.selected_photo_indices if 0 <= i < len(pending.photos) + ] if not selected: raise PipelineError("LLM returned no usable photo indices.") - hike_date = _derive_hike_date(stats, photos) - memory = Memory( - hike_input=hike_input, - gpx_stats=stats, + hike_input=pending.hike_input, + gpx_stats=pending.gpx_stats, narrative=narrative, selected_photos=selected, + # Convert across the (deliberately separate, see ADR-006) Style + # enums in this module and trailstory.models — they share string + # values so the round-trip is lossless. + style=_ModelStyle(pending.style.value), ) try: @@ -152,15 +247,27 @@ def run_pipeline( memory=memory, output_dir=workspace.output_dir, slug=workspace.slug, - hike_date=hike_date, - location=location, + hike_date=pending.hike_date, + location=pending.location, ) except HtmlRenderError as exc: raise PipelineError(str(exc)) from exc - _persist_state(workspace, memory, hike_date=hike_date, location=location, style=style) - logger.info("rendered %s for workspace %s (style=%s)", out_path.name, workspace.slug, style) - return memory, out_path + _persist_state( + workspace, + memory, + hike_date=pending.hike_date, + location=pending.location, + style=pending.style, + ) + workspace.pending_state_path.unlink(missing_ok=True) + logger.info( + "streamed pipeline rendered %s for workspace %s (style=%s)", + out_path.name, + workspace.slug, + pending.style.value, + ) + yield PipelineStreamRendered(slug=workspace.slug, output_path=out_path) def render_carousel(workspace: Workspace) -> list[Path]: @@ -209,6 +316,83 @@ def __init__( self.style = style +class _PendingState: + """Pre-LLM inputs needed to resume into the streaming narrative call.""" + + __slots__ = ( + "gpx_stats", + "hike_date", + "hike_input", + "location", + "photos", + "style", + ) + + def __init__( + self, + hike_input: HikeInput, + gpx_stats: GpxStats, + photos: list[PhotoMeta], + hike_date: date | None, + location: str | None, + style: Style, + ) -> None: + self.hike_input = hike_input + self.gpx_stats = gpx_stats + self.photos = photos + self.hike_date = hike_date + self.location = location + self.style = style + + +def _persist_pending_state( + workspace: Workspace, + *, + hike_input: HikeInput, + gpx_stats: GpxStats, + photos: list[PhotoMeta], + hike_date: date | None, + location: str | None, + style: Style, +) -> None: + """Write the pre-LLM state ``stream_pipeline`` will resume from. + + The photos list (paths into ``workspace.resized_dir``) is what + survives the BackgroundTask cleanup of the raw uploads; we capture + the resolved metadata here so the SSE endpoint can pick up without + reaching back into the input dir. + """ + payload = { + "hike_input": hike_input.model_dump(mode="json"), + "gpx_stats": gpx_stats.model_dump(mode="json"), + "photos": [p.model_dump(mode="json") for p in photos], + "hike_date": hike_date.isoformat() if hike_date else None, + "location": location, + "style": style.value, + } + workspace.pending_state_path.write_text(json.dumps(payload), encoding="utf-8") + + +def _load_pending_state(workspace: Workspace) -> _PendingState: + if not workspace.pending_state_path.is_file(): + raise PipelineError("workspace has no pending state to stream from") + raw = json.loads(workspace.pending_state_path.read_text(encoding="utf-8")) + hike_input = HikeInput.model_validate(raw["hike_input"]) + gpx_stats = GpxStats.model_validate(raw["gpx_stats"]) + photos = [PhotoMeta.model_validate(p) for p in raw["photos"]] + hike_date_raw = raw.get("hike_date") + hike_date = date.fromisoformat(hike_date_raw) if hike_date_raw else None + style_raw = raw.get("style") or Style.default().value + return _PendingState( + hike_input, + gpx_stats, + photos, + hike_date, + raw.get("location"), + Style(style_raw), + ) + + def _persist_state( workspace: Workspace, memory: Memory, @@ -257,11 +441,10 @@ def _single_gpx_file(gpx_dir: Path) -> Path: return files[0] -def _derive_hike_date(stats: object, photos: list[PhotoMeta]) -> date: +def _derive_hike_date(stats: GpxStats, photos: list[PhotoMeta]) -> date: """Same heuristic as the CLI: GPX trackpoint time wins, then photo, then today.""" - waypoints = getattr(stats, "waypoints", None) or [] - for w in waypoints: - ts: datetime | None = getattr(w, "time", None) + for w in stats.waypoints: + ts: datetime | None = w.time if ts is not None: return ts.date() if photos: @@ -275,7 +458,12 @@ def _derive_hike_date(stats: object, photos: list[PhotoMeta]) -> date: "CAROUSEL_QUALITY", "NarrativeOutput", "PipelineError", + "PipelineStreamChunk", + "PipelineStreamEvent", + "PipelineStreamRendered", + "PipelineStreamRetry", "Style", + "prepare_pipeline", "render_carousel", - "run_pipeline", + "stream_pipeline", ] diff --git a/web/routes.py b/web/routes.py index fd6f512..6550c7c 100644 --- a/web/routes.py +++ b/web/routes.py @@ -1,14 +1,20 @@ """HTTP route handlers for the web builder. -Six endpoints, all stateless from the user's point of view: +Eight endpoints, all stateless from the user's point of view: * ``GET /`` — landing page + builder form. -* ``POST /generate`` — multipart upload; runs the pipeline - and 303-redirects to the memory page. +* ``POST /generate`` — multipart upload; runs the prep + phase (parse + photo load) and + returns the generating page. +* ``GET /generate/{slug}/stream`` — Server-Sent Events stream that + runs the LLM call and renders + the HTML; emits chunk / retry / + done / error events. * ``GET /memory/{slug}`` — serves the rendered HTML. * ``POST /memory/{slug}/carousel`` — generates the IG carousel on demand. * ``GET /privacy`` — plain-language privacy page. * ``GET /healthz`` — uptime probe. +* ``GET /memory/{slug}/carousel/{filename}`` — serves a single slide. Heavy lifting (parse / load / narrative / render) lives in ``web.pipeline``; this module is just request validation, file I/O, @@ -19,19 +25,35 @@ from __future__ import annotations +import json import logging import threading -from collections.abc import Callable, Iterable +from collections.abc import Callable, Iterable, Iterator from pathlib import Path from typing import Annotated, Final from fastapi import APIRouter, BackgroundTasks, Form, HTTPException, Request, UploadFile -from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, RedirectResponse, Response +from fastapi.responses import ( + FileResponse, + HTMLResponse, + JSONResponse, + Response, + StreamingResponse, +) from fastapi.templating import Jinja2Templates from trailstory.config import Settings from trailstory.llm.client import AnthropicClient -from web.pipeline import PipelineError, Style, render_carousel, run_pipeline +from web.pipeline import ( + PipelineError, + PipelineStreamChunk, + PipelineStreamRendered, + PipelineStreamRetry, + Style, + prepare_pipeline, + render_carousel, + stream_pipeline, +) from web.storage import Storage, Workspace logger = logging.getLogger(__name__) @@ -100,7 +122,7 @@ async def healthz() -> dict[str, str]: # ── pipeline ───────────────────────────────────────────────────────────────── -@router.post("/generate") +@router.post("/generate", response_class=HTMLResponse) async def generate( request: Request, background_tasks: BackgroundTasks, @@ -110,11 +132,16 @@ async def generate( gpx: UploadFile | None = None, photos: list[UploadFile] | None = None, ) -> Response: - """Run the pipeline against an upload and redirect to the memory page. + """Validate the upload, run the prep phase, return the generating page. + + The generating page connects to ``GET /generate/{slug}/stream`` via + Server-Sent Events to run the LLM call. We persist the parsed inputs + (``pending.json``) before responding so the SSE endpoint can pick up + even after the BackgroundTask has wiped the raw uploads. Raises a 4xx if the inputs are missing, oversized, or unsupported; - a 502 if the LLM call fails. The 303 redirect is what makes the - "POST then GET" pattern work without resubmitting on refresh. + pipeline parse / photo-load errors surface as 400 here rather than + in the SSE stream so the user gets immediate feedback. """ if gpx is None or not gpx.filename: raise HTTPException(status_code=400, detail="GPX file is required") @@ -132,20 +159,16 @@ async def generate( try: await _save_gpx(gpx, workspace) await _save_photos(photos, workspace) - client = _client_factory(request)() settings = _settings(request) - memory, _ = run_pipeline( + prepare_pipeline( workspace, description=description, style=chosen_style, - client=client, photo_max_edge=settings.photo_max_edge, photo_quality=settings.photo_quality, location=(location or None), ) except PipelineError as exc: - # Pipeline failed after we created the workspace; don't leave - # half-baked state on disk. storage.delete_workspace(workspace) raise HTTPException(status_code=400, detail=str(exc)) from exc except HTTPException: @@ -155,19 +178,84 @@ async def generate( storage.delete_workspace(workspace) raise finally: - # Make sure we always wipe raw uploads even if generation - # succeeded — the background task is the privacy guarantee. + # Wipe raw uploads as soon as the response is sent. The pending + # state captured in ``prepare_pipeline`` already references the + # resized JPEGs, so the SSE call that follows does not need the + # originals. background_tasks.add_task(storage.cleanup_inputs, workspace) _bump_counter() logger.info( - "generated memory %s (style=%s, photos=%d)", + "prepared memory %s for streaming (style=%s)", workspace.slug, chosen_style.value, - len(memory.selected_photos), ) - # 303 because we are switching from POST to GET. - return RedirectResponse(url=f"/memory/{workspace.slug}", status_code=303) + return _templates(request).TemplateResponse( + request, + "generating.html.j2", + { + "slug": workspace.slug, + "style": chosen_style.value, + "retention_minutes": storage.retention_seconds // 60, + }, + ) + + +@router.get("/generate/{slug}/stream") +async def generate_stream(request: Request, slug: str) -> Response: + """Server-Sent Events stream that drives narrative generation. + + Reads the pending state written by ``POST /generate``, calls the + streaming LLM, and emits four kinds of events: + + * ``chunk`` — JSON ``{"text": "..."}`` for each text delta. + * ``status`` — JSON ``{"phase": "writing"|"regenerating"|"rendering"}``. + * ``done`` — JSON ``{"redirect": "/memory/"}`` once the HTML + is rendered. + * ``error`` — JSON ``{"error": "..."}`` if anything fails. + + The SSE format is plain text/event-stream — no framework dependency + on the front end beyond the standard ``EventSource`` API (htmx's + ``sse-swap`` extension also consumes this shape unchanged). + """ + storage = _storage(request) + workspace = storage.get_workspace(slug) + if workspace is None: + raise HTTPException(status_code=404, detail="Memory not found or expired") + if not workspace.pending_state_path.is_file(): + raise HTTPException(status_code=404, detail="Memory has already been generated or expired") + + client = _client_factory(request)() + + def event_stream() -> Iterator[bytes]: + yield _sse_event("status", {"phase": "writing"}) + try: + for event in stream_pipeline(workspace, client=client): + if isinstance(event, PipelineStreamChunk): + yield _sse_event("chunk", {"text": event.text}) + elif isinstance(event, PipelineStreamRetry): + yield _sse_event("status", {"phase": "regenerating", "reason": event.reason}) + elif isinstance(event, PipelineStreamRendered): + yield _sse_event("status", {"phase": "rendering"}) + yield _sse_event("done", {"redirect": f"/memory/{event.slug}"}) + except PipelineError as exc: + logger.warning("stream pipeline failed for %s: %s", slug, exc) + yield _sse_event("error", {"error": str(exc)}) + except Exception: + # Anything that isn't a PipelineError is unexpected — log + # the trace and tell the client we failed without leaking + # internals. + logger.exception("unexpected stream pipeline error for %s", slug) + yield _sse_event("error", {"error": "internal error during generation"}) + raise + + headers = { + # Disable any reverse-proxy buffering — SSE only works if the + # bytes reach the client as they are written. + "Cache-Control": "no-cache", + "X-Accel-Buffering": "no", + } + return StreamingResponse(event_stream(), media_type="text/event-stream", headers=headers) @router.get("/memory/{slug}", response_class=HTMLResponse) @@ -210,6 +298,13 @@ async def memory_carousel_slide(request: Request, slug: str, filename: str) -> R only legal shape is ``NN_.jpg`` — but we still validate the path stays inside the carousel dir so a malicious caller can't escape via ``../``. + + Sets ``Content-Disposition: attachment; filename="-.jpg"`` + so desktop clicks on the fallback download links save with a + meaningful filename. iOS Safari's ``navigator.share({files: [...]})`` + path ignores Content-Disposition — that flow goes through fetched + blobs and an explicit ``File`` constructor — so the header here is + purely for the desktop fallback. """ storage = _storage(request) workspace = storage.get_workspace(slug) @@ -221,7 +316,15 @@ async def memory_carousel_slide(request: Request, slug: str, filename: str) -> R raise HTTPException(status_code=400, detail="Invalid slide path") if not target.is_file(): raise HTTPException(status_code=404, detail="Slide not found") - return FileResponse(target, media_type="image/jpeg") + # Use the slide's own filename — `01_photo.jpg` etc. — namespaced by + # slug so multiple downloads land with distinct names in the user's + # Downloads folder. + download_name = f"{slug}-{target.name}" + return FileResponse( + target, + media_type="image/jpeg", + headers={"Content-Disposition": f'attachment; filename="{download_name}"'}, + ) # ── upload validation + persistence ────────────────────────────────────────── @@ -307,6 +410,26 @@ def _client_factory(request: Request) -> Callable[[], AnthropicClient]: return factory +# ── SSE helpers ────────────────────────────────────────────────────────────── + + +def _sse_event(name: str, data: dict[str, object]) -> bytes: + """Encode a Server-Sent Event with a named event type and JSON payload. + + The wire format is:: + + event: + data: + \n + + The trailing blank line is what tells the browser this event is + complete. We always JSON-encode the payload so the front-end can + parse it with one ``JSON.parse(e.data)`` call. + """ + payload = json.dumps(data, ensure_ascii=False) + return f"event: {name}\ndata: {payload}\n\n".encode() + + # ── counter ────────────────────────────────────────────────────────────────── diff --git a/web/storage.py b/web/storage.py index 7aa60ef..a9e864c 100644 --- a/web/storage.py +++ b/web/storage.py @@ -59,6 +59,12 @@ RESIZED_SUBDIR: Final[str] = "resized" OUTPUT_SUBDIR: Final[str] = "output" STATE_FILENAME: Final[str] = "state.json" +# Written by ``POST /generate`` after parsing the GPX and loading the +# photos but *before* the LLM call. ``GET /generate/{slug}/stream`` reads +# this file to know what to feed the streaming narrative call. Removed +# once the final ``state.json`` has been written so the carousel route's +# state-loader has a single source of truth. +PENDING_STATE_FILENAME: Final[str] = "pending.json" class StorageError(Exception): @@ -96,6 +102,15 @@ def state_path(self) -> Path: """Where the persisted ``Memory`` JSON lives.""" return self.output_dir / STATE_FILENAME + @property + def pending_state_path(self) -> Path: + """Where the pre-LLM (parsed inputs) JSON lives during streaming. + + Written by ``/generate``, read by ``/generate/{slug}/stream``, + and unlinked once the final ``state.json`` has been written. + """ + return self.output_dir / PENDING_STATE_FILENAME + @property def carousel_dir(self) -> Path: """Where ``render_instagram_carousel`` writes its slides. diff --git a/web/templates/generating.html.j2 b/web/templates/generating.html.j2 new file mode 100644 index 0000000..183afb3 --- /dev/null +++ b/web/templates/generating.html.j2 @@ -0,0 +1,90 @@ +{% extends "base.html.j2" %} + +{% block title %}Trailstory — writing your memory…{% endblock %} + +{% block content %} +
+

+ Writing your memory… + One more pass — the model needed a tidier draft… + Rendering the page… +

+

+ The page will open as soon as the narrative is ready. You can keep this + tab in the foreground; closing it doesn't lose the work — the page is + cached for {{ retention_minutes if retention_minutes else 30 }} minutes + at /memory/{{ slug }}. +

+ + +
+ +

+ The first draft didn't parse cleanly. We're trying once more before + giving up — usually fixes itself. +

+ + + + + +

+ Your raw upload has already been deleted. Only the privacy-stripped + JPEGs and the rendered page remain. + Privacy details. +

+
+{% endblock %} diff --git a/web/templates/landing.html.j2 b/web/templates/landing.html.j2 index 6c73163..ffcda04 100644 --- a/web/templates/landing.html.j2 +++ b/web/templates/landing.html.j2 @@ -125,7 +125,7 @@

Files are deleted as soon as your page is rendered. - Read the privacy page. + How we handle your photos →

{% endblock %} diff --git a/web/templates/privacy.html.j2 b/web/templates/privacy.html.j2 index 0bee45f..7a296ea 100644 --- a/web/templates/privacy.html.j2 +++ b/web/templates/privacy.html.j2 @@ -12,35 +12,87 @@ do not retain your raw uploads after the page is rendered.

-

What we do with your upload

+

Lifecycle of one upload

  1. - Your GPX file and photos are written to a temporary directory on the - server, identified only by a random one-time slug. + Upload received. Your GPX file and photos are written + to a temporary directory on the server, identified only by a random + one-time slug. No name, no email, no account.
  2. - We parse the GPX, resize each photo, strip GPS coordinates from photo - metadata, and send a structured prompt — your description plus the hike - stats — to Anthropic's Claude API to write the narrative. + Parsing & resizing. We parse the GPX, resize each + photo to a maximum edge of 1800px, and strip GPS coordinates and other + identifying EXIF metadata from the resized JPEGs. The originals are + never re-served.
  3. - We render a single self-contained HTML file with the photos embedded as - base64 data URIs. The file works offline and can be sent over any - messenger. + Narrative call to Anthropic. A structured prompt — your + description plus the hike statistics (distance, elevation, duration) — + is sent to Anthropic's Claude API to write the EN/RU/DE narrative. + Photo bytes are not sent to the API; only their chronological order + and a numeric index.
  4. - Your raw GPX and original photos are deleted as soon as the - response is sent. Only the resized, GPS-stripped JPEGs and the - rendered HTML page remain. + HTML rendered. A single self-contained HTML file is + written, with the privacy-stripped JPEGs embedded as base64 data URIs. + The file works offline and can be sent over any messenger.
  5. - The rendered page stays on disk for {{ retention_minutes }} minutes so - you can re-share the link or generate an Instagram carousel. After that - window the entire workspace is deleted by an automatic sweep. + Raw upload deleted. Your raw GPX and original photos + are wiped from the temporary directory as soon as the response is + sent — typically within a second of the page loading. Only the + privacy-stripped JPEGs and the rendered HTML page remain. +
    6. +
  6. + Workspace expiry. The rendered page stays on disk for + {{ retention_minutes }} minutes so you can re-share + the link or generate an Instagram carousel. After that window the + entire workspace is deleted by an automatic sweep.
+

Verify it yourself

+ +

+ Trailstory is open source. Every behaviour described above is enforced + by the code in this repository — read it, check it, file an issue if + anything looks wrong: +

+ + +

What we do not do

    @@ -62,15 +114,6 @@ own privacy and retention practices apply to that single API call.

    -

    Verify it yourself

    - -

    - Trailstory is open source. Every behaviour described above is enforced by - the code in this repository — read it, check it, file an issue if anything - looks wrong: - {{ repo_url }}. -

    -

    ← Back to the builder