Skip to content

Expose bounded historical imports as authenticated async HTTP jobs #378

Description

@cdelalama

msgvault serve already exposes POST /api/v1/sync/{account} to trigger an incremental sync (handleTriggerSync -> scheduler.TriggerSync), and msgvault sync-full already supports bounded historical imports via --after, --before, --limit, --query, and --noresume.

The missing piece for remote deployments and operator UIs is an authenticated async HTTP surface for sync-full-style imports with a job_id and a status endpoint, similar in spirit to how #130 added aggregate API endpoints to support the remote TUI.

Gap

  • POST /api/v1/sync/{account} only triggers an incremental sync and requires the account to already be scheduled, so it does not cover an initial bounded import for a newly added account.
  • The trigger is fire-and-forget (202 Accepted, no id), so a UI cannot follow a long import's progress, even though the syncer already reports progress and final summary counts.

Requested shape

Exact routing is your call. This could extend POST /api/v1/sync/{account} with an optional body, or be a sibling endpoint such as POST /api/v1/imports.

Inputs:

  • account/email
  • after
  • before
  • limit
  • query
  • noresume

Response:

  • 202 Accepted
  • job_id

Status endpoint:

  • GET /api/v1/imports/{job_id}
  • status: queued/running/done/failed
  • processed/added/skipped while running
  • final summary counts when complete
  • sanitized error
  • started_at / finished_at

In-memory job tracking would be fine for a first cut; persistence across restarts is entirely your call.

Why

A web UI can already inspect accounts, stats, search, and scheduler state over HTTP, but historical imports still require spawning msgvault sync-full locally. That forces deployment wrappers to add a local bridge/sidecar just to call the CLI from a web UI.

We built one for a NAS deployment; happy to share the shape as evidence, but the goal is not for msgvault to implement our wrapper policy. The useful upstream primitive would be the import job API itself.

Non-goals

  • Not backup/restore; that is separate and already tracked in Add first-class backup and restore support for Msgvault state #339.
  • Not Docker control.
  • Not token exposure.
  • Not arbitrary command execution.
  • Not asking msgvault to match any wrapper's policy. Local deployments can keep their own auth, backup gates, and audit logs; msgvault would own only the import primitive and job lifecycle.

Happy to send a PR if this shape sounds right.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions