Skip to content

fix(sharepoint_online): recover from stale drive deltas and sync content via Graph - #4370

Draft
Jan-Kazlouski-elastic wants to merge 2 commits into
mainfrom
fix/1940-spo-graph-content-and-delta-recovery
Draft

fix(sharepoint_online): recover from stale drive deltas and sync content via Graph#4370
Jan-Kazlouski-elastic wants to merge 2 commits into
mainfrom
fix/1940-spo-graph-content-and-delta-recovery

Conversation

@Jan-Kazlouski-elastic

Copy link
Copy Markdown
Contributor

Relates to https://github.com/elastic/sdh-search/issues/1940

Fixes SharePoint Online sync failures for tenants using client-secret auth with DLS disabled, as reported in the SDH above.

  • Recover from Graph drive delta 410 Gone on any delta request (full sync, incremental, and mid-pagination) with a single restart from root/delta per drive
  • Route list attachments and site pages through Microsoft Graph instead of SharePoint REST (avoids Unauthorized with app-only credentials)
  • Fetch site page details (canvasLayout, sharepointIds) for body content and stable SharePoint list item ids
  • Use deterministic attachment document ids for stable indexing across syncs
  • Warn in validate_config when DLS is enabled with client-secret auth (REST permissions path limitation)

Checklists

Pre-Review Checklist

  • this PR does NOT contain credentials of any kind, such as API keys or username/passwords (double check config.yml.example)
  • this PR has a meaningful title
  • this PR links to all relevant github issues that it fixes or partially addresses
  • if there is no GH issue, please create it. Each PR should have a link to an issue
  • this PR has a thorough description
  • Covered the changes with automated tests
  • Tested the changes locally
  • Added a label for each target release version (example: v7.13.2, v7.14.0, v8.0.0)
  • For bugfixes: backport safely to all minor branches still receiving patch releases
  • Considered corresponding documentation changes
  • Contributed any configuration settings changes to the configuration reference
  • if you added or changed Rich Configurable Fields for a Native Connector, you made a corresponding PR in Kibana

Changes Requiring Extra Attention

  • Security-related changes (encryption, TLS, SSRF, etc)
  • New external service dependencies added.

Related Pull Requests

Release Note

SharePoint Online connector now recovers automatically from expired drive delta links (410 Gone) and fetches site pages and list attachments via Microsoft Graph, improving sync reliability for app-only authentication.

… paths

Recover incremental sync when drive delta tokens expire (410 Gone),
fetch list attachments and site pages via Microsoft Graph instead of
SharePoint REST, and warn when DLS is enabled with client-secret auth.

Part of elastic/sdh-search#1940
- Recover from 410 Gone on any drive delta request (full sync, incremental,
  and mid-pagination) with a single restart from root/delta
- Fetch site page details (canvasLayout, sharepointIds) via Graph for content
  and stable SharePoint list item ids for DLS REST compatibility
- Use stable attachment document ids from list/item/attachment graph ids
- Add clear_drive_delta_link helper and expanded unit tests

Part of elastic/sdh-search#1940
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant