Skip to content

pdp: accept urls array and provider host/cids in pull piece requests - #1522

Open
snadrus wants to merge 3 commits into
mainfrom
cursor/pdpv0-pull-piece-url-sources-a02b
Open

snadrus wants to merge 3 commits into
mainfrom
cursor/pdpv0-pull-piece-url-sources-a02b

Conversation

@snadrus

@snadrus snadrus commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Fixes #1252

Summary

POST /pdp/piece/pull still accepts the current pieces: [{pieceCid, sourceUrl}] form. Additional sources are top-level so there is no nested plural under pieces[]:

{
  "pieces": [
    { "pieceCid": "<CommP-v2-CID>", "sourceUrl": "https://example.com/piece/bafy..." }
  ],
  "urls": ["https://backup.example.com/piece/bafy..."],
  "provider": {
    "host": "sp.example.com",
    "cids": ["<optional-retrieval-CID>"]
  }
}

All three fields are optional. Curio assembles them into the same per-URL pull item list used today.

  • urls entries must end in /piece/{cid}; the CID is taken from the path.
  • Provider URLs are assembled as https://{host}/piece/{cid}.
  • If provider.cids is omitted, piece CIDs already collected from pieces and urls are used.
  • provider.cids without provider.host is rejected.

Test plan

  • AssembledSources() covers legacy pieces, top-level urls, provider host/cids, host fallback to known CIDs, JSON form, and de-dupe
  • provider.cids without provider.host returns an error
  • Handler tests for mixed sources and provider-only requests
  • Existing pull validation and handler tests still pass (go test ./pdp -tags skiff for the non-DB pull cases)
  • DB-backed backpressure tests were not run here (no local Yugabyte)
Open in Web Open in Cursor 

Keep the existing sourceUrl field and also accept optional urls and
provider {host, cids}. The receiving provider assembles these into the
same per-URL pull item list used today.

Co-authored-by: andy <andy@minimintfarm.com>
@FilOzzy FilOzzy added the team/fs-wg Items being worked on or tracked by the "FS Working Group". See FilOzone/github-mgmt #10 label Sep 9, 2026
@FilOzzy FilOzzy added this to FOC Sep 9, 2026
@github-project-automation github-project-automation Bot moved this to 📌 Triage in FOC Sep 9, 2026
Return a clear validation error whenever cids are supplied without a
host, including when other source URLs are also present.

Co-authored-by: andy <andy@minimintfarm.com>
@FilOzzy FilOzzy moved this from 📌 Triage to ⌨️ In Progress in FOC Sep 9, 2026
pieces[] stays the current {pieceCid, sourceUrl} form. urls and
provider live at the top level so there is no nested plural under
pieces. The receiving provider still assembles them into the same
per-URL pull item list.

Co-authored-by: andy <andy@minimintfarm.com>
@snadrus
snadrus marked this pull request as ready for review September 9, 2026 23:30
@snadrus
snadrus requested review from a team and ZenGround0 as code owners September 9, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team/fs-wg Items being worked on or tracked by the "FS Working Group". See FilOzone/github-mgmt #10

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PDPv0: pull piece should accept an array of URLs instead of just 1

3 participants