Skip to content

feat(skills): self-service skill submission with admin review - #36605

Open
devin-ai-integration[bot] wants to merge 2 commits into
litellm_internal_stagingfrom
litellm_skill_submission_review
Open

feat(skills): self-service skill submission with admin review#36605
devin-ai-integration[bot] wants to merge 2 commits into
litellm_internal_stagingfrom
litellm_skill_submission_review

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Only admins could add skills
  • Users had no way to submit skills
  • Admins had no review step before publishing

How it solves it:

  • Non-admin submissions land pending and unpublished
  • Admins approve or reject, with notes
  • Only approved skills reach the public hub

User Flow

Before: a developer who wants their team's skill on the gateway cannot add it themselves, so an admin has to do every submission by hand

  1. They open http://localhost:4000/ui/?page=skills and see the skills table with no way to add one, since the button is admin-only
  2. They call POST http://localhost:4000/claude-code/plugins with their key and get 401 "Only proxy admin allowed", so the skill never lands
  3. An admin has to register it for them, and whatever the admin registers is published to http://localhost:4000/claude-code/marketplace.json immediately with no review step

After: the same developer submits it themselves, and it stays private until an admin approves it

  1. They open http://localhost:4000/ui/?page=skills, click "+ Submit Skill", fill in the name and GitHub source, and see "Skill submitted for administrator review"
  2. Their row shows a "Pending Review" badge, and GET http://localhost:4000/claude-code/plugins with their key returns the skill with "approval_status": "pending_review" and "enabled": false
  3. GET http://localhost:4000/claude-code/marketplace.json and GET http://localhost:4000/public/skill_hub do not list it, so claude plugin install cannot pick it up yet
  4. An admin opens the same page, clicks "Awaiting review (1)", and clicks Approve on the row, or Reject and types a reason
  5. On approve the badge flips to "Active" and the skill now appears in http://localhost:4000/claude-code/marketplace.json and http://localhost:4000/public/skill_hub
  6. On reject the badge reads "Rejected", the submitter sees the reviewer's note, and the skill stays absent from both public lists

Another user who has nothing to do with the submission cannot see a pending or rejected skill at all: GET http://localhost:4000/claude-code/plugins omits it and GET http://localhost:4000/claude-code/plugins/{name} returns 404 for them, while the submitter and admins can read it

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Captured against a live proxy on localhost:4000 with the Admin UI dev server on localhost:3000, at commit 7652824

Submitting as an internal user, then as an admin:

curl -sX POST localhost:4000/claude-code/plugins -H "Authorization: Bearer $USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"curl-submitted-skill","source":{"source":"github","repo":"org/curl-skill"},"version":"0.9.0"}'
# {"status":"success","action":"submitted_for_review","name":"curl-submitted-skill","enabled":false,"approval_status":"pending_review"}

curl -sX POST localhost:4000/claude-code/plugins -H "Authorization: Bearer $ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"admin-created-skill","source":{"source":"github","repo":"org/admin-skill"},"version":"1.0.0"}'
# {"status":"success","action":"created","name":"admin-created-skill","enabled":true,"approval_status":"active"}

The pending submission is not served anywhere public, and an unrelated internal user cannot see it:

curl -s localhost:4000/claude-code/marketplace.json | jq -r '.plugins[].name'
# admin-created-skill
curl -s localhost:4000/public/skill_hub | jq -r '.skills[].name'
# admin-created-skill
curl -s localhost:4000/claude-code/plugins -H "Authorization: Bearer $OTHER_USER_KEY" | jq -r '.plugins[].name'
# admin-created-skill
curl -so /dev/null -w '%{http_code}\n' localhost:4000/claude-code/plugins/ui-submitted-skill -H "Authorization: Bearer $OTHER_USER_KEY"
# 404

Enabling an unapproved skill is refused, approving publishes it:

curl -sX POST localhost:4000/claude-code/plugins/curl-submitted-skill/enable -H "Authorization: Bearer $ADMIN_KEY"
# 409 {"error":"Skill 'curl-submitted-skill' is awaiting review. Approve it via POST /claude-code/plugins/curl-submitted-skill/approve"}

curl -sX POST localhost:4000/claude-code/plugins/ui-submitted-skill/approve -H "Authorization: Bearer $ADMIN_KEY"
# {"status":"success","name":"ui-submitted-skill","approval_status":"active","enabled":true,"reviewed_by":"...","reviewed_at":"2026-08-12T03:35:..."}
curl -s localhost:4000/claude-code/marketplace.json | jq -r '.plugins[].name'
# admin-created-skill
# ui-submitted-skill

Rejecting keeps it private and stores the note the submitter reads:

curl -sX POST localhost:4000/claude-code/plugins/curl-submitted-skill/reject -H "Authorization: Bearer $ADMIN_KEY" \
  -H "Content-Type: application/json" -d '{"review_notes":"Point the source at the reviewed internal fork"}'
# {"status":"success","name":"curl-submitted-skill","approval_status":"rejected","enabled":false,"review_notes":"Point the source at the reviewed internal fork"}
curl -s localhost:4000/public/skill_hub | jq -r '.skills[].name'
# admin-created-skill
# ui-submitted-skill

UI screenshots for the submit form, the pending badge, the admin review queue, the approve and reject dialogs, and the submitter's view of a rejection are in the docs PR: BerriAI/litellm-docs#870

Type

🆕 New Feature

Caveats (if any)

  • Rows created before this default to active
  • Editing an approved skill sends it back to review
  • Docs land in a separate litellm-docs PR

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/99fe8136193b4643b27bccdea40a1f07
Requested by: @yassin-berriai

Non-admin registrations land as approval_status=pending_review and disabled, so only skills an admin approves reach marketplace.json and the public Skill Hub. Admins approve or reject with notes through new /claude-code/plugins/{name}/approve and /reject routes, and the Skills page gets a submit button for everyone plus a pending queue with approve and reject for admins.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot requested a review from a team August 12, 2026 03:03
@yassin-berriai yassin-berriai self-assigned this Aug 12, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds self-service Claude Code skill submission, administrator review, ownership-scoped visibility, and publication filtering

  • Adds approval metadata and a database migration across all synchronized Prisma schemas
  • Adds approve and reject APIs, submitter update and withdrawal permissions, and public-feed filtering
  • Updates the skills dashboard with submission, review queue, status badges, and review dialogs
  • Expands endpoint, UI, and authorization tests for the new workflow

Confidence Score: 3/5

The PR should not merge until approval is made conditional on the exact skill content the administrator reviewed

Concurrent approval and submitter update requests can leave changed manifest content active and publicly visible without review

Files Needing Attention: litellm/proxy/anthropic_endpoints/claude_code_endpoints/claude_code_marketplace.py

Security Review

A concurrent submitter update and administrator approval can publish manifest content that the administrator did not review

Important Files Changed

Filename Overview
litellm/proxy/anthropic_endpoints/claude_code_endpoints/claude_code_marketplace.py Implements ownership, review, and publication logic, but approval can race with submitter updates and publish unreviewed content
litellm/proxy/public_endpoints/public_endpoints.py Restricts the public Skill Hub to skills that are both approved and enabled
litellm-proxy-extras/litellm_proxy_extras/migrations/20260812000000_add_claudecodeplugin_approval_status/migration.sql Adds nullable review metadata, an active default for legacy rows, and an approval-status index
schema.prisma Adds approval and review fields consistently with the proxy and migration schema copies
litellm/proxy/_types.py Allows authenticated users through route-level checks for submission and owned-resource operations while review routes remain excluded
ui/litellm-dashboard/src/app/(dashboard)/skills/_components/ClaudeCodePluginsPanel.tsx Adds self-service submission controls, administrator review filtering, and review actions
ui/litellm-dashboard/src/components/networking.tsx Adds the dashboard API wrapper for approving and rejecting submitted skills

Reviews (1): Last reviewed commit: "feat(skills): self-service skill submiss..." | Re-trigger Greptile

Comment on lines +685 to +695
plugin: Final[_PluginRecord] = await repository.table.update(
where={"name": plugin_name}, # mutable-ok: prisma query arguments must be plain dicts
data={ # mutable-ok: prisma query arguments must be plain dicts
"approval_status": approval_status,
"review_notes": review_notes,
"reviewed_by": user_api_key_dict.user_id,
"reviewed_at": reviewed_at,
"enabled": approval_status == SKILL_ACTIVE,
"updated_at": reviewed_at,
},
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Approval races content updates

If approval overlaps a submitter update, it can activate the changed manifest without review, publishing unreviewed content. How this was verified: Both paths independently update the same row without concurrency control.

Comment on lines +675 to +685
existing: Final[_PluginRecord | None] = await repository.table.find_unique(
where={"name": plugin_name} # mutable-ok: prisma query arguments must be plain dicts
)
if not existing:
raise _error_response(404, f"Plugin '{plugin_name}' not found")

if _as_approval_status(existing.approval_status) == approval_status:
raise _error_response(400, f"Skill '{plugin_name}' is already {approval_status}")

reviewed_at: Final = datetime.now(timezone.utc)
plugin: Final[_PluginRecord] = await repository.table.update(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Review path queries database directly

_record_review directly reads and updates the table, bypassing the required request-path database helper convention and increasing regression risk.

Rule Used: What: In critical path of request, there should be... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.75281% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...s/claude_code_endpoints/claude_code_marketplace.py 96.96% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@codspeed-hq

codspeed-hq Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_skill_submission_review (4cccfe6) with litellm_internal_staging (b4f5e46)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants