-
Notifications
You must be signed in to change notification settings - Fork 18
performance: switch to object-based cursor from offset-based page cursors #86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…sor for pagination - response header will use Link: <https://api.github.com/organizations/###/secret-scanning/alerts?per_page=1&after=CAESBggBEgIIAioTCgwIwb6GxwYQwICyiQMQnZGNRP&secret_type=password>; rel="next" instead of Link: <https://api.github.com/organizations/###/secret-scanning/alerts?per_page=1&page=2>; rel="next", <https://api.github.com/organizations/###/secret-scanning/alerts?per_page=1&page=93323>; rel="last"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR switches GitHub API endpoints from offset-based pagination to cursor-based pagination for better performance when handling large numbers of alerts, avoiding potential 503 responses.
- Replaces
page=1query parameters withafter=to force object-based cursor pagination - Updates URL construction across secret scanning, Dependabot, and code scanning modules
- Maintains same functionality while improving API performance for large datasets
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/secret_scanning.py | Updates API URLs to use cursor-based pagination for repository, organization, and enterprise secret scanning alerts |
| src/dependabot.py | Updates API URLs to use cursor-based pagination for Dependabot alerts across all scopes |
| src/code_scanning.py | Updates API URLs to use cursor-based pagination for code scanning alerts across all scopes |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| - List of _all_ dependency alerts on the repository | ||
| """ | ||
| url = f"{api_endpoint}/repos/{repo_name}/dependabot/alerts?per_page=100&page=1" | ||
| url = f"{api_endpoint}/repos/{repo_name}/dependabot/alerts?per_page=100&after=" |
Copilot
AI
Oct 6, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The empty after= parameter may not be valid for GitHub's API. Consider either omitting the after parameter entirely for the first request or using a valid cursor value.
| - List of _all_ code scanning alerts on the repository | ||
| """ | ||
| url = f"{api_endpoint}/repos/{repo_name}/code-scanning/alerts?per_page=100&page=1" | ||
| url = f"{api_endpoint}/repos/{repo_name}/code-scanning/alerts?per_page=100&after=" |
Copilot
AI
Oct 6, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The empty after= parameter may not be valid for GitHub's API. Consider either omitting the after parameter entirely for the first request or using a valid cursor value.
Improve performance for large numbers of alerts - avoid potential 503 responses
Response header will now use object IDs:
Instead of page based:
At least with dependabot, omitting the
afterempty param falls back to page number based cursor. Using that as aafter=placeholder to force the behavior.