Skip to content

Conversation

@felickz
Copy link
Contributor

@felickz felickz commented Oct 6, 2025

Improve performance for large numbers of alerts - avoid potential 503 responses

Response header will now use object IDs:

Link: <https://api.github.com/organizations/###/secret-scanning/alerts?per_page=1&after=CAESBggBEgIIAioTCgwIwb6GxwYQwICyiQMQnZGNRP&secret_type=password>; rel="next"

Instead of page based:

Link: <https://api.github.com/organizations/###/secret-scanning/alerts?per_page=1&page=2>; rel="next", <https://api.github.com/organizations/###/secret-scanning/alerts?per_page=1&page=93323>; rel="last"

At least with dependabot, omitting the after empty param falls back to page number based cursor. Using that as a after= placeholder to force the behavior.

> gh api "/enterprises/ENT/dependabot/alerts" --include
HTTP/2.0 200 OK
...
Link: https://api.github.com/enterprises/ENT/dependabot/alerts?per_page=1&page=2; rel="next", https://api.github.com/enterprises/ENT/dependabot/alerts?per_page=1&page=104697; rel="last"

@felickz felickz requested a review from aegilops as a code owner October 6, 2025 17:20
@felickz felickz requested a review from Copilot October 6, 2025 18:18
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR switches GitHub API endpoints from offset-based pagination to cursor-based pagination for better performance when handling large numbers of alerts, avoiding potential 503 responses.

  • Replaces page=1 query parameters with after= to force object-based cursor pagination
  • Updates URL construction across secret scanning, Dependabot, and code scanning modules
  • Maintains same functionality while improving API performance for large datasets

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/secret_scanning.py Updates API URLs to use cursor-based pagination for repository, organization, and enterprise secret scanning alerts
src/dependabot.py Updates API URLs to use cursor-based pagination for Dependabot alerts across all scopes
src/code_scanning.py Updates API URLs to use cursor-based pagination for code scanning alerts across all scopes

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@felickz felickz requested a review from Copilot October 6, 2025 18:31
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

- List of _all_ dependency alerts on the repository
"""
url = f"{api_endpoint}/repos/{repo_name}/dependabot/alerts?per_page=100&page=1"
url = f"{api_endpoint}/repos/{repo_name}/dependabot/alerts?per_page=100&after="
Copy link

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The empty after= parameter may not be valid for GitHub's API. Consider either omitting the after parameter entirely for the first request or using a valid cursor value.

Copilot uses AI. Check for mistakes.
- List of _all_ code scanning alerts on the repository
"""
url = f"{api_endpoint}/repos/{repo_name}/code-scanning/alerts?per_page=100&page=1"
url = f"{api_endpoint}/repos/{repo_name}/code-scanning/alerts?per_page=100&after="
Copy link

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The empty after= parameter may not be valid for GitHub's API. Consider either omitting the after parameter entirely for the first request or using a valid cursor value.

Copilot uses AI. Check for mistakes.
@felickz felickz merged commit 2b801a8 into main Oct 7, 2025
8 checks passed
@felickz felickz deleted the id-based-cursors branch October 7, 2025 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants