Skip to content

Extract paginatedFetch and fetchWithRetry into src/api-utils.ts#19

Closed
shouze wants to merge 1 commit intomainfrom
refactor/dry-api-pagination-retry
Closed

Extract paginatedFetch and fetchWithRetry into src/api-utils.ts#19
shouze wants to merge 1 commit intomainfrom
refactor/dry-api-pagination-retry

Conversation

@shouze
Copy link
Contributor

@shouze shouze commented Feb 22, 2026

What does this PR do?

  • Add src/api-utils.ts with two pure-async helpers:
    • paginatedFetch: replaces the three duplicated while-pagination loops
    • fetchWithRetry: retries on 429/503 with exponential backoff and Retry-After header support; returns last response after maxRetries exhausted
  • Add src/api-utils.test.ts with 14 unit tests (mocked fetch, instant timers)
  • Refactor src/api.ts to consume the new helpers:
    • searchCode: use fetchWithRetry instead of bare fetch
    • fetchAllResults: use fetchWithRetry for raw file content resolution
    • fetchRepoTeams (list teams): replace while loop with paginatedFetch
    • fetchRepoTeams (repos per team): replace while loop with paginatedFetch
    • Fix: silent break on HTTP error for team repos now logs a warning and returns [], instead of swallowing errors silently
    • Extract githubHeaders() helper to deduplicate header objects

Closes #17

How did you verify your code works?

- Add src/api-utils.ts with two pure-async helpers:
  - paginatedFetch<T>: replaces the three duplicated while-pagination loops
  - fetchWithRetry: retries on 429/503 with exponential backoff and Retry-After
    header support; returns last response after maxRetries exhausted
- Add src/api-utils.test.ts with 14 unit tests (mocked fetch, instant timers)
- Refactor src/api.ts to consume the new helpers:
  - searchCode: use fetchWithRetry instead of bare fetch
  - fetchAllResults: use fetchWithRetry for raw file content resolution
  - fetchRepoTeams (list teams): replace while loop with paginatedFetch
  - fetchRepoTeams (repos per team): replace while loop with paginatedFetch
  - Fix: silent break on HTTP error for team repos now logs a warning and
    returns [], instead of swallowing errors silently
  - Extract githubHeaders() helper to deduplicate header objects

Closes #17
@shouze shouze self-assigned this Feb 22, 2026
Copilot AI review requested due to automatic review settings February 22, 2026 23:02
@github-actions
Copy link

Coverage after merging refactor/dry-api-pagination-retry into main will be

97.35%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src
   aggregate.ts100%100%100%100%
   api-utils.ts97.96%100%83.33%100%
   api.ts98.40%100%100%98.10%243–245
   group.ts100%100%100%100%
   output.ts99.12%100%94.74%99.52%58
   render.ts97.20%100%100%97.14%142–145
   upgrade.ts77.01%100%88.89%75.64%128, 130–132, 79–93
src/render
   filter.ts100%100%100%100%
   highlight.ts99.29%100%100%99.01%184–185
   rows.ts100%100%100%100%
   selection.ts100%100%100%100%
   summary.ts100%100%100%100%

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extracts shared pagination and retry logic from src/api.ts into a new helper module, adds unit tests for the extracted helpers, and updates the GitHub API client to use the new implementations (including improved handling of team-repo fetch errors).

Changes:

  • Add fetchWithRetry (429/503 retry with backoff + Retry-After support) and paginatedFetch helpers in a new src/api-utils.ts.
  • Add src/api-utils.test.ts to unit test retry and pagination behavior with mocked fetch and instant timers.
  • Refactor src/api.ts to use the new helpers and deduplicate request headers via a githubHeaders() helper.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/api.ts Refactors API calls to use retry/pagination helpers and consolidates GitHub headers.
src/api-utils.ts Introduces shared retry + pagination helper functions.
src/api-utils.test.ts Adds unit coverage for the new helpers with mocked globals.

Comment on lines +19 to +23
export async function fetchWithRetry(
url: string,
options: RequestInit,
maxRetries = 3,
): Promise<Response> {
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/api-utils.ts introduces a new side-effectful module (uses fetch/setTimeout) even though the repo docs state that I/O should be isolated to api.ts, tui.ts, and github-code-search.ts (see AGENTS.md:94-96). To stay within that boundary, either keep these helpers inside api.ts, or make api-utils.ts purely computational (e.g., export backoff/pagination logic that takes injected fetch/sleep functions from api.ts). If the intent is to expand the allowed I/O surface, the architecture docs should be updated accordingly.

Copilot uses AI. Check for mistakes.
Comment on lines +3 to +4
// Pure-async helpers with no side effects beyond network I/O. These are the
// only place in the codebase that knows about GitHub rate-limit semantics.
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file header describes these as “Pure-async helpers with no side effects beyond network I/O”, but the implementation also depends on timers and nondeterminism (setTimeout and Math.random() jitter). Please adjust the comment to match reality (or remove jitter) so future readers don’t treat this module as pure/deterministic.

Suggested change
// Pure-async helpers with no side effects beyond network I/O. These are the
// only place in the codebase that knows about GitHub rate-limit semantics.
// Async helpers that encapsulate pagination, retry, and GitHub rate-limit
// semantics for network calls.

Copilot uses AI. Check for mistakes.
Comment on lines +153 to +154
const result = await paginatedFetch(async (page) => pages[page - 1] ?? [], 2);
void callCount;
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test declares callCount but never increments or asserts on it, and void callCount; just masks the unused-variable warning. Either remove callCount entirely or use it to assert how many pages were fetched, so the test meaningfully verifies multi-page behavior.

Suggested change
const result = await paginatedFetch(async (page) => pages[page - 1] ?? [], 2);
void callCount;
const result = await paginatedFetch(async (page) => {
callCount++;
return pages[page - 1] ?? [];
}, 2);
expect(callCount).toBe(3);

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor: extract pagination helper and exponential backoff retry into src/api-utils.ts

2 participants