Skip to content

PERFORMANCE_OPTIMIZATIONS.md

Latest commit

 

History

History
69 lines (53 loc) · 2.42 KB

PERFORMANCE_OPTIMIZATIONS.md

File metadata and controls

69 lines (53 loc) · 2.42 KB

Performance Optimizations Applied

Speed Improvements Made

1. Parallel LLM Calls (2x faster)

  • Before: Recruiter and archetype LLM calls ran sequentially (~20-30s total)
  • After: Both calls run in parallel using asyncio.gather() (~10-15s total)
  • Impact: ~50% reduction in LLM wait time

2. Reduced LLM Payload Size

  • Before: Sent full profile JSON dump (could be 50KB+)
  • After: Created compact summary (~500 bytes) with only essential info
  • Impact: Faster LLM processing, lower costs

3. Limited Token Usage

  • Before: No token limits (LLM could generate long responses)
  • After: max_tokens=300 for recruiter, max_tokens=200 for archetype
  • Impact: Faster responses, more predictable timing

4. Optimized GitHub API Calls

  • Before: Checked ALL repos with detailed API calls
  • After:
    • Only process top 30 repos by stars
    • Skip detailed checks for forks (they're less important)
    • Limit commit activity to 200 events
  • Impact: ~70% reduction in GitHub API calls

5. Reduced Timeout

  • Before: 30 second timeout
  • After: 20 second timeout
  • Impact: Faster failure detection, better UX

6. Better Error Handling

  • Before: Commit activity could fail silently
  • After: Graceful fallback if commit activity fails
  • Impact: More reliable, doesn't block entire request

7. Frontend Loading States

  • Before: Generic "Analyzing..." message
  • After: Progress messages ("Fetching GitHub profile...", "Computing scores...")
  • Impact: Better perceived performance

Expected Performance

Before Optimizations:

  • GitHub fetch: 10-20 seconds (many API calls)
  • LLM calls: 20-30 seconds (sequential)
  • Total: 30-50 seconds

After Optimizations:

  • GitHub fetch: 3-8 seconds (limited repos)
  • LLM calls: 10-15 seconds (parallel)
  • Total: 13-23 seconds

~60% faster overall!

Trade-offs

  1. Top 30 repos only: Still captures most important projects (sorted by stars)
  2. Limited commit activity: Still shows trend, just less granular
  3. Compact LLM summary: Still includes all scoring dimensions, just more concise

Further Optimizations Possible

If still too slow, consider:

  1. Caching: Cache GitHub profile data for 1 hour
  2. Streaming: Stream results as they come in
  3. Background jobs: Move LLM calls to background queue
  4. CDN: Cache static frontend assets