Skip to content

fix(embed): skip oversized chunks instead of crashing#339

Open
JasonOA888 wants to merge 1 commit intotobi:mainfrom
JasonOA888:fix/embed-oversized-chunks
Open

fix(embed): skip oversized chunks instead of crashing#339
JasonOA888 wants to merge 1 commit intotobi:mainfrom
JasonOA888:fix/embed-oversized-chunks

Conversation

@JasonOA888
Copy link

Problem

qmd embed crashes when the first chunk exceeds the embedding model context window (2048 tokens for EmbeddingGemma-300M):

Error: Failed to get embedding dimensions from first chunk

On Apple Silicon, this can also trigger a GGML Metal cleanup assertion resulting in SIGABRT.

Solution

Instead of failing on the first chunk, try embedding chunks sequentially until one succeeds:

  1. Iterate through chunks attempting to embed each one
  2. Log warning for chunks that fail (oversized, context window exceeded)
  3. Use first successful embedding to determine vector dimensions
  4. Only throw error if ALL chunks fail

Behavior Changes

  • Before: Crash if first chunk exceeds context window
  • After: Skip oversized chunks with warning, continue with valid chunks

Fixes #303

When first chunk exceeds embedding model context window (e.g., 2048
tokens for EmbeddingGemma), qmd embed crashes with unhandled error.

Changes:
- Try embedding chunks sequentially until one succeeds to get dimensions
- Log warning for skipped chunks that fail embedding
- Only throw error if ALL chunks fail (not just first)
- Prevents SIGABRT on Apple Silicon from GGML Metal cleanup assertion

Fixes tobi#303
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

qmd embed crashes when first chunk exceeds EmbeddingGemma context size (should skip gracefully)

1 participant