feat: add Google AI (Gemini) embeddings for files and passages

[wayne-accelerandos]

Problem

When using Google AI embedding models (e.g., google_ai/gemini-embedding-001), two separate code paths were broken:

1. Passages/archival memory — request_embeddings in GoogleVertexClient raised NotImplementedError, causing 500 errors on any archival memory operation with a Google AI embedding config.

2. Files/sources — sources.py, folders.py, and agent_serialization_manager.py always used OpenAIEmbedder regardless of the source's embedding_config. This silently produced OpenAI embedding vectors for file chunks. Since semantic_search_files queries use the agent's Google AI embedding config, search vectors never matched indexed file vectors, returning empty or incorrect results.

Changes

1. Implement GoogleVertexClient.request_embeddings() (letta/llm_api/google_vertex_client.py)

• Replaces empty/whitespace strings with single space (Google AI rejects empty strings)
• Batches at max 100 texts per call (Google AI limit vs OpenAI's 2048)
• Parallel batch processing via asyncio.gather()

Both GoogleAIClient and GoogleVertexClient inherit this implementation.

2. Add GoogleAIEmbedder (letta/services/file_processor/embedder/google_ai_embedder.py)

New embedder for the file processing pipeline, mirroring the OpenAIEmbedder interface:

• Global semaphore (3 concurrent requests) to avoid API quota exhaustion
• Recursive halving retry on batch failure (same strategy as OpenAIEmbedder)
• OpenTelemetry tracing via log_event

3. Route Google AI through GoogleAIEmbedder (sources.py, folders.py, agent_serialization_manager.py)

Added elif embedding_config.embedding_endpoint_type == "google_ai" before the OpenAI fallback in the existing embedder selection chain: Turbopuffer → Pinecone → Google AI → OpenAI.

Why the embedder must match

The embedding model used to index file chunks must be the same model used to embed search queries. Mixing OpenAI vectors (file index) with Google AI vectors (search query) produces semantically meaningless results — the vector spaces are incompatible.

Tests Added

• tests/test_google_ai_file_embedder.py — 12 unit tests for GoogleAIEmbedder (all mocked, no API key needed)
• tests/test_embeddings.py — 14 new unit tests for request_embeddings() on both GoogleAIClient and GoogleVertexClient (all mocked)
• tests/test_google_ai_file_embeddings_integration.py — 4 integration tests (require GEMINI_API_KEY, skipped otherwise)
• tests/configs/embedding_model_configs/google_ai_embed.json — config for parametrized embedding tests

Note: There are 3 pre-existing failures in tests/test_file_processor.py unrelated to this PR.

Running the tests

# Unit tests (no API key needed)
uv run pytest tests/test_google_ai_file_embedder.py tests/test_embeddings.py -m "unit and google" -v

# Integration tests (requires GEMINI_API_KEY)
GEMINI_API_KEY=your_key uv run pytest tests/test_google_ai_file_embeddings_integration.py -v