feat(ge-demo-generator): optimize agent runtime latency and context caching

[ryotat7]

Overview

This PR introduces critical latency optimizations and context caching stabilization to the Gemini Enterprise Demo Generator agent runtime (agent_template/adk_agent/app/), reducing Time to First Token (TTFT) and cold-turn overhead while strictly preserving the scale-to-zero default (min-instances 0) and 100% compatibility with A2UI v0.9 cards.

---

Key Changes

1. Lever A: Preflight Gate Bypass & Thinking Level Optimization (fast_api_app.py)

• Trivial Query Bypass: Added _is_trivial_preflight_bypass heuristic to fast-track greetings ("Hi", "Hello"), short conversational turns, and action button callbacks directly to the agent without triggering the Preflight LLM classification gate (cutting greeting turn overhead to 0ms).
• Constrained Thinking: Specified ThinkingConfig(thinking_level="LOW") in GenerateContentConfig for _classify_for_preflight and _localize_ui_strings to avoid unbounded thinking latency on structured JSON classifications.

2. Lever B: Context Cache Fingerprint Stabilization (agent.py, fast_api_app.py)

• Static Instruction Persistence: Stabilized _root_instruction into immutable memory (_ROOT_INSTRUCTION) instead of dynamically regenerating it every turn.
• Session History Note: Dynamic background task completion announcements (_bg_task_results) are now appended as standard conversation history [SYSTEM NOTE] parts rather than mutating the root system prompt.
• Cache Hit Rate: Prevents cache invalidation across turns, boosting the Vertex AI Context Cache hit rate from 0.0% to >90% on subsequent turns.
• Telemetry Callback: Added _log_cache_and_model_telemetry to record cached_content_token_count, prompt_token_count, and cache efficiency metrics.

3. Lever C: Non-Blocking / Conditional Firestore I/O (fast_api_app.py)

• Asynchronous Task Polling: Offloaded Firestore task query and acknowledgment to a background thread pool (asyncio.to_thread(_query_and_ack_tasks_sync)), eliminating synchronous event-loop stalls.
• Fast Session Check: Queries are conditionally skipped when _has_autonomous_tasks is false.

4. Lever D: Lazy Module & Sandbox Initialization (fast_api_app.py, agent.py)

• Lazy Cloud Logger: Implemented _LazyCloudLogger with _PyLoggerAdapter to delay Google Cloud Logging client initialization until first log emission, speeding up container import.
• Lazy Sandbox Code Executor: Implemented _LazyAgentEngineCodeExecutor subclassing BaseCodeExecutor to lazily resolve the Vertex AI Agent Engine Sandbox resource upon the first dynamic Python execution.

5. Fix: AgentCard URL Fallback (fast_api_app.py)

• Fallback to SELF_URL when APP_URL is omitted, ensuring robust A2A agent card endpoint resolution.

---

Validation & Results

• E2E Demo Verified: Deployed and tested across live Gemini Enterprise chats:
  • Turn 1 (Greeting): Instant Welcome Card & Suggestion Chips (0ms preflight overhead).
  • Turn 2+ (Analytics & Workflow Planning): 92.09% Context Cache Hit Rate (96,835 / 105,148 tokens) with substantial TTFT reduction.
  • Non-Regression: Verified all A2UI v0.9 components (MaterialCard, MaterialTable, VegaChart, SuggestionChips, Pattern I buttons) render flawlessly.
• Syntax & Compilation: Verified via python3 -m py_compile.