feat(memory): Cross Agent Memory

[RyamL1221]

feat: Cross-Agent Shared Memory & Natural Language Injection

Summary

Elevates the kernel's memory subsystem from agent-scoped storage to a first-class OS service for cross-agent personalization. Agents like ProfileAgent and TaskAgent can now write shared memories that the kernel automatically discovers and injects into AssistantAgent's LLM context — with JSON content converted to natural language so smaller models can actually use it.

Problem

The kernel's ContextInjector resolved user_id from the requesting agent's name instead of from memory metadata. When assistant_agent made an llm_chat call, the injector searched for user_id=assistant_agent, missing all profile/task memories written by other agents under the real user identifier (e.g., user_id=alex_chen). Additionally, raw JSON injected into system prompts hurt small model performance — Phase 2 benchmark scores were worse than Phase 1 across 30 trials.

Changes

Cross-Agent Shared Memory Retrieval

• _apply_sharing_filter() in providers/base.py — centralized post-retrieval filter enforcing the privacy invariant: memories from other agents are only returned when sharing_policy="shared"
• MemoryManager.known_user_ids registry — tracks real user_ids as agents write memories, enabling cross-agent discovery without requiring the requesting agent to have prior memories
• _analyze_query_to_memory() fix — preserves the full SDK metadata dict (user_id, owner_agent, sharing_policy, memory_type) on MemoryNote.metadata instead of silently dropping it
• Mem0Provider API fix — client.search() now uses filters={"user_id": ...} instead of top-level kwargs (required by Mem0 >= 0.1.29)
• Cross-agent metadata storage — Mem0Provider.add_memory() writes owner_agent, sharing_policy, memory_type into the Mem0 metadata dict so _apply_sharing_filter() can read them on retrieval
• Over-fetch strategy — shared retrieval requests 4× max_memories from Mem0 to prevent private conversation memories from crowding out shared memories in the native relevance ranking

User ID Propagation

• ContextInjector exposes resolved_user_id in injection diagnostics
• SyscallExecutor passes resolved_user_id to ConversationExtractor.extract_async()
• ConversationExtractor stores conversation memories under the real user_id (not agent name), with full cross-agent metadata (owner_agent, memory_type, sharing_policy)

Natural Language Memory Formatting

• MemoryFormatter (memory_formatter.py) — pure function converting JSON to natural language at inject time, with three tiers:
  • "profile" → hardcoded template: "User profile: Their name is Alice. They prefer coding in Python..."
  • "task_context" → hardcoded template: "Current task context: Working on project AIOS. Goals: fix bug..."
  • Any other memory_type with valid JSON → generic fallback: type name becomes label (e.g., "research_notes" → "Research notes: topic: transformers. summary: attention is all you need.")
  • "conversation" / plain text → pass-through
• Integrated into ContextInjector.inject() between relevance filtering and token budget truncation
• Generic fallback means any agent can define custom memory_type values and get readable formatting without kernel changes

Injection Diagnostics

• inject() returns tuple[LLMQuery, dict] with structured metrics: candidate_count, injected_count, source_agents, memory_types, prompt_tokens_before/after, resolved_user_id

Files Changed (10 files, +1103 / -127)

File	Change
aios/memory/context_injector.py	Cross-agent retrieval, known_user_ids fallback, formatting step, diagnostics, over-fetch, logging
aios/memory/memory_formatter.py	New — format_memory(), _format_profile(), _format_task_context(), _format_generic()
aios/memory/manager.py	known_user_ids registry, metadata preservation in _analyze_query_to_memory()
aios/memory/conversation_extractor.py	user_id parameter, full cross-agent metadata on stored memories
aios/memory/providers/base.py	_apply_sharing_filter(), _enrich_metadata()
aios/memory/providers/mem0.py	filters API, cross-agent metadata storage, search/retrieval filtering
aios/memory/providers/in_house.py	Integrated _apply_sharing_filter() and _enrich_metadata()
aios/memory/providers/zep.py	Integrated _apply_sharing_filter() and _enrich_metadata()
aios/memory/note.py	Added metadata dict attribute
aios/syscall/syscall.py	Diagnostics unpacking, resolved_user_id propagation to extractor

Commit History

Commit	Description
de722ef	Offloaded memory operations to the kernel — _apply_sharing_filter(), _enrich_metadata(), provider integration, MemoryNote.metadata
4b32f89	Fixed agent memory retrieval — MemoryManager metadata preservation, agent_name injection into query params
d28659c	Cross-agent memory discovery — known_user_ids registry, restructured inject() to not early-return on empty own memories
9da127c	Fixed user_id namespace mismatch — ConversationExtractor propagates resolved_user_id, SyscallExecutor wiring
db068bd	Fixed context injector for Mem0 — filters API, over-fetch strategy, diagnostic logging
0d7094d	Natural language formatting — MemoryFormatter module with hardcoded profile/task templates and generic JSON fallback, integration into ContextInjector

Backward Compatibility

• Agents that don't write user_id or sharing_policy in metadata continue to work as before (own-agent-scoped injection only)
• The sharing_policy filter ensures no private memories leak cross-agent
• Existing auto_inject: true/false and auto_extract: true/false config flags are unchanged
• No SDK (Cerebrum) changes required — agents continue to use create_memory() and llm_chat() as before

Testing

Validated via the Cerebrum SDK benchmark harness:

python benchmarks/shared_memory/run_evaluation.py --trials 30 --condition phase1
# restart kernel
python benchmarks/shared_memory/run_evaluation.py --trials 30 --condition phase2

• Phase 1 (private): 0 cross-agent memories injected ✓
• Phase 2 (shared): profile + task_context memories injected with natural language formatting ✓

Known Limitations

• known_user_ids is in-memory only — not persisted across kernel restarts
• Mem0 fact extraction with small local models (qwen2.5:7b) may fail, limiting cross-restart recall
• Over-fetch (4×) may become inefficient with very large memory stores
• Single user_id assumption in the known_user_ids fallback — multi-user scenarios need SDK-level user_id propagation