diff --git a/CHANGELOG.md b/CHANGELOG.md index f95bfc3..bec209c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,91 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/). +## [2.0.0] - 2026-03-30 + +Dhee V2: Self-Evolving Cognition Plugin. This release transforms Dhee from a memory layer into a **self-improving cognition plugin** that can make any agent — local or cloud, software or embodied — a HyperAgent that gets better with every interaction. + +### Added — Phase 1: Universal Plugin + +- **DheePlugin** (`dhee/adapters/base.py`): Framework-agnostic entry point wrapping Engram + Buddhi behind 4 tools (remember/recall/context/checkpoint), with session lifecycle (frozen snapshot pattern) and trajectory recording for skill mining. +- **DheeEdge** (`dhee/edge/`): Minimal-footprint offline plugin for hardware/humanoid deployment. All-local inference (GGUF + ONNX), embodiment hooks (`on_sensor_input`, `on_action_result`, `predict_environment`), <500MB working set. +- **BuddhiMini** (`dhee/mini/`): Scaffold for trainable model with 3 new task heads (`[MEMORY_OP]`, `[HEURISTIC]`, `[RETRIEVAL_JUDGE]`) on top of DheeModel. Includes `TraceSegmenter` that splits agent trajectories into `[REASON]/[ACT]/[MEMORY_OP]` spans for structured training data. +- Export `DheePlugin` from `dhee.__init__` and `dhee/adapters/__init__`. +- `pyproject.toml`: Added `edge` optional dependency group. +- `SamskaraCollector.get_training_data()`: Exports SFT samples, DPO pairs, and vasana reports for the training pipeline. +- `DheeLLM`: 3 new convenience methods (`classify_memory_op`, `generate_heuristic`, `judge_retrieval`). + +### Added — Phase 2: Self-Evolving Cognition + +- **ContrastiveStore** (`dhee/core/contrastive.py`): Success/failure pair storage with MaTTS re-ranking. Inspired by *ReasoningBank* (arXiv:2509.25140). Auto-creates pairs from `checkpoint(what_worked=..., what_failed=...)`. Exports DPO training pairs. +- **HeuristicDistiller** (`dhee/core/heuristic.py`): Distills abstract reasoning patterns at 3 levels (specific / domain / universal) from agent trajectories. Inspired by *ERL: Efficient Reinforcement Learning* (arXiv:2603.24639). Deduplicates via Jaccard similarity. +- **MetaBuddhi** (`dhee/core/meta_buddhi.py`): Self-referential cognition loop — proposes retrieval strategy mutations, evaluates them against samskara signals, promotes or rolls back. Inspired by *DGM-Hyperagents* (arXiv:2603.19461). The improvement procedure can improve itself. +- **RetrievalStrategy** (`dhee/core/strategy.py`): Versioned scoring weights stored as human-readable JSON files. Tunable knobs: semantic/keyword weights, recency boost, contrastive boost, heuristic relevance, context budgets. +- **ProgressiveTrainer** (`dhee/mini/progressive_trainer.py`): 3-stage training pipeline (SFT → DPO → RL gate). Inspired by *AgeMem* (arXiv:2601.01885). Weights samples by vasana degradation signals. Minimum thresholds prevent training on insufficient data. +- **HyperContext** gains `contrasts` and `heuristics` fields — agents now receive contrastive evidence (do/avoid) and learned heuristics at session start. +- **Buddhi** auto-wiring: `reflect()` auto-creates contrastive pairs and distills heuristics. `get_hyper_context()` populates contrasts and heuristics. +- **HybridSearcher**: Added `contrastive_boost` parameter — results aligned with past successes score higher. +- **EvolutionLayer**: Now runs dual loops — Nididhyasana (model training) + MetaBuddhi (strategy improvement). +- **SkillMiner**: Triggers heuristic distillation after successful skill mining. + +### Added — Phase 3: Scale + +- **EvolvingGraph** (`dhee/core/graph_evolution.py`): Extends KnowledgeGraph with entity versioning (append-only JSONL), personalized PageRank per user/agent, and schema-free entity extraction via LLM (entities are typed as `DYNAMIC` when they don't match the fixed schema). +- **HiveMemory** (`dhee/hive/hive_memory.py`): Multi-agent shared cognition on top of engram-bus. Agents publish insights, heuristics, and skills to the hive. Quality gating via Wilson score lower bound. Voting and adoption tracking. +- **CRDT Sync** (`dhee/hive/sync.py`): Offline/edge sync protocol. LWW-Register for content, G-Counter for votes, OR-Set for adoption lists. `SyncEnvelope` wire format (JSON over bytes). Nodes converge after arbitrary offline periods. +- **Framework Adapters**: + - `dhee/adapters/openai_funcs.py` — `OpenAIToolAdapter` with `tool_definitions()` and `execute()` dispatch. Works with any API-compatible provider. + - `dhee/adapters/langchain.py` — `get_dhee_tools()` returns 4 LangChain `BaseTool` instances. Lazy import — no hard dependency. + - `dhee/adapters/autogen.py` — `get_autogen_functions()` for v0.2, `get_autogen_tool_specs()` for v0.4+. `register_dhee_tools()` for auto-registration. + - `dhee/adapters/system_prompt.py` — `generate_snapshot()` renders HyperContext as a frozen system prompt block. Configurable sections, minimal mode for edge. +- **EdgeTrainer** (`dhee/edge/edge_trainer.py`): On-device micro-training. LoRA rank-4, CPU-only, <2GB RAM. Deferred training mode for GGUF models. Vasana-weighted sample emphasis. +- **KnowledgeGraph**: Added `DYNAMIC` entity type, `save()`/`load()` JSON persistence. + +### Changed + +- **Version**: 1.0.0 → 2.0.0 +- **MCP server** (`dhee/mcp_slim.py`): Refactored to wrap `DheePlugin` as backing singleton. +- **pyproject.toml**: Updated description, keywords, classifier to Production/Stable. + +### Research References + +This release was informed by the following research (March 2026): + +| Paper | Key Idea Applied | +|-------|-----------------| +| *DGM-Hyperagents* (arXiv:2603.19461) | Self-referential meta-agents that modify their own improvement procedure → MetaBuddhi | +| *ERL* (arXiv:2603.24639) | Distill trajectories into abstract heuristics, not raw logs → HeuristicDistiller | +| *ReasoningBank* (arXiv:2509.25140) | Contrastive learning from success/failure pairs, MaTTS scoring → ContrastiveStore | +| *AgeMem* (arXiv:2601.01885) | Memory ops as RL-optimized tool calls, 3-stage progressive training → ProgressiveTrainer | +| *Structured Agent Distillation* (arXiv:2505.13820) | [REASON]/[ACT] segmented traces for training small models → TraceSegmenter | + +### Migration from V1 + +V2 is backwards-compatible with V1. Existing code using `Memory`, `Engram`, or `Dhee` classes continues to work unchanged. The new `DheePlugin` is additive — adopt it when you want the self-evolution capabilities. + +```python +# V1 (still works) +from dhee import Memory +m = Memory() +m.add("fact") + +# V2 (new universal plugin) +from dhee import DheePlugin +p = DheePlugin() +p.remember("fact") +ctx = p.context("what am I working on?") +prompt = p.session_start("fixing auth bug") +``` + +--- + +## [1.0.0] - 2026-03-22 + +### Changed +- Renamed project from Engram to Dhee. +- Clean repository for public push. +- All imports updated (`engram.*` → `dhee.*`). + ## [0.4.0] - 2025-02-09 ### Added diff --git a/dhee/__init__.py b/dhee/__init__.py index 200eedb..bdc65d2 100644 --- a/dhee/__init__.py +++ b/dhee/__init__.py @@ -23,7 +23,8 @@ from dhee.memory.core import CoreMemory from dhee.memory.smart import SmartMemory from dhee.memory.main import FullMemory -from dhee.simple import Engram +from dhee.simple import Engram, Dhee +from dhee.adapters.base import DheePlugin from dhee.core.category import CategoryProcessor, Category, CategoryType, CategoryMatch from dhee.core.echo import EchoProcessor, EchoDepth, EchoResult from dhee.configs.base import MemoryConfig, FadeMemConfig, EchoMemConfig, CategoryMemConfig, ScopeConfig @@ -31,7 +32,7 @@ # Default: CoreMemory (lightest, zero-config) Memory = CoreMemory -__version__ = "1.0.0" +__version__ = "2.0.0" __all__ = [ # Tiered memory classes "CoreMemory", @@ -39,7 +40,10 @@ "FullMemory", "Memory", # Simplified interface + "Dhee", "Engram", + # Universal plugin + "DheePlugin", # CategoryMem "CategoryProcessor", "Category", diff --git a/dhee/adapters/__init__.py b/dhee/adapters/__init__.py new file mode 100644 index 0000000..85a7879 --- /dev/null +++ b/dhee/adapters/__init__.py @@ -0,0 +1,13 @@ +"""Dhee adapters — universal plugin interface for any agent framework. + +Available adapters: + - DheePlugin: Base universal plugin (remember/recall/context/checkpoint) + - OpenAIToolAdapter: OpenAI function calling (tools= parameter) + - get_dhee_tools: LangChain BaseTool wrappers + - get_autogen_functions: AutoGen v0.2 callables + schemas + - generate_snapshot: Frozen system prompt for non-tool-calling agents +""" + +from dhee.adapters.base import DheePlugin + +__all__ = ["DheePlugin"] diff --git a/dhee/adapters/autogen.py b/dhee/adapters/autogen.py new file mode 100644 index 0000000..8d4b4e1 --- /dev/null +++ b/dhee/adapters/autogen.py @@ -0,0 +1,268 @@ +"""AutoGen adapter — wraps DheePlugin tools as AutoGen-callable functions. + +Supports both AutoGen v0.2 (register_for_llm/register_for_execution) and +the newer AG2 / AutoGen 0.4+ patterns. + +Usage with AutoGen v0.2: + from dhee import DheePlugin + from dhee.adapters.autogen import get_autogen_functions, register_dhee_tools + + plugin = DheePlugin() + + # Option 1: Get callables + schemas for manual registration + functions = get_autogen_functions(plugin) + + # Option 2: Auto-register on an assistant + executor pair + register_dhee_tools(plugin, assistant=assistant, executor=user_proxy) + +Usage with AG2 / AutoGen 0.4+: + from dhee.adapters.autogen import get_autogen_tool_specs + + specs = get_autogen_tool_specs(plugin) + # Pass to ConversableAgent(tools=specs) +""" + +from __future__ import annotations + +import json +import logging +from typing import Any, Callable, Dict, List, Optional, Tuple + +logger = logging.getLogger(__name__) + + +# --------------------------------------------------------------------------- +# Tool callables +# --------------------------------------------------------------------------- + +def _make_callables(plugin: Any) -> Dict[str, Callable]: + """Create plain callables wrapping DheePlugin methods.""" + + def remember(content: str, user_id: str = "default") -> str: + """Store a fact, preference, or observation to memory.""" + result = plugin.remember(content=content, user_id=user_id) + return json.dumps(result, default=str) + + def recall(query: str, user_id: str = "default", limit: int = 5) -> str: + """Search memory for relevant facts.""" + results = plugin.recall(query=query, user_id=user_id, limit=limit) + return json.dumps(results, default=str) + + def context( + task_description: str = "", user_id: str = "default", + ) -> str: + """HyperAgent session bootstrap. Returns full cognition context.""" + result = plugin.context( + task_description=task_description or None, user_id=user_id, + ) + return json.dumps(result, default=str) + + def checkpoint( + summary: str, + task_type: str = "", + outcome_score: float = -1.0, + what_worked: str = "", + what_failed: str = "", + remember_to: str = "", + ) -> str: + """Save session state and learnings.""" + kwargs: Dict[str, Any] = {"summary": summary} + if task_type: + kwargs["task_type"] = task_type + if outcome_score >= 0: + kwargs["outcome_score"] = outcome_score + if what_worked: + kwargs["what_worked"] = what_worked + if what_failed: + kwargs["what_failed"] = what_failed + if remember_to: + kwargs["remember_to"] = remember_to + result = plugin.checkpoint(**kwargs) + return json.dumps(result, default=str) + + return { + "dhee_remember": remember, + "dhee_recall": recall, + "dhee_context": context, + "dhee_checkpoint": checkpoint, + } + + +# --------------------------------------------------------------------------- +# AutoGen v0.2 schemas +# --------------------------------------------------------------------------- + +_AUTOGEN_SCHEMAS: List[Dict[str, Any]] = [ + { + "name": "dhee_remember", + "description": ( + "Store a fact, preference, or observation to memory. " + "Zero LLM calls, one embedding call." + ), + "parameters": { + "type": "object", + "properties": { + "content": { + "type": "string", + "description": "The fact to remember", + }, + "user_id": { + "type": "string", + "description": "User identifier (default: 'default')", + "default": "default", + }, + }, + "required": ["content"], + }, + }, + { + "name": "dhee_recall", + "description": ( + "Search memory for relevant facts. Returns top-K ranked by relevance." + ), + "parameters": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "What you're trying to remember", + }, + "user_id": { + "type": "string", + "description": "User identifier", + "default": "default", + }, + "limit": { + "type": "integer", + "description": "Max results (default: 5)", + "default": 5, + }, + }, + "required": ["query"], + }, + }, + { + "name": "dhee_context", + "description": ( + "HyperAgent session bootstrap. Returns performance, insights, " + "intentions, warnings, heuristics, and memories." + ), + "parameters": { + "type": "object", + "properties": { + "task_description": { + "type": "string", + "description": "What you're about to work on", + "default": "", + }, + "user_id": { + "type": "string", + "description": "User identifier", + "default": "default", + }, + }, + }, + }, + { + "name": "dhee_checkpoint", + "description": ( + "Save session state and learnings. Records outcomes, synthesizes " + "insights, stores intentions." + ), + "parameters": { + "type": "object", + "properties": { + "summary": { + "type": "string", + "description": "What you were working on", + }, + "task_type": { + "type": "string", + "description": "Task category (e.g., 'bug_fix')", + "default": "", + }, + "outcome_score": { + "type": "number", + "description": "0.0-1.0 outcome score (-1 to skip)", + "default": -1.0, + }, + "what_worked": { + "type": "string", + "description": "Approach that worked", + "default": "", + }, + "what_failed": { + "type": "string", + "description": "Approach that failed", + "default": "", + }, + "remember_to": { + "type": "string", + "description": "Future intention: 'remember to X when Y'", + "default": "", + }, + }, + "required": ["summary"], + }, + }, +] + + +# --------------------------------------------------------------------------- +# Public API +# --------------------------------------------------------------------------- + +def get_autogen_functions( + plugin: Any, +) -> List[Tuple[Callable, Dict[str, Any]]]: + """Get (callable, schema) pairs for AutoGen v0.2 registration. + + Returns: + List of (function, schema_dict) tuples ready for + register_for_llm / register_for_execution. + """ + callables = _make_callables(plugin) + return [ + (callables[schema["name"]], schema) + for schema in _AUTOGEN_SCHEMAS + ] + + +def register_dhee_tools( + plugin: Any, + assistant: Any, + executor: Any, +) -> None: + """Register Dhee tools on an AutoGen v0.2 assistant + executor pair. + + Args: + plugin: A DheePlugin instance. + assistant: An AssistantAgent (or ConversableAgent) for LLM. + executor: A UserProxyAgent (or ConversableAgent) for execution. + """ + callables = _make_callables(plugin) + + for schema in _AUTOGEN_SCHEMAS: + name = schema["name"] + fn = callables[name] + + # Register for LLM (tool definition) + assistant.register_for_llm( + name=name, + description=schema["description"], + )(fn) + + # Register for execution + executor.register_for_execution(name=name)(fn) + + +def get_autogen_tool_specs(plugin: Any) -> List[Dict[str, Any]]: + """Get tool specs for AG2 / AutoGen 0.4+ ConversableAgent(tools=...). + + Returns a list of dicts with 'function' and 'schema' keys. + """ + callables = _make_callables(plugin) + return [ + {"function": callables[schema["name"]], "schema": schema} + for schema in _AUTOGEN_SCHEMAS + ] diff --git a/dhee/adapters/base.py b/dhee/adapters/base.py new file mode 100644 index 0000000..3c16e2c --- /dev/null +++ b/dhee/adapters/base.py @@ -0,0 +1,583 @@ +"""DheePlugin — universal cognition plugin for any agent framework. + +This is THE entry point for integrating Dhee into any agent. It wraps the +full Engram + Buddhi stack behind a framework-agnostic API that mirrors +the 4 MCP tools (remember/recall/context/checkpoint) and adds: + + - session_start/session_end lifecycle (Hermes-style frozen snapshot) + - Trajectory recording for skill mining + self-evolution + - Framework export helpers (OpenAI functions, system prompt block) + +Usage: + from dhee import DheePlugin + + # Zero-config (in-memory, mock provider) + plugin = DheePlugin(in_memory=True) + + # Production (auto-detects provider from env) + plugin = DheePlugin() + + # Edge/hardware (fully offline) + plugin = DheePlugin(offline=True, data_dir="/data/dhee") + + # Framework integration + tools = plugin.as_openai_functions() # for OpenAI function calling + prompt = plugin.session_start("fixing auth bug") # frozen snapshot +""" + +from __future__ import annotations + +import json +import logging +import os +import textwrap +import time +import uuid +from pathlib import Path +from typing import Any, Callable, Dict, List, Optional, Union + +logger = logging.getLogger(__name__) + + +class DheePlugin: + """Universal cognition plugin that makes any agent a HyperAgent. + + Wraps Engram (memory) + Buddhi (cognition) behind 4 tools that work + with MCP, OpenAI functions, LangChain, AutoGen, or direct Python. + + Args: + data_dir: Storage directory. Defaults to ~/.dhee. + provider: "openai", "gemini", "ollama", or None (auto-detect). + user_id: Default user ID for all operations. + in_memory: Use in-memory storage (for testing). + offline: Force fully offline mode (no API calls). + config: Override MemoryConfig directly. + """ + + def __init__( + self, + data_dir: Optional[Union[str, Path]] = None, + provider: Optional[str] = None, + user_id: str = "default", + in_memory: bool = False, + offline: bool = False, + config=None, + ): + self._user_id = user_id + self._offline = offline + self._active_trajectories: Dict[str, Any] = {} + + # Resolve provider + if offline and provider is None: + provider = "mock" + + # Build the Engram (memory) layer + from dhee.simple import Engram + self._engram = Engram( + provider=provider, + data_dir=data_dir, + in_memory=in_memory, + ) + + # Build the Buddhi (cognition) layer + from dhee.core.buddhi import Buddhi + buddhi_dir = str(self._engram.data_dir / "buddhi") + self._buddhi = Buddhi(data_dir=buddhi_dir) + + # Session tracking + self._session_id: Optional[str] = None + self._session_start_time: Optional[float] = None + + # ------------------------------------------------------------------ + # Properties + # ------------------------------------------------------------------ + + @property + def data_dir(self) -> Path: + return self._engram.data_dir + + @property + def provider(self) -> str: + return self._engram.provider + + @property + def buddhi(self): + return self._buddhi + + # ------------------------------------------------------------------ + # Tool 1: remember + # ------------------------------------------------------------------ + + def remember( + self, + content: str, + user_id: Optional[str] = None, + metadata: Optional[Dict[str, Any]] = None, + ) -> Dict[str, Any]: + """Store a fact, preference, or observation. + + 0 LLM calls on hot path. 1 embedding call. Intention auto-detection + checks for "remember to X when Y" patterns. + """ + uid = user_id or self._user_id + result = self._engram.add(content, user_id=uid, infer=False, metadata=metadata) + + response: Dict[str, Any] = {"stored": True} + if isinstance(result, dict): + rs = result.get("results", []) + if rs: + response["id"] = rs[0].get("id") + + # Buddhi: detect intentions in the content + intention = self._buddhi.on_memory_stored(content=content, user_id=uid) + if intention: + response["detected_intention"] = intention.to_dict() + + return response + + # ------------------------------------------------------------------ + # Tool 2: recall + # ------------------------------------------------------------------ + + def recall( + self, + query: str, + user_id: Optional[str] = None, + limit: int = 5, + ) -> List[Dict[str, Any]]: + """Search memory for relevant facts. 0 LLM calls. 1 embedding.""" + uid = user_id or self._user_id + results = self._engram.search(query, user_id=uid, limit=limit) + return [ + { + "memory": r.get("memory", r.get("content", "")), + "score": round(r.get("composite_score", r.get("score", 0.0)), 3), + "id": r.get("id", ""), + } + for r in results + ] + + # ------------------------------------------------------------------ + # Tool 3: context + # ------------------------------------------------------------------ + + def context( + self, + task_description: Optional[str] = None, + user_id: Optional[str] = None, + ) -> Dict[str, Any]: + """HyperAgent session bootstrap. Returns everything the agent needs.""" + uid = user_id or self._user_id + hyper_ctx = self._buddhi.get_hyper_context( + user_id=uid, + task_description=task_description, + memory=self._engram._memory, + ) + return hyper_ctx.to_dict() + + # ------------------------------------------------------------------ + # Tool 4: checkpoint + # ------------------------------------------------------------------ + + def checkpoint( + self, + summary: str, + task_type: Optional[str] = None, + outcome_score: Optional[float] = None, + what_worked: Optional[str] = None, + what_failed: Optional[str] = None, + key_decision: Optional[str] = None, + remember_to: Optional[str] = None, + trigger_keywords: Optional[List[str]] = None, + status: str = "paused", + decisions: Optional[List[str]] = None, + todos: Optional[List[str]] = None, + files_touched: Optional[List[str]] = None, + repo: Optional[str] = None, + user_id: Optional[str] = None, + agent_id: str = "dhee", + ) -> Dict[str, Any]: + """Save session state. Where the cognition happens. + + 1. Session digest → cross-agent handoff + 2. Batch enrichment → 1 LLM call per ~10 memories + 3. Outcome recording → performance tracking + 4. Insight synthesis → transferable learnings + 5. Intention storage → prospective memory + """ + uid = user_id or self._user_id + result: Dict[str, Any] = {} + + # 1. Session digest + try: + from dhee.core.kernel import save_session_digest + digest = save_session_digest( + task_summary=summary, agent_id=agent_id, repo=repo, + status=status, decisions_made=decisions, + files_touched=files_touched, todos_remaining=todos, + ) + result["session_saved"] = True + if isinstance(digest, dict): + result["session_id"] = digest.get("session_id") + except Exception: + result["session_saved"] = False + + # 2. Batch enrichment + memory = self._engram._memory + if hasattr(memory, "enrich_pending"): + try: + enrich_result = memory.enrich_pending( + user_id=uid, batch_size=10, max_batches=5, + ) + enriched = enrich_result.get("enriched_count", 0) + if enriched > 0: + result["memories_enriched"] = enriched + except Exception: + pass + + # 3. Outcome recording + if task_type and outcome_score is not None: + score = max(0.0, min(1.0, float(outcome_score))) + insight = self._buddhi.record_outcome( + user_id=uid, task_type=task_type, score=score, + ) + result["outcome_recorded"] = True + if insight: + result["auto_insight"] = insight.to_dict() + + # 4. Insight synthesis + if any([what_worked, what_failed, key_decision]): + insights = self._buddhi.reflect( + user_id=uid, task_type=task_type or "general", + what_worked=what_worked, what_failed=what_failed, + key_decision=key_decision, + ) + result["insights_created"] = len(insights) + + # 5. Intention storage + if remember_to: + intention = self._buddhi.store_intention( + user_id=uid, description=remember_to, + trigger_keywords=trigger_keywords, + ) + result["intention_stored"] = intention.to_dict() + + return result + + # ------------------------------------------------------------------ + # Session lifecycle (Hermes-style frozen snapshot) + # ------------------------------------------------------------------ + + def session_start( + self, + task_description: Optional[str] = None, + user_id: Optional[str] = None, + ) -> str: + """Start a session and return a frozen system prompt block. + + The system prompt contains the full HyperContext rendered as text. + Inject it into your agent's system prompt at session start. + The snapshot is frozen — writes during the session update storage + but don't change this prompt, preserving LLM prefix caches. + """ + uid = user_id or self._user_id + self._session_id = str(uuid.uuid4()) + self._session_start_time = time.time() + + ctx = self.context(task_description=task_description, user_id=uid) + return self._render_system_prompt(ctx, task_description) + + def session_end( + self, + summary: str, + outcome_score: Optional[float] = None, + task_type: Optional[str] = None, + what_worked: Optional[str] = None, + what_failed: Optional[str] = None, + **kwargs, + ) -> Dict[str, Any]: + """End a session. Shorthand for checkpoint with session metadata.""" + result = self.checkpoint( + summary=summary, outcome_score=outcome_score, + task_type=task_type, what_worked=what_worked, + what_failed=what_failed, status="completed", **kwargs, + ) + self._session_id = None + self._session_start_time = None + return result + + # ------------------------------------------------------------------ + # Trajectory recording (for skill mining + self-evolution) + # ------------------------------------------------------------------ + + def begin_trajectory( + self, + task_description: str, + user_id: Optional[str] = None, + agent_id: str = "default", + ): + """Start recording a trajectory for this task. + + Returns a TrajectoryRecorder — call .record_step() on each action, + then pass it to end_trajectory() when done. + """ + from dhee.skills.trajectory import TrajectoryRecorder + uid = user_id or self._user_id + recorder = TrajectoryRecorder( + task_description=task_description, + user_id=uid, + agent_id=agent_id, + ) + self._active_trajectories[recorder.id] = recorder + return recorder + + def end_trajectory( + self, + recorder, + success: bool, + outcome_summary: str = "", + ) -> Dict[str, Any]: + """Finalize a trajectory and feed it into the learning pipeline.""" + trajectory = recorder.finalize(success=success, outcome_summary=outcome_summary) + self._active_trajectories.pop(recorder.id, None) + + result: Dict[str, Any] = { + "trajectory_id": trajectory.id, + "steps": len(trajectory.steps), + "success": success, + } + + # Store trajectory as memory for skill mining + try: + from dhee.skills.trajectory import TrajectoryStore + store = TrajectoryStore(memory=self._engram._memory) + store.save(trajectory) + result["stored"] = True + except Exception: + result["stored"] = False + + return result + + # ------------------------------------------------------------------ + # Framework export: OpenAI function calling + # ------------------------------------------------------------------ + + def as_openai_functions(self) -> List[Dict[str, Any]]: + """Return the 4 tools as OpenAI function calling schemas. + + Use with: client.chat.completions.create(tools=plugin.as_openai_functions()) + """ + return [ + { + "type": "function", + "function": { + "name": "remember", + "description": ( + "Store a fact, preference, or observation to memory. " + "0 LLM calls, 1 embedding. Fast." + ), + "parameters": { + "type": "object", + "properties": { + "content": { + "type": "string", + "description": "The fact to remember", + }, + "user_id": { + "type": "string", + "description": "User identifier (default: 'default')", + }, + }, + "required": ["content"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "recall", + "description": ( + "Search memory for relevant facts. Returns top-K ranked by relevance. " + "0 LLM calls, 1 embedding." + ), + "parameters": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "What you're trying to remember", + }, + "user_id": { + "type": "string", + "description": "User identifier", + }, + "limit": { + "type": "integer", + "description": "Max results (default: 5)", + }, + }, + "required": ["query"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "context", + "description": ( + "HyperAgent session bootstrap. Returns performance, insights, " + "intentions, warnings, and memories. Call once at session start." + ), + "parameters": { + "type": "object", + "properties": { + "task_description": { + "type": "string", + "description": "What you're about to work on", + }, + "user_id": { + "type": "string", + "description": "User identifier", + }, + }, + }, + }, + }, + { + "type": "function", + "function": { + "name": "checkpoint", + "description": ( + "Save session state and learnings. Records outcomes, " + "synthesizes insights, stores intentions." + ), + "parameters": { + "type": "object", + "properties": { + "summary": { + "type": "string", + "description": "What you were working on", + }, + "task_type": { + "type": "string", + "description": "Task category (e.g., 'bug_fix')", + }, + "outcome_score": { + "type": "number", + "description": "0.0-1.0 outcome score", + }, + "what_worked": { + "type": "string", + "description": "Approach that worked", + }, + "what_failed": { + "type": "string", + "description": "Approach that failed", + }, + "remember_to": { + "type": "string", + "description": "Future intention: 'remember to X when Y'", + }, + "trigger_keywords": { + "type": "array", + "items": {"type": "string"}, + "description": "Keywords that trigger the intention", + }, + }, + "required": ["summary"], + }, + }, + }, + ] + + # ------------------------------------------------------------------ + # Framework export: system prompt + # ------------------------------------------------------------------ + + def as_system_prompt( + self, + task_description: Optional[str] = None, + user_id: Optional[str] = None, + ) -> str: + """Generate a frozen system prompt block from current HyperContext. + + For agents that don't support tool calling — inject this into + the system prompt so the LLM has full context. + """ + uid = user_id or self._user_id + ctx = self.context(task_description=task_description, user_id=uid) + return self._render_system_prompt(ctx, task_description) + + # ------------------------------------------------------------------ + # Internal: render HyperContext as text + # ------------------------------------------------------------------ + + def _render_system_prompt( + self, ctx: Dict[str, Any], task: Optional[str] = None, + ) -> str: + """Render HyperContext dict as a human-readable system prompt block.""" + parts = ["## Dhee Cognition Context"] + + if task: + parts.append(f"\n**Current task:** {task}") + + # Performance + perf = ctx.get("performance", []) + if perf: + parts.append("\n### Performance History") + for p in perf: + direction = "improving" if p.get("trend", 0) > 0 else "declining" + parts.append( + f"- **{p['task_type']}**: avg={p['avg_score']:.2f}, " + f"trend={p['trend']:+.3f} ({direction}), " + f"attempts={p['total_attempts']}" + ) + + # Warnings + warnings = ctx.get("warnings", []) + if warnings: + parts.append("\n### Warnings") + for w in warnings: + parts.append(f"- {w}") + + # Insights + insights = ctx.get("insights", []) + if insights: + parts.append("\n### Insights from Past Work") + for i in insights[:5]: + parts.append(f"- [{i['type']}] {i['content']}") + + # Intentions + intentions = ctx.get("intentions", []) + if intentions: + parts.append("\n### Triggered Reminders") + for i in intentions: + parts.append(f"- {i['description']}") + + # Contrasts (Phase 2) + contrasts = ctx.get("contrasts", []) + if contrasts: + parts.append("\n### Contrastive Evidence (Do / Avoid)") + for c in contrasts[:3]: + parts.append(f"- **Do:** {c.get('do', '')[:150]}") + parts.append(f" **Avoid:** {c.get('avoid', '')[:150]}") + + # Heuristics (Phase 2) + heuristics = ctx.get("heuristics", []) + if heuristics: + parts.append("\n### Learned Heuristics") + for h in heuristics[:3]: + parts.append( + f"- [{h.get('level', 'domain')}] {h.get('heuristic', '')[:200]}" + ) + + # Memories + memories = ctx.get("memories", []) + if memories: + parts.append("\n### Relevant Memories") + for m in memories[:5]: + mem_text = m.get("memory", "")[:200] + if mem_text: + parts.append(f"- {mem_text}") + + return "\n".join(parts) diff --git a/dhee/adapters/langchain.py b/dhee/adapters/langchain.py new file mode 100644 index 0000000..a13a4a6 --- /dev/null +++ b/dhee/adapters/langchain.py @@ -0,0 +1,236 @@ +"""LangChain adapter — wraps DheePlugin tools as LangChain BaseTool instances. + +Usage: + from dhee import DheePlugin + from dhee.adapters.langchain import get_dhee_tools + + plugin = DheePlugin() + tools = get_dhee_tools(plugin) + + # Use with any LangChain agent: + agent = create_react_agent(llm, tools) + + # Or pick individual tools: + remember_tool, recall_tool, context_tool, checkpoint_tool = tools +""" + +from __future__ import annotations + +import json +import logging +from typing import Any, Dict, List, Optional, Type + +logger = logging.getLogger(__name__) + +# Lazy import — LangChain is optional +_HAS_LANGCHAIN = None + + +def _check_langchain() -> bool: + global _HAS_LANGCHAIN + if _HAS_LANGCHAIN is None: + try: + from langchain_core.tools import BaseTool # noqa: F401 + _HAS_LANGCHAIN = True + except ImportError: + _HAS_LANGCHAIN = False + return _HAS_LANGCHAIN + + +def _get_base_classes(): + """Import LangChain base classes (raises ImportError if not installed).""" + from langchain_core.tools import BaseTool + from langchain_core.callbacks import CallbackManagerForToolRun + try: + from pydantic import BaseModel, Field + except ImportError: + from langchain_core.pydantic_v1 import BaseModel, Field + return BaseTool, CallbackManagerForToolRun, BaseModel, Field + + +# --------------------------------------------------------------------------- +# Tool implementations +# --------------------------------------------------------------------------- + +def _make_remember_tool(plugin: Any): + BaseTool, CallbackManagerForToolRun, BaseModel, Field = _get_base_classes() + + class RememberInput(BaseModel): + content: str = Field(description="The fact, preference, or observation to remember") + user_id: Optional[str] = Field(default=None, description="User identifier") + + class DheeRemember(BaseTool): + name: str = "dhee_remember" + description: str = ( + "Store a fact, preference, or observation to memory. " + "Zero LLM calls, one embedding call. Fast." + ) + args_schema: Type[BaseModel] = RememberInput + _plugin: Any = None + + def __init__(self, plugin: Any, **kwargs): + super().__init__(**kwargs) + self._plugin = plugin + + def _run( + self, + content: str, + user_id: Optional[str] = None, + run_manager: Optional[CallbackManagerForToolRun] = None, + ) -> str: + result = self._plugin.remember(content=content, user_id=user_id) + return json.dumps(result, default=str) + + return DheeRemember(plugin=plugin) + + +def _make_recall_tool(plugin: Any): + BaseTool, CallbackManagerForToolRun, BaseModel, Field = _get_base_classes() + + class RecallInput(BaseModel): + query: str = Field(description="What you're trying to remember") + user_id: Optional[str] = Field(default=None, description="User identifier") + limit: int = Field(default=5, description="Maximum results to return") + + class DheeRecall(BaseTool): + name: str = "dhee_recall" + description: str = ( + "Search memory for relevant facts. Returns top-K results " + "ranked by relevance. Zero LLM calls, one embedding." + ) + args_schema: Type[BaseModel] = RecallInput + _plugin: Any = None + + def __init__(self, plugin: Any, **kwargs): + super().__init__(**kwargs) + self._plugin = plugin + + def _run( + self, + query: str, + user_id: Optional[str] = None, + limit: int = 5, + run_manager: Optional[CallbackManagerForToolRun] = None, + ) -> str: + results = self._plugin.recall(query=query, user_id=user_id, limit=limit) + return json.dumps(results, default=str) + + return DheeRecall(plugin=plugin) + + +def _make_context_tool(plugin: Any): + BaseTool, CallbackManagerForToolRun, BaseModel, Field = _get_base_classes() + + class ContextInput(BaseModel): + task_description: Optional[str] = Field( + default=None, description="What you're about to work on", + ) + user_id: Optional[str] = Field(default=None, description="User identifier") + + class DheeContext(BaseTool): + name: str = "dhee_context" + description: str = ( + "HyperAgent session bootstrap. Returns performance snapshots, " + "insights, intentions, warnings, heuristics, and relevant memories. " + "Call once at the start of a task." + ) + args_schema: Type[BaseModel] = ContextInput + _plugin: Any = None + + def __init__(self, plugin: Any, **kwargs): + super().__init__(**kwargs) + self._plugin = plugin + + def _run( + self, + task_description: Optional[str] = None, + user_id: Optional[str] = None, + run_manager: Optional[CallbackManagerForToolRun] = None, + ) -> str: + result = self._plugin.context( + task_description=task_description, user_id=user_id, + ) + return json.dumps(result, default=str) + + return DheeContext(plugin=plugin) + + +def _make_checkpoint_tool(plugin: Any): + BaseTool, CallbackManagerForToolRun, BaseModel, Field = _get_base_classes() + + class CheckpointInput(BaseModel): + summary: str = Field(description="What you were working on") + task_type: Optional[str] = Field( + default=None, description="Task category (e.g., 'bug_fix')", + ) + outcome_score: Optional[float] = Field( + default=None, description="0.0-1.0 outcome score", + ) + what_worked: Optional[str] = Field( + default=None, description="Approach that worked", + ) + what_failed: Optional[str] = Field( + default=None, description="Approach that failed", + ) + remember_to: Optional[str] = Field( + default=None, description="Future intention: 'remember to X when Y'", + ) + + class DheeCheckpoint(BaseTool): + name: str = "dhee_checkpoint" + description: str = ( + "Save session state and learnings. Records outcomes, " + "synthesizes insights from what worked/failed, stores intentions." + ) + args_schema: Type[BaseModel] = CheckpointInput + _plugin: Any = None + + def __init__(self, plugin: Any, **kwargs): + super().__init__(**kwargs) + self._plugin = plugin + + def _run( + self, + summary: str, + task_type: Optional[str] = None, + outcome_score: Optional[float] = None, + what_worked: Optional[str] = None, + what_failed: Optional[str] = None, + remember_to: Optional[str] = None, + run_manager: Optional[CallbackManagerForToolRun] = None, + ) -> str: + result = self._plugin.checkpoint( + summary=summary, task_type=task_type, + outcome_score=outcome_score, what_worked=what_worked, + what_failed=what_failed, remember_to=remember_to, + ) + return json.dumps(result, default=str) + + return DheeCheckpoint(plugin=plugin) + + +# --------------------------------------------------------------------------- +# Public API +# --------------------------------------------------------------------------- + +def get_dhee_tools(plugin: Any) -> List[Any]: + """Create LangChain tool instances from a DheePlugin. + + Returns: + [DheeRemember, DheeRecall, DheeContext, DheeCheckpoint] + + Raises: + ImportError: If langchain-core is not installed. + """ + if not _check_langchain(): + raise ImportError( + "langchain-core is required for LangChain integration. " + "Install it with: pip install langchain-core" + ) + + return [ + _make_remember_tool(plugin), + _make_recall_tool(plugin), + _make_context_tool(plugin), + _make_checkpoint_tool(plugin), + ] diff --git a/dhee/adapters/openai_funcs.py b/dhee/adapters/openai_funcs.py new file mode 100644 index 0000000..0702dfa --- /dev/null +++ b/dhee/adapters/openai_funcs.py @@ -0,0 +1,183 @@ +"""OpenAI function calling adapter for DheePlugin. + +Generates tool definitions compatible with: + - OpenAI Chat Completions API (tools parameter) + - Any OpenAI-compatible API (Ollama, vLLM, LiteLLM, etc.) + +Usage with the OpenAI SDK: + from dhee.adapters.openai_funcs import OpenAIToolAdapter + + adapter = OpenAIToolAdapter(plugin) + response = client.chat.completions.create( + model="gpt-4", + messages=messages, + tools=adapter.tool_definitions(), + ) + + # Execute the function call + for call in response.choices[0].message.tool_calls: + result = adapter.execute(call.function.name, json.loads(call.function.arguments)) +""" + +from __future__ import annotations + +import json +import logging +from typing import Any, Callable, Dict, List, Optional + +logger = logging.getLogger(__name__) + + +class OpenAIToolAdapter: + """Wraps DheePlugin as OpenAI-compatible function calling tools. + + Provides tool_definitions() for the API request and execute() for + dispatching tool calls from the response. + """ + + def __init__(self, plugin: Any): + """ + Args: + plugin: A DheePlugin instance. + """ + self._plugin = plugin + self._dispatchers: Dict[str, Callable] = { + "remember": self._exec_remember, + "recall": self._exec_recall, + "context": self._exec_context, + "checkpoint": self._exec_checkpoint, + "session_start": self._exec_session_start, + "session_end": self._exec_session_end, + } + + def tool_definitions(self, include_session: bool = False) -> List[Dict[str, Any]]: + """Return OpenAI-format tool definitions. + + Args: + include_session: If True, also includes session_start and + session_end as callable tools. + """ + tools = self._plugin.as_openai_functions() + + if include_session: + tools.extend([ + { + "type": "function", + "function": { + "name": "session_start", + "description": ( + "Start a Dhee cognition session. Returns a frozen context " + "block. Call once at the beginning of a task." + ), + "parameters": { + "type": "object", + "properties": { + "task_description": { + "type": "string", + "description": "What you're about to work on", + }, + }, + }, + }, + }, + { + "type": "function", + "function": { + "name": "session_end", + "description": ( + "End the current Dhee session and save learnings." + ), + "parameters": { + "type": "object", + "properties": { + "summary": { + "type": "string", + "description": "What you accomplished", + }, + "outcome_score": { + "type": "number", + "description": "0.0-1.0 outcome score", + }, + "what_worked": { + "type": "string", + "description": "Approach that worked", + }, + "what_failed": { + "type": "string", + "description": "Approach that failed", + }, + }, + "required": ["summary"], + }, + }, + }, + ]) + + return tools + + def execute(self, function_name: str, arguments: Dict[str, Any]) -> str: + """Execute a tool call and return the JSON-encoded result. + + This is the glue between OpenAI's tool_call response and DheePlugin. + + Args: + function_name: The function name from the tool call. + arguments: Parsed JSON arguments from the tool call. + + Returns: + JSON string suitable for a tool message. + """ + dispatcher = self._dispatchers.get(function_name) + if not dispatcher: + return json.dumps({"error": f"Unknown function: {function_name}"}) + + try: + result = dispatcher(arguments) + return json.dumps(result, default=str, ensure_ascii=False) + except Exception as e: + logger.warning("Tool execution failed for %s: %s", function_name, e) + return json.dumps({"error": str(e)}) + + def _exec_remember(self, args: Dict[str, Any]) -> Any: + return self._plugin.remember( + content=args["content"], + user_id=args.get("user_id"), + ) + + def _exec_recall(self, args: Dict[str, Any]) -> Any: + return self._plugin.recall( + query=args["query"], + user_id=args.get("user_id"), + limit=args.get("limit", 5), + ) + + def _exec_context(self, args: Dict[str, Any]) -> Any: + return self._plugin.context( + task_description=args.get("task_description"), + user_id=args.get("user_id"), + ) + + def _exec_checkpoint(self, args: Dict[str, Any]) -> Any: + return self._plugin.checkpoint( + summary=args["summary"], + task_type=args.get("task_type"), + outcome_score=args.get("outcome_score"), + what_worked=args.get("what_worked"), + what_failed=args.get("what_failed"), + remember_to=args.get("remember_to"), + trigger_keywords=args.get("trigger_keywords"), + ) + + def _exec_session_start(self, args: Dict[str, Any]) -> Any: + prompt = self._plugin.session_start( + task_description=args.get("task_description"), + ) + return {"system_prompt": prompt} + + def _exec_session_end(self, args: Dict[str, Any]) -> Any: + return self._plugin.session_end( + summary=args["summary"], + outcome_score=args.get("outcome_score"), + what_worked=args.get("what_worked"), + what_failed=args.get("what_failed"), + ) diff --git a/dhee/adapters/system_prompt.py b/dhee/adapters/system_prompt.py new file mode 100644 index 0000000..60b39f9 --- /dev/null +++ b/dhee/adapters/system_prompt.py @@ -0,0 +1,245 @@ +"""Frozen snapshot generator — renders DheePlugin context as a system prompt. + +For agents that don't support tool calling (e.g., simple prompt → completion +workflows, humanoid robot controllers, voice assistants), this module generates +a self-contained system prompt block that includes the full HyperContext. + +The "frozen snapshot" pattern (from NousResearch Hermes Agent architecture): + 1. At session start, load HyperContext into the system prompt. + 2. During the session, the system prompt is NEVER mutated — this preserves + LLM KV-cache / prefix caches for fast inference. + 3. At session end, new knowledge is written to storage for next time. + +Usage: + from dhee import DheePlugin + from dhee.adapters.system_prompt import generate_snapshot, SnapshotConfig + + plugin = DheePlugin() + prompt = generate_snapshot(plugin, task="fixing auth bug") + + # Or with custom config: + config = SnapshotConfig( + include_memories=True, + include_heuristics=True, + max_memories=10, + include_tool_instructions=True, + ) + prompt = generate_snapshot(plugin, task="fixing auth bug", config=config) +""" + +from __future__ import annotations + +import logging +import textwrap +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional + +logger = logging.getLogger(__name__) + + +@dataclass +class SnapshotConfig: + """Controls what goes into the frozen snapshot.""" + + include_performance: bool = True + include_warnings: bool = True + include_insights: bool = True + include_intentions: bool = True + include_contrasts: bool = True + include_heuristics: bool = True + include_memories: bool = True + include_tool_instructions: bool = False + include_hive: bool = False + + max_performance: int = 5 + max_warnings: int = 5 + max_insights: int = 5 + max_intentions: int = 5 + max_contrasts: int = 3 + max_heuristics: int = 3 + max_memories: int = 10 + + # Prefix/suffix for wrapping the snapshot + header: str = "## Dhee Cognition Context (Frozen Snapshot)" + footer: str = "" + + +# Tool usage instructions (for agents that CAN call tools after loading snapshot) +_TOOL_INSTRUCTIONS = """\ +### Available Memory Tools +- **remember(content)** — store a new fact/observation +- **recall(query)** — search memory for relevant facts +- **context(task)** — load full HyperContext (already loaded above) +- **checkpoint(summary, ...)** — save session state and learnings + +Use `remember` proactively when you learn new facts. Use `recall` before +answering questions that may depend on stored knowledge. Use `checkpoint` +at natural breakpoints and at session end. +""" + + +def generate_snapshot( + plugin: Any, + task: Optional[str] = None, + user_id: Optional[str] = None, + config: Optional[SnapshotConfig] = None, + hive: Optional[Any] = None, +) -> str: + """Generate a frozen system prompt snapshot from DheePlugin. + + Args: + plugin: A DheePlugin instance. + task: Current task description. + user_id: User identifier. + config: Snapshot configuration. Defaults to include everything. + hive: Optional HiveMemory instance for multi-agent context. + + Returns: + A complete system prompt block as a string. + """ + cfg = config or SnapshotConfig() + ctx = plugin.context(task_description=task, user_id=user_id) + + parts: List[str] = [cfg.header] + + if task: + parts.append(f"\n**Current task:** {task}") + + # Performance + if cfg.include_performance: + perf = ctx.get("performance", [])[:cfg.max_performance] + if perf: + parts.append("\n### Performance History") + for p in perf: + trend = p.get("trend", 0) + direction = "improving" if trend > 0 else "declining" if trend < 0 else "stable" + parts.append( + f"- **{p['task_type']}**: avg={p['avg_score']:.2f}, " + f"trend={p['trend']:+.3f} ({direction}), " + f"attempts={p['total_attempts']}" + ) + + # Warnings + if cfg.include_warnings: + warnings = ctx.get("warnings", [])[:cfg.max_warnings] + if warnings: + parts.append("\n### Warnings") + for w in warnings: + parts.append(f"- {w}") + + # Insights + if cfg.include_insights: + insights = ctx.get("insights", [])[:cfg.max_insights] + if insights: + parts.append("\n### Insights from Past Work") + for i in insights: + parts.append(f"- [{i.get('type', 'general')}] {i['content']}") + + # Intentions (triggered reminders) + if cfg.include_intentions: + intentions = ctx.get("intentions", [])[:cfg.max_intentions] + if intentions: + parts.append("\n### Triggered Reminders") + for i in intentions: + parts.append(f"- {i['description']}") + + # Contrastive evidence + if cfg.include_contrasts: + contrasts = ctx.get("contrasts", [])[:cfg.max_contrasts] + if contrasts: + parts.append("\n### Contrastive Evidence (Do / Avoid)") + for c in contrasts: + do_text = c.get("do", "")[:200] + avoid_text = c.get("avoid", "")[:200] + parts.append(f"- **Do:** {do_text}") + parts.append(f" **Avoid:** {avoid_text}") + confidence = c.get("confidence") + if confidence is not None: + parts.append(f" *confidence: {confidence:.1%}*") + + # Heuristics + if cfg.include_heuristics: + heuristics = ctx.get("heuristics", [])[:cfg.max_heuristics] + if heuristics: + parts.append("\n### Learned Heuristics") + for h in heuristics: + level = h.get("level", "domain") + text = h.get("heuristic", "")[:250] + parts.append(f"- [{level}] {text}") + + # Memories + if cfg.include_memories: + memories = ctx.get("memories", [])[:cfg.max_memories] + if memories: + parts.append("\n### Relevant Memories") + for m in memories: + mem_text = m.get("memory", "")[:250] + score = m.get("score", 0) + if mem_text: + parts.append(f"- {mem_text}") + if score > 0: + parts.append(f" *(relevance: {score:.2f})*") + + # Hive context + if cfg.include_hive and hive: + try: + hive_block = hive.get_context_block(limit=3) + hive_insights = hive_block.get("hive_insights", []) + hive_heuristics = hive_block.get("hive_heuristics", []) + + if hive_insights or hive_heuristics: + parts.append("\n### Hive Knowledge (from other agents)") + for hi in hive_insights: + parts.append( + f"- [insight from {hi['source']}] " + f"{hi['content'].get('content', '')[:150]}" + ) + for hh in hive_heuristics: + parts.append( + f"- [heuristic from {hh['source']}] " + f"{hh['content'].get('heuristic', '')[:150]}" + ) + except Exception as e: + logger.debug("Hive context failed: %s", e) + + # Tool instructions + if cfg.include_tool_instructions: + parts.append("\n" + _TOOL_INSTRUCTIONS.strip()) + + # Meta + meta = ctx.get("meta", {}) + if meta: + meta_parts = [] + for key in ["insight_count", "intention_count", "contrast_count", "heuristic_count"]: + val = meta.get(key, 0) + if val > 0: + meta_parts.append(f"{key.replace('_count', '')}s: {val}") + if meta_parts: + parts.append(f"\n*Loaded: {', '.join(meta_parts)}*") + + if cfg.footer: + parts.append(f"\n{cfg.footer}") + + return "\n".join(parts) + + +def generate_minimal_snapshot( + plugin: Any, + task: Optional[str] = None, + user_id: Optional[str] = None, +) -> str: + """Generate a minimal snapshot — just warnings, intentions, and top memories. + + Suitable for edge/embedded agents with tight context budgets. + """ + cfg = SnapshotConfig( + include_performance=False, + include_insights=False, + include_contrasts=False, + include_heuristics=False, + max_warnings=3, + max_intentions=3, + max_memories=3, + header="## Dhee Context (Minimal)", + ) + return generate_snapshot(plugin, task=task, user_id=user_id, config=cfg) diff --git a/dhee/core/buddhi.py b/dhee/core/buddhi.py index 649653d..5f2b755 100644 --- a/dhee/core/buddhi.py +++ b/dhee/core/buddhi.py @@ -170,6 +170,10 @@ class HyperContext: # Top relevant memories (context) memories: List[Dict[str, Any]] + # Phase 2: contrastive pairs + heuristics + contrasts: List[Dict[str, Any]] = field(default_factory=list) + heuristics: List[Dict[str, Any]] = field(default_factory=list) + def to_dict(self) -> Dict[str, Any]: return { "user_id": self.user_id, @@ -180,6 +184,8 @@ def to_dict(self) -> Dict[str, Any]: "skills": self.skills[:5], "intentions": [i.to_dict() for i in self.intentions], "warnings": self.warnings, + "contrasts": self.contrasts[:5], + "heuristics": self.heuristics[:5], "memories": [ {"id": m.get("id"), "memory": m.get("memory", "")[:500], "strength": m.get("strength", 1.0)} @@ -189,6 +195,8 @@ def to_dict(self) -> Dict[str, Any]: "n_insights": len(self.insights), "n_active_intentions": len(self.intentions), "n_warnings": len(self.warnings), + "n_contrasts": len(self.contrasts), + "n_heuristics": len(self.heuristics), "performance_tracked": len(self.performance) > 0, }, } @@ -245,8 +253,37 @@ def __init__(self, data_dir: Optional[str] = None): self._performance: Dict[str, List[Dict[str, Any]]] = {} # task_type -> records self._query_sequences: Dict[str, List[str]] = {} # user_id -> recent queries + # Phase 2 subsystems (lazy-initialized) + self._contrastive = None + self._heuristic_distiller = None + self._meta_buddhi = None + self._load_state() + def _get_contrastive(self): + if self._contrastive is None: + from dhee.core.contrastive import ContrastiveStore + self._contrastive = ContrastiveStore( + data_dir=os.path.join(self._data_dir, "contrastive") + ) + return self._contrastive + + def _get_heuristic_distiller(self): + if self._heuristic_distiller is None: + from dhee.core.heuristic import HeuristicDistiller + self._heuristic_distiller = HeuristicDistiller( + data_dir=os.path.join(self._data_dir, "heuristics") + ) + return self._heuristic_distiller + + def _get_meta_buddhi(self): + if self._meta_buddhi is None: + from dhee.core.meta_buddhi import MetaBuddhi + self._meta_buddhi = MetaBuddhi( + data_dir=os.path.join(self._data_dir, "meta_buddhi") + ) + return self._meta_buddhi + # ------------------------------------------------------------------ # Core API: The HyperAgent entry point # ------------------------------------------------------------------ @@ -323,6 +360,28 @@ def get_hyper_context( if len(seq) > 50: self._query_sequences[user_id] = seq[-50:] + # 9. Contrastive pairs (Phase 2: ReasoningBank pattern) + contrasts = [] + try: + store = self._get_contrastive() + pairs = store.retrieve_contrasts( + task_description or "", user_id=user_id, limit=5, + ) + contrasts = [p.to_compact() for p in pairs] + except Exception: + pass + + # 10. Heuristics (Phase 2: ERL pattern) + heuristics = [] + try: + distiller = self._get_heuristic_distiller() + relevant = distiller.retrieve_relevant( + task_description or "", user_id=user_id, limit=5, + ) + heuristics = [h.to_compact() for h in relevant] + except Exception: + pass + return HyperContext( user_id=user_id, session_id=str(uuid.uuid4()), @@ -333,6 +392,8 @@ def get_hyper_context( intentions=triggered, warnings=warnings, memories=memories, + contrasts=contrasts, + heuristics=heuristics, ) # ------------------------------------------------------------------ @@ -790,6 +851,34 @@ def reflect( ) new_insights.append(insight) + # Phase 2: Auto-create contrastive pair when both sides provided + if what_worked and what_failed: + try: + store = self._get_contrastive() + store.add_pair( + task_description=f"{task_type} task", + success_approach=what_worked, + failure_approach=what_failed, + task_type=task_type, + user_id=user_id, + ) + except Exception: + pass + + # Phase 2: Distill heuristic from what_worked + if what_worked: + try: + distiller = self._get_heuristic_distiller() + distiller.distill_from_trajectory( + task_description=f"{task_type} task", + task_type=task_type, + what_worked=what_worked, + what_failed=what_failed, + user_id=user_id, + ) + except Exception: + pass + return new_insights # ------------------------------------------------------------------ diff --git a/dhee/core/contrastive.py b/dhee/core/contrastive.py new file mode 100644 index 0000000..ec443dd --- /dev/null +++ b/dhee/core/contrastive.py @@ -0,0 +1,274 @@ +"""Contrastive Memory — learn from success/failure pairs. + +Based on ReasoningBank (arXiv:2509.25140): Memory-Aware Test-Time Scaling. +The key insight: storing BOTH what worked AND what failed for similar tasks +produces dramatically better future decisions than storing successes alone. + +Every time checkpoint() receives both what_worked and what_failed, +a ContrastivePair is created. These pairs are: + 1. Surfaced in HyperContext as "contrasts" — the agent sees what to do AND avoid + 2. Used for DPO training data in BuddhiMini's progressive trainer + 3. Used to re-rank retrieval results (contrastive boost) + +The MaTTS scoring algorithm re-ranks retrieval candidates by checking +whether they align with success approaches or failure approaches for +similar past tasks. +""" + +from __future__ import annotations + +import json +import logging +import os +import time +import uuid +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional + +logger = logging.getLogger(__name__) + + +@dataclass +class ContrastivePair: + """A success/failure pair from a completed task.""" + + id: str + task_description: str + task_type: str + success_approach: str + failure_approach: str + outcome_delta: float # how much better success was (0-1) + created_at: float + user_id: str = "default" + tags: List[str] = field(default_factory=list) + validation_count: int = 0 # times this contrast proved useful + + def to_dict(self) -> Dict[str, Any]: + return { + "id": self.id, + "task_description": self.task_description, + "task_type": self.task_type, + "success_approach": self.success_approach, + "failure_approach": self.failure_approach, + "outcome_delta": self.outcome_delta, + "created_at": self.created_at, + "user_id": self.user_id, + "tags": self.tags, + "validation_count": self.validation_count, + } + + @classmethod + def from_dict(cls, d: Dict[str, Any]) -> ContrastivePair: + return cls( + id=d["id"], + task_description=d["task_description"], + task_type=d.get("task_type", "general"), + success_approach=d["success_approach"], + failure_approach=d["failure_approach"], + outcome_delta=d.get("outcome_delta", 0.5), + created_at=d.get("created_at", time.time()), + user_id=d.get("user_id", "default"), + tags=d.get("tags", []), + validation_count=d.get("validation_count", 0), + ) + + def to_compact(self) -> Dict[str, str]: + """Compact format for HyperContext — what the agent sees.""" + return { + "task": self.task_description[:200], + "do": self.success_approach[:300], + "avoid": self.failure_approach[:300], + "confidence": round(min(1.0, 0.5 + 0.1 * self.validation_count), 2), + } + + +class ContrastiveStore: + """Stores and retrieves contrastive pairs. + + Persistence: JSONL file on disk. Retrieval: keyword matching + (no embedder dependency — works on edge devices). + """ + + def __init__(self, data_dir: Optional[str] = None): + self._dir = data_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "contrastive" + ) + os.makedirs(self._dir, exist_ok=True) + self._pairs: Dict[str, ContrastivePair] = {} + self._load() + + def add_pair( + self, + task_description: str, + success_approach: str, + failure_approach: str, + task_type: str = "general", + outcome_delta: float = 0.5, + user_id: str = "default", + tags: Optional[List[str]] = None, + ) -> ContrastivePair: + """Add a contrastive pair from a completed task.""" + pair = ContrastivePair( + id=str(uuid.uuid4()), + task_description=task_description, + task_type=task_type, + success_approach=success_approach, + failure_approach=failure_approach, + outcome_delta=outcome_delta, + created_at=time.time(), + user_id=user_id, + tags=tags or [task_type], + ) + self._pairs[pair.id] = pair + self._append(pair) + return pair + + def retrieve_contrasts( + self, + task_description: str, + user_id: str = "default", + limit: int = 3, + ) -> List[ContrastivePair]: + """Find contrastive pairs relevant to a task description. + + Uses word-overlap scoring — fast, no embedder required. + """ + query_words = set(task_description.lower().split()) + if not query_words: + return [] + + scored: List[tuple] = [] + for pair in self._pairs.values(): + if pair.user_id != user_id: + continue + pair_words = set(pair.task_description.lower().split()) + pair_words |= set(pair.task_type.lower().split()) + pair_words |= set(t.lower() for t in pair.tags) + overlap = len(query_words & pair_words) + if overlap > 0: + # Boost by validation count and outcome_delta + score = overlap + pair.validation_count * 0.5 + pair.outcome_delta + scored.append((pair, score)) + + scored.sort(key=lambda x: x[1], reverse=True) + return [p for p, _ in scored[:limit]] + + def matts_score( + self, + query: str, + candidate_texts: List[str], + user_id: str = "default", + ) -> List[float]: + """Memory-Aware Test-Time Scaling (MaTTS) — re-rank by contrastive evidence. + + For each candidate, compute how much it aligns with success approaches + vs failure approaches from relevant contrastive pairs. + + Returns a list of boost factors (0.0 to 1.0) parallel to candidate_texts. + """ + if not candidate_texts: + return [] + + # Find relevant contrasts + contrasts = self.retrieve_contrasts(query, user_id=user_id, limit=5) + if not contrasts: + return [0.0] * len(candidate_texts) + + boosts = [] + for text in candidate_texts: + text_lower = text.lower() + text_words = set(text_lower.split()) + success_signal = 0.0 + failure_signal = 0.0 + + for pair in contrasts: + success_words = set(pair.success_approach.lower().split()) + failure_words = set(pair.failure_approach.lower().split()) + + success_overlap = len(text_words & success_words) + failure_overlap = len(text_words & failure_words) + + success_signal += success_overlap * pair.outcome_delta + failure_signal += failure_overlap * pair.outcome_delta + + # Normalize to 0-1 range. Positive = aligns with success. + total = success_signal + failure_signal + if total > 0: + boost = (success_signal - failure_signal) / total + boosts.append(max(0.0, min(1.0, (boost + 1) / 2))) + else: + boosts.append(0.0) + + return boosts + + def validate(self, pair_id: str) -> None: + """Mark a contrastive pair as validated (proved useful).""" + pair = self._pairs.get(pair_id) + if pair: + pair.validation_count += 1 + self._save_all() + + def get_dpo_pairs(self, limit: int = 50) -> List[Dict[str, str]]: + """Export contrastive pairs as DPO training data.""" + pairs = sorted( + self._pairs.values(), + key=lambda p: p.validation_count + p.outcome_delta, + reverse=True, + ) + return [ + { + "prompt": f"[TASK] {p.task_description}\n[TYPE] {p.task_type}", + "chosen": p.success_approach, + "rejected": p.failure_approach, + } + for p in pairs[:limit] + ] + + def get_stats(self) -> Dict[str, Any]: + return { + "total_pairs": len(self._pairs), + "validated_pairs": sum( + 1 for p in self._pairs.values() if p.validation_count > 0 + ), + "task_types": list({p.task_type for p in self._pairs.values()}), + } + + # ------------------------------------------------------------------ + # Persistence + # ------------------------------------------------------------------ + + def _append(self, pair: ContrastivePair) -> None: + path = os.path.join(self._dir, "pairs.jsonl") + try: + with open(path, "a", encoding="utf-8") as f: + f.write(json.dumps(pair.to_dict(), ensure_ascii=False) + "\n") + except OSError as e: + logger.debug("Failed to append contrastive pair: %s", e) + + def _save_all(self) -> None: + path = os.path.join(self._dir, "pairs.jsonl") + try: + with open(path, "w", encoding="utf-8") as f: + for pair in self._pairs.values(): + f.write(json.dumps(pair.to_dict(), ensure_ascii=False) + "\n") + except OSError as e: + logger.debug("Failed to save contrastive pairs: %s", e) + + def _load(self) -> None: + path = os.path.join(self._dir, "pairs.jsonl") + if not os.path.exists(path): + return + try: + with open(path, "r", encoding="utf-8") as f: + for line in f: + line = line.strip() + if not line: + continue + try: + data = json.loads(line) + pair = ContrastivePair.from_dict(data) + self._pairs[pair.id] = pair + except (json.JSONDecodeError, KeyError): + continue + except OSError as e: + logger.debug("Failed to load contrastive pairs: %s", e) diff --git a/dhee/core/evolution.py b/dhee/core/evolution.py index 9d871c7..305a84a 100644 --- a/dhee/core/evolution.py +++ b/dhee/core/evolution.py @@ -97,6 +97,16 @@ def __init__( except Exception as e: logger.debug("Nididhyasana init skipped: %s", e) + # Phase 2: MetaBuddhi (self-referential cognition) + self._meta_buddhi = None + try: + from dhee.core.meta_buddhi import MetaBuddhi + self._meta_buddhi = MetaBuddhi( + data_dir=os.path.join(self._data_dir, "meta_buddhi"), + ) + except Exception as e: + logger.debug("MetaBuddhi init skipped: %s", e) + # ------------------------------------------------------------------ # WRITE path hooks # ------------------------------------------------------------------ @@ -276,28 +286,48 @@ def on_answer_corrected( def check_evolution(self) -> Optional[Dict[str, Any]]: """Check if auto-evolution should trigger. Call periodically. + Runs two loops: + 1. Nididhyasana: model weight updates (SFT/DPO/RL) + 2. MetaBuddhi: retrieval strategy mutations (propose/evaluate/promote) + Returns evolution cycle result if triggered, None otherwise. """ - if not self._nididhyasana: - return None + result: Optional[Dict[str, Any]] = None - try: - should, reason = self._nididhyasana.should_evolve() - if should: - logger.info("Auto-evolution triggered: %s", reason) - cycle = self._nididhyasana.evolve() - if cycle: - return { - "cycle_id": cycle.cycle_id, - "verdict": cycle.verdict, - "karma_net": cycle.karma_net, - "hot_swapped": cycle.hot_swapped, - "error": cycle.error, - } - except Exception as e: - logger.debug("Evolution check failed: %s", e) + # 1. Nididhyasana: model training + if self._nididhyasana: + try: + should, reason = self._nididhyasana.should_evolve() + if should: + logger.info("Auto-evolution triggered: %s", reason) + cycle = self._nididhyasana.evolve() + if cycle: + result = { + "cycle_id": cycle.cycle_id, + "verdict": cycle.verdict, + "karma_net": cycle.karma_net, + "hot_swapped": cycle.hot_swapped, + "error": cycle.error, + } + except Exception as e: + logger.debug("Nididhyasana check failed: %s", e) + + # 2. MetaBuddhi: strategy improvement proposals + if self._meta_buddhi and self._samskara: + try: + signals = self._samskara.get_training_signals() + vasana_report = signals.get("vasana_report") + degrading = signals.get("degrading_dimensions", []) + if degrading: + attempt = self._meta_buddhi.propose_improvement( + vasana_report=vasana_report, + ) + if attempt: + logger.info("MetaBuddhi proposed: %s", attempt.rationale) + except Exception as e: + logger.debug("MetaBuddhi check failed: %s", e) - return None + return result # ------------------------------------------------------------------ # Status and persistence @@ -319,6 +349,9 @@ def get_status(self) -> Dict[str, Any]: if self._nididhyasana: status["nididhyasana"] = self._nididhyasana.get_status() + if self._meta_buddhi: + status["meta_buddhi"] = self._meta_buddhi.get_stats() + return status def flush(self) -> None: diff --git a/dhee/core/graph.py b/dhee/core/graph.py index e72cec2..aa90cb7 100644 --- a/dhee/core/graph.py +++ b/dhee/core/graph.py @@ -6,6 +6,7 @@ from __future__ import annotations +import os import re import json import logging @@ -52,6 +53,7 @@ class EntityType(str, Enum): PROJECT = "project" TOOL = "tool" PREFERENCE = "preference" + DYNAMIC = "dynamic" # Schema-free entity discovered at runtime UNKNOWN = "unknown" @@ -541,6 +543,25 @@ def stats(self) -> Dict[str, Any]: }, } + # ── Persistence ── + + def save(self, path: str) -> None: + """Persist graph to a JSON file on disk.""" + os.makedirs(os.path.dirname(path) or ".", exist_ok=True) + tmp = path + ".tmp" + with open(tmp, "w", encoding="utf-8") as f: + json.dump(self.to_dict(), f, ensure_ascii=False) + os.replace(tmp, path) + + @classmethod + def load(cls, path: str, llm=None) -> "KnowledgeGraph": + """Load graph from a JSON file. Returns empty graph if file missing.""" + if not os.path.exists(path): + return cls(llm=llm) + with open(path, "r", encoding="utf-8") as f: + data = json.load(f) + return cls.from_dict(data, llm=llm) + # ── Causal language detection (module-level) ── diff --git a/dhee/core/graph_evolution.py b/dhee/core/graph_evolution.py new file mode 100644 index 0000000..3192c53 --- /dev/null +++ b/dhee/core/graph_evolution.py @@ -0,0 +1,600 @@ +"""Evolving knowledge graph — versioned entities, personalized PageRank, schema-free extraction. + +Extends KnowledgeGraph (graph.py) with three capabilities: + +1. **Entity versioning**: Every entity mutation is stored as a version snapshot. + Queries can ask "what was X at time T?" and diffs show how entities evolve. + +2. **Personalized PageRank**: Per-user / per-agent importance scores over + the entity–memory graph. Guides retrieval toward what matters *to this user*. + +3. **Schema-free extraction**: Uses BuddhiMini (or any LLM) to discover + entity types at runtime, stored as EntityType.DYNAMIC with a type_label. +""" + +from __future__ import annotations + +import json +import logging +import os +import time +from collections import defaultdict +from dataclasses import dataclass, field +from datetime import datetime, timezone +from typing import Any, Dict, List, Optional, Set, Tuple + +from dhee.core.graph import ( + Entity, + EntityType, + KnowledgeGraph, + Relationship, + RelationType, +) + +logger = logging.getLogger(__name__) + + +# --------------------------------------------------------------------------- +# Entity Versioning +# --------------------------------------------------------------------------- + +@dataclass +class EntityVersion: + """A point-in-time snapshot of an entity's state.""" + + entity_name: str + version: int + timestamp: str # ISO-8601 + entity_type: str + type_label: Optional[str] = None # For DYNAMIC entities + aliases: List[str] = field(default_factory=list) + memory_ids: List[str] = field(default_factory=list) + metadata: Dict[str, Any] = field(default_factory=dict) + change_reason: str = "" # What triggered this version + + def to_dict(self) -> Dict[str, Any]: + return { + "entity_name": self.entity_name, + "version": self.version, + "timestamp": self.timestamp, + "entity_type": self.entity_type, + "type_label": self.type_label, + "aliases": self.aliases, + "memory_ids": self.memory_ids, + "metadata": self.metadata, + "change_reason": self.change_reason, + } + + @classmethod + def from_dict(cls, data: Dict[str, Any]) -> "EntityVersion": + return cls( + entity_name=data["entity_name"], + version=data["version"], + timestamp=data["timestamp"], + entity_type=data["entity_type"], + type_label=data.get("type_label"), + aliases=data.get("aliases", []), + memory_ids=data.get("memory_ids", []), + metadata=data.get("metadata", {}), + change_reason=data.get("change_reason", ""), + ) + + +def _now_iso() -> str: + return datetime.now(timezone.utc).isoformat() + + +class EntityVersionStore: + """Append-only version log for entity snapshots. + + Stored as JSONL on disk — one line per version, O(1) append. + """ + + def __init__(self, path: str): + self._path = path + self._versions: Dict[str, List[EntityVersion]] = defaultdict(list) + self._load() + + def _load(self) -> None: + if not os.path.exists(self._path): + return + try: + with open(self._path, "r", encoding="utf-8") as f: + for line in f: + line = line.strip() + if not line: + continue + v = EntityVersion.from_dict(json.loads(line)) + self._versions[v.entity_name].append(v) + except (OSError, json.JSONDecodeError) as e: + logger.warning("Failed to load entity versions: %s", e) + + def record(self, entity: Entity, reason: str = "") -> EntityVersion: + """Snapshot the current state of an entity.""" + history = self._versions[entity.name] + version_num = (history[-1].version + 1) if history else 1 + + v = EntityVersion( + entity_name=entity.name, + version=version_num, + timestamp=_now_iso(), + entity_type=entity.entity_type.value, + type_label=entity.metadata.get("type_label"), + aliases=sorted(entity.aliases), + memory_ids=sorted(entity.memory_ids), + metadata=dict(entity.metadata), + change_reason=reason, + ) + + history.append(v) + self._append(v) + return v + + def _append(self, v: EntityVersion) -> None: + try: + os.makedirs(os.path.dirname(self._path) or ".", exist_ok=True) + with open(self._path, "a", encoding="utf-8") as f: + f.write(json.dumps(v.to_dict(), ensure_ascii=False) + "\n") + except OSError as e: + logger.warning("Failed to persist entity version: %s", e) + + def get_history(self, entity_name: str) -> List[EntityVersion]: + """All versions of an entity, oldest first.""" + return list(self._versions.get(entity_name, [])) + + def get_at_time(self, entity_name: str, iso_time: str) -> Optional[EntityVersion]: + """Get the entity version that was current at a given time.""" + history = self._versions.get(entity_name, []) + result = None + for v in history: + if v.timestamp <= iso_time: + result = v + else: + break + return result + + def diff(self, entity_name: str, v1: int, v2: int) -> Dict[str, Any]: + """Compute the difference between two versions of an entity.""" + history = self._versions.get(entity_name, []) + ver_map = {v.version: v for v in history} + old = ver_map.get(v1) + new = ver_map.get(v2) + if not old or not new: + return {"error": "version not found"} + + changes: Dict[str, Any] = {} + if old.entity_type != new.entity_type: + changes["entity_type"] = {"old": old.entity_type, "new": new.entity_type} + if old.type_label != new.type_label: + changes["type_label"] = {"old": old.type_label, "new": new.type_label} + + old_aliases = set(old.aliases) + new_aliases = set(new.aliases) + if old_aliases != new_aliases: + changes["aliases_added"] = sorted(new_aliases - old_aliases) + changes["aliases_removed"] = sorted(old_aliases - new_aliases) + + old_mids = set(old.memory_ids) + new_mids = set(new.memory_ids) + if old_mids != new_mids: + changes["memories_added"] = sorted(new_mids - old_mids) + changes["memories_removed"] = sorted(old_mids - new_mids) + + # Metadata diff (shallow) + for key in set(old.metadata) | set(new.metadata): + old_val = old.metadata.get(key) + new_val = new.metadata.get(key) + if old_val != new_val: + changes.setdefault("metadata", {})[key] = { + "old": old_val, "new": new_val, + } + + return { + "entity": entity_name, + "from_version": v1, + "to_version": v2, + "changes": changes, + } + + @property + def entity_count(self) -> int: + return len(self._versions) + + +# --------------------------------------------------------------------------- +# Personalized PageRank +# --------------------------------------------------------------------------- + +class PersonalizedPageRank: + """Per-user importance ranking over the entity–memory graph. + + Runs a standard power-iteration PageRank seeded from a user's + interaction history (memories they wrote, entities they mentioned). + Results are cached and refreshed when the graph changes. + """ + + def __init__( + self, + damping: float = 0.85, + iterations: int = 20, + tolerance: float = 1e-6, + ): + self.damping = damping + self.iterations = iterations + self.tolerance = tolerance + self._cache: Dict[str, Dict[str, float]] = {} # user_id -> {node -> score} + + def compute( + self, + graph: KnowledgeGraph, + seed_memory_ids: Optional[Set[str]] = None, + user_id: str = "default", + ) -> Dict[str, float]: + """Compute personalized PageRank for a user. + + Args: + graph: The knowledge graph. + seed_memory_ids: Memories that form the user's personalization + vector. If None, uses all memories. + user_id: Cache key. + + Returns: + Dict mapping node IDs (memory IDs and "entity:name") to scores. + """ + # Build adjacency from relationships + adj: Dict[str, Set[str]] = defaultdict(set) + all_nodes: Set[str] = set() + + for rel in graph.relationships: + adj[rel.source_id].add(rel.target_id) + adj[rel.target_id].add(rel.source_id) + all_nodes.add(rel.source_id) + all_nodes.add(rel.target_id) + + # Add entity nodes + for entity_name, entity in graph.entities.items(): + enode = f"entity:{entity_name}" + all_nodes.add(enode) + for mid in entity.memory_ids: + adj[mid].add(enode) + adj[enode].add(mid) + all_nodes.add(mid) + + if not all_nodes: + return {} + + n = len(all_nodes) + node_list = sorted(all_nodes) + node_idx = {node: i for i, node in enumerate(node_list)} + + # Personalization vector: uniform over seeds, zero elsewhere + personalization = [0.0] * n + if seed_memory_ids: + seeds_in_graph = [ + node_idx[mid] for mid in seed_memory_ids if mid in node_idx + ] + if seeds_in_graph: + weight = 1.0 / len(seeds_in_graph) + for idx in seeds_in_graph: + personalization[idx] = weight + else: + personalization = [1.0 / n] * n + else: + personalization = [1.0 / n] * n + + # Power iteration + scores = [1.0 / n] * n + for _ in range(self.iterations): + new_scores = [0.0] * n + for i, node in enumerate(node_list): + neighbors = adj.get(node, set()) + if not neighbors: + # Dangling node — distribute uniformly + share = scores[i] / n + for j in range(n): + new_scores[j] += share + else: + share = scores[i] / len(neighbors) + for nb in neighbors: + if nb in node_idx: + new_scores[node_idx[nb]] += share + + # Apply damping + personalization + for i in range(n): + new_scores[i] = ( + (1 - self.damping) * personalization[i] + + self.damping * new_scores[i] + ) + + # Check convergence + delta = sum(abs(new_scores[i] - scores[i]) for i in range(n)) + scores = new_scores + if delta < self.tolerance: + break + + result = {node_list[i]: scores[i] for i in range(n)} + self._cache[user_id] = result + return result + + def get_top_entities( + self, + graph: KnowledgeGraph, + user_id: str = "default", + seed_memory_ids: Optional[Set[str]] = None, + limit: int = 20, + ) -> List[Tuple[str, float]]: + """Get top-ranked entities for a user.""" + if user_id not in self._cache: + self.compute(graph, seed_memory_ids=seed_memory_ids, user_id=user_id) + + scores = self._cache.get(user_id, {}) + entity_scores = [ + (name.replace("entity:", ""), score) + for name, score in scores.items() + if name.startswith("entity:") + ] + entity_scores.sort(key=lambda x: x[1], reverse=True) + return entity_scores[:limit] + + def boost_retrieval( + self, + memory_ids: List[str], + user_id: str = "default", + ) -> Dict[str, float]: + """Get PageRank boost factors for a set of candidate memory IDs.""" + scores = self._cache.get(user_id, {}) + if not scores: + return {} + return {mid: scores.get(mid, 0.0) for mid in memory_ids} + + def invalidate(self, user_id: Optional[str] = None) -> None: + """Clear cached scores. Call when graph changes.""" + if user_id: + self._cache.pop(user_id, None) + else: + self._cache.clear() + + +# --------------------------------------------------------------------------- +# Schema-Free Entity Extraction +# --------------------------------------------------------------------------- + +_SCHEMA_FREE_PROMPT = """Extract entities from the following text. For each entity, provide: +- name: The entity name +- type: A descriptive type (e.g., "person", "technology", "framework", "metric", + "emotion", "event", "disease", "recipe" — any type that fits, not limited to a fixed set) +- relevance: How important this entity is to the text (0.0 to 1.0) + +Text: {content} + +Return a JSON array. Example: +[{{"name": "FastAPI", "type": "framework", "relevance": 0.9}}] + +Return ONLY the JSON array:""" + + +def extract_entities_schema_free( + content: str, + memory_id: str, + graph: KnowledgeGraph, + llm: Any = None, + min_relevance: float = 0.3, +) -> List[Entity]: + """Extract entities without a fixed type schema. + + Uses LLM to discover entity types at runtime. Discovered types are + stored as EntityType.DYNAMIC with a ``type_label`` in metadata. + + Falls back to graph.extract_entities() regex path if no LLM. + """ + if not llm: + return graph.extract_entities(content, memory_id, use_llm=False) + + prompt = _SCHEMA_FREE_PROMPT.format(content=content[:2000]) + + try: + response = llm.generate(prompt) + arr_start = response.find("[") + if arr_start < 0: + return graph.extract_entities(content, memory_id, use_llm=False) + + items, _ = json.JSONDecoder().raw_decode(response, arr_start) + except Exception as e: + logger.debug("Schema-free extraction failed (%s), falling back to regex", e) + return graph.extract_entities(content, memory_id, use_llm=False) + + # Known EntityType values (lowercase) + _known_types = {t.value for t in EntityType} + + entities: List[Entity] = [] + for item in items: + name = item.get("name", "").strip() + if not name: + continue + + relevance = float(item.get("relevance", 0.5)) + if relevance < min_relevance: + continue + + raw_type = item.get("type", "unknown").strip().lower() + + # Map to existing enum if possible; otherwise DYNAMIC + if raw_type in _known_types: + entity_type = EntityType(raw_type) + type_label = None + else: + entity_type = EntityType.DYNAMIC + type_label = raw_type + + entity = graph._get_or_create_entity(name, entity_type) + entity.memory_ids.add(memory_id) + entity.metadata["relevance"] = max( + entity.metadata.get("relevance", 0.0), relevance, + ) + if type_label: + entity.metadata["type_label"] = type_label + + entities.append(entity) + + graph.memory_entities[memory_id] = {e.name for e in entities} + return entities + + +# --------------------------------------------------------------------------- +# EvolvingGraph — wraps it all together +# --------------------------------------------------------------------------- + +class EvolvingGraph: + """Knowledge graph with entity versioning, PageRank, and schema-free extraction. + + Drop-in extension of KnowledgeGraph — delegates core graph operations + and adds evolution capabilities on top. + """ + + def __init__( + self, + data_dir: Optional[str] = None, + llm: Any = None, + damping: float = 0.85, + ): + self._data_dir = data_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "graph", + ) + os.makedirs(self._data_dir, exist_ok=True) + + graph_path = os.path.join(self._data_dir, "graph.json") + self.graph = KnowledgeGraph.load(graph_path, llm=llm) + + self._versions = EntityVersionStore( + os.path.join(self._data_dir, "entity_versions.jsonl"), + ) + self._pagerank = PersonalizedPageRank(damping=damping) + self._llm = llm + + # ── Entity operations (versioned) ── + + def extract_and_version( + self, + content: str, + memory_id: str, + reason: str = "new_memory", + schema_free: bool = True, + ) -> List[Entity]: + """Extract entities from content and record versions for any changes.""" + if schema_free and self._llm: + entities = extract_entities_schema_free( + content, memory_id, self.graph, llm=self._llm, + ) + else: + entities = self.graph.extract_entities(content, memory_id) + + # Record version for each entity that was touched + for entity in entities: + self._versions.record(entity, reason=reason) + + # Invalidate PageRank caches (graph changed) + self._pagerank.invalidate() + + return entities + + def update_entity( + self, + entity_name: str, + updates: Dict[str, Any], + reason: str = "update", + ) -> Optional[Entity]: + """Update an entity's fields and record the version.""" + entity = self.graph.entities.get(entity_name) + if not entity: + return None + + if "entity_type" in updates: + entity.entity_type = EntityType(updates["entity_type"]) + if "aliases" in updates: + entity.aliases.update(updates["aliases"]) + if "metadata" in updates: + entity.metadata.update(updates["metadata"]) + + self._versions.record(entity, reason=reason) + self._pagerank.invalidate() + return entity + + def get_entity_history(self, entity_name: str) -> List[EntityVersion]: + return self._versions.get_history(entity_name) + + def get_entity_at_time( + self, entity_name: str, iso_time: str, + ) -> Optional[EntityVersion]: + return self._versions.get_at_time(entity_name, iso_time) + + def entity_diff( + self, entity_name: str, v1: int, v2: int, + ) -> Dict[str, Any]: + return self._versions.diff(entity_name, v1, v2) + + # ── PageRank ── + + def compute_pagerank( + self, + user_id: str = "default", + seed_memory_ids: Optional[Set[str]] = None, + ) -> Dict[str, float]: + return self._pagerank.compute( + self.graph, seed_memory_ids=seed_memory_ids, user_id=user_id, + ) + + def get_important_entities( + self, + user_id: str = "default", + seed_memory_ids: Optional[Set[str]] = None, + limit: int = 20, + ) -> List[Tuple[str, float]]: + return self._pagerank.get_top_entities( + self.graph, user_id=user_id, + seed_memory_ids=seed_memory_ids, limit=limit, + ) + + def pagerank_boost( + self, + memory_ids: List[str], + user_id: str = "default", + ) -> Dict[str, float]: + return self._pagerank.boost_retrieval(memory_ids, user_id=user_id) + + # ── Graph delegation ── + + def add_relationship(self, *args, **kwargs) -> Relationship: + rel = self.graph.add_relationship(*args, **kwargs) + self._pagerank.invalidate() + return rel + + def link_by_shared_entities(self, memory_id: str) -> List[Relationship]: + rels = self.graph.link_by_shared_entities(memory_id) + if rels: + self._pagerank.invalidate() + return rels + + def get_related_memories(self, *args, **kwargs): + return self.graph.get_related_memories(*args, **kwargs) + + def get_causal_chain(self, *args, **kwargs): + return self.graph.get_causal_chain(*args, **kwargs) + + def get_memory_graph(self, memory_id: str) -> Dict[str, Any]: + return self.graph.get_memory_graph(memory_id) + + # ── Persistence ── + + def save(self) -> None: + """Persist graph to disk. Version store auto-persists on append.""" + graph_path = os.path.join(self._data_dir, "graph.json") + self.graph.save(graph_path) + + def stats(self) -> Dict[str, Any]: + base = self.graph.stats() + base["versioned_entities"] = self._versions.entity_count + base["dynamic_entities"] = sum( + 1 for e in self.graph.entities.values() + if e.entity_type == EntityType.DYNAMIC + ) + return base diff --git a/dhee/core/heuristic.py b/dhee/core/heuristic.py new file mode 100644 index 0000000..89767e6 --- /dev/null +++ b/dhee/core/heuristic.py @@ -0,0 +1,370 @@ +"""Heuristic Distillation — abstract transferable reasoning patterns. + +Based on ERL (arXiv:2603.24639): distill trajectories into abstract heuristics, +not raw logs. +7.8% over baselines by learning transferable patterns. + +The key insight: raw trajectory logs are too specific to transfer across tasks. +Abstract heuristics like "decompose auth problems into token lifecycle stages" +transfer where "fixed JWT refresh in auth.py line 42" does not. + +Three abstraction levels: + specific — "When JWT tokens expire, check refresh logic first" + domain — "For authentication bugs, trace the token lifecycle" + universal — "When debugging, start with the most constrained component" + +Heuristics are: + 1. Surfaced in HyperContext alongside insights + 2. Used as training data for BuddhiMini's [HEURISTIC] task head + 3. Validated/invalidated by outcomes, like insights +""" + +from __future__ import annotations + +import json +import logging +import os +import time +import uuid +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional + +logger = logging.getLogger(__name__) + + +@dataclass +class Heuristic: + """An abstract, transferable reasoning pattern.""" + + id: str + content: str + abstraction_level: str # specific | domain | universal + source_task_types: List[str] + confidence: float # 0-1, updated by outcomes + created_at: float + user_id: str = "default" + validation_count: int = 0 + invalidation_count: int = 0 + tags: List[str] = field(default_factory=list) + + def strength(self) -> float: + total = self.validation_count + self.invalidation_count + if total == 0: + return self.confidence + return self.confidence * (self.validation_count / total) + + def to_dict(self) -> Dict[str, Any]: + return { + "id": self.id, + "content": self.content, + "abstraction_level": self.abstraction_level, + "source_task_types": self.source_task_types, + "confidence": round(self.confidence, 3), + "strength": round(self.strength(), 3), + "created_at": self.created_at, + "user_id": self.user_id, + "validation_count": self.validation_count, + "invalidation_count": self.invalidation_count, + "tags": self.tags, + } + + @classmethod + def from_dict(cls, d: Dict[str, Any]) -> Heuristic: + return cls( + id=d["id"], + content=d["content"], + abstraction_level=d.get("abstraction_level", "domain"), + source_task_types=d.get("source_task_types", []), + confidence=d.get("confidence", 0.5), + created_at=d.get("created_at", time.time()), + user_id=d.get("user_id", "default"), + validation_count=d.get("validation_count", 0), + invalidation_count=d.get("invalidation_count", 0), + tags=d.get("tags", []), + ) + + def to_compact(self) -> Dict[str, Any]: + """Compact format for HyperContext.""" + return { + "heuristic": self.content[:300], + "level": self.abstraction_level, + "confidence": round(self.strength(), 2), + "applies_to": self.source_task_types[:3], + } + + +class HeuristicDistiller: + """Distills abstract heuristics from trajectories and task outcomes. + + Works in two modes: + 1. With LLM: asks the model to generalize from concrete experiences + 2. Without LLM: extracts patterns heuristically from trajectory structure + + Either way, the output is the same: a Heuristic dataclass stored to disk. + """ + + def __init__(self, data_dir: Optional[str] = None, llm=None): + self._dir = data_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "heuristics" + ) + os.makedirs(self._dir, exist_ok=True) + self._llm = llm + self._heuristics: Dict[str, Heuristic] = {} + self._load() + + def distill_from_trajectory( + self, + task_description: str, + task_type: str, + what_worked: str, + what_failed: Optional[str] = None, + user_id: str = "default", + level: str = "domain", + ) -> Heuristic: + """Distill a heuristic from a single trajectory's outcome. + + Called after checkpoint() when what_worked is provided. + """ + content = self._abstract(task_description, task_type, what_worked, what_failed, level) + + heuristic = Heuristic( + id=str(uuid.uuid4()), + content=content, + abstraction_level=level, + source_task_types=[task_type], + confidence=0.6, + created_at=time.time(), + user_id=user_id, + tags=[task_type, level], + ) + + # Deduplicate: if a very similar heuristic exists, boost it instead + existing = self._find_similar(content, user_id) + if existing: + existing.validation_count += 1 + existing.confidence = min(1.0, existing.confidence + 0.05) + if task_type not in existing.source_task_types: + existing.source_task_types.append(task_type) + self._save_all() + return existing + + self._heuristics[heuristic.id] = heuristic + self._append(heuristic) + return heuristic + + def distill_from_cluster( + self, + task_descriptions: List[str], + task_type: str, + common_patterns: List[str], + user_id: str = "default", + ) -> List[Heuristic]: + """Distill heuristics from a cluster of similar trajectories. + + Called by SkillMiner after clustering successful trajectories. + Finds what's common across multiple successes → more abstract. + """ + if not common_patterns: + return [] + + heuristics = [] + for pattern in common_patterns[:5]: + # Cluster-derived patterns are more reliable + level = "domain" if len(task_descriptions) >= 3 else "specific" + h = Heuristic( + id=str(uuid.uuid4()), + content=pattern, + abstraction_level=level, + source_task_types=[task_type], + confidence=min(0.9, 0.5 + 0.1 * len(task_descriptions)), + created_at=time.time(), + user_id=user_id, + tags=[task_type, level, "cluster_derived"], + ) + + existing = self._find_similar(pattern, user_id) + if existing: + existing.validation_count += 1 + existing.confidence = min(1.0, existing.confidence + 0.05) + heuristics.append(existing) + else: + self._heuristics[h.id] = h + heuristics.append(h) + + self._save_all() + return heuristics + + def retrieve_relevant( + self, + task_description: str, + user_id: str = "default", + limit: int = 5, + ) -> List[Heuristic]: + """Find heuristics relevant to a task, sorted by strength.""" + query_words = set(task_description.lower().split()) + if not query_words: + return [] + + scored: List[tuple] = [] + for h in self._heuristics.values(): + if h.user_id != user_id or h.strength() < 0.1: + continue + h_words = set(h.content.lower().split()) + h_words |= set(t.lower() for t in h.tags) + h_words |= set(t.lower() for t in h.source_task_types) + overlap = len(query_words & h_words) + if overlap > 0: + # Higher abstraction = more likely to be relevant + level_bonus = {"universal": 0.3, "domain": 0.15, "specific": 0.0} + score = overlap + h.strength() + level_bonus.get(h.abstraction_level, 0) + scored.append((h, score)) + + scored.sort(key=lambda x: x[1], reverse=True) + return [h for h, _ in scored[:limit]] + + def validate(self, heuristic_id: str, validated: bool = True) -> None: + """Update a heuristic based on outcome feedback.""" + h = self._heuristics.get(heuristic_id) + if not h: + return + if validated: + h.validation_count += 1 + h.confidence = min(1.0, h.confidence + 0.05) + else: + h.invalidation_count += 1 + h.confidence = max(0.0, h.confidence - 0.1) + self._save_all() + + def get_stats(self) -> Dict[str, Any]: + by_level = {} + for h in self._heuristics.values(): + by_level[h.abstraction_level] = by_level.get(h.abstraction_level, 0) + 1 + return { + "total": len(self._heuristics), + "by_level": by_level, + "avg_confidence": ( + sum(h.confidence for h in self._heuristics.values()) / len(self._heuristics) + if self._heuristics else 0.0 + ), + } + + # ------------------------------------------------------------------ + # Abstraction engine + # ------------------------------------------------------------------ + + def _abstract( + self, + task_description: str, + task_type: str, + what_worked: str, + what_failed: Optional[str], + level: str, + ) -> str: + """Generate an abstract heuristic from concrete experience.""" + if self._llm: + return self._abstract_with_llm( + task_description, task_type, what_worked, what_failed, level + ) + return self._abstract_heuristic( + task_description, task_type, what_worked, what_failed, level + ) + + def _abstract_with_llm( + self, task: str, task_type: str, worked: str, + failed: Optional[str], level: str, + ) -> str: + """Use LLM to generate an abstract heuristic.""" + prompt = ( + f"Given this experience, write ONE abstract {level}-level heuristic " + f"that would help with similar {task_type} tasks in the future.\n\n" + f"Task: {task}\n" + f"What worked: {worked}\n" + ) + if failed: + prompt += f"What failed: {failed}\n" + prompt += ( + f"\nWrite a single sentence heuristic at the '{level}' level:\n" + f"- specific: directly references this task's domain\n" + f"- domain: applies to all {task_type} tasks\n" + f"- universal: applies to any problem-solving task\n" + f"\nHeuristic:" + ) + try: + result = self._llm.generate(prompt) + return result.strip()[:500] + except Exception: + return self._abstract_heuristic(task, task_type, worked, failed, level) + + def _abstract_heuristic( + self, task: str, task_type: str, worked: str, + failed: Optional[str], level: str, + ) -> str: + """Rule-based heuristic abstraction (no LLM needed).""" + if level == "universal": + # Strip domain specifics, keep the reasoning pattern + return f"When facing {task_type} tasks: {worked[:200]}" + elif level == "domain": + return f"For {task_type}: {worked[:250]}" + else: + return f"On tasks like '{task[:80]}': {worked[:250]}" + + # ------------------------------------------------------------------ + # Deduplication + # ------------------------------------------------------------------ + + def _find_similar(self, content: str, user_id: str) -> Optional[Heuristic]: + """Find an existing heuristic with high word overlap.""" + content_words = set(content.lower().split()) + if len(content_words) < 3: + return None + + for h in self._heuristics.values(): + if h.user_id != user_id: + continue + h_words = set(h.content.lower().split()) + if not h_words: + continue + overlap = len(content_words & h_words) + jaccard = overlap / len(content_words | h_words) + if jaccard > 0.6: + return h + return None + + # ------------------------------------------------------------------ + # Persistence + # ------------------------------------------------------------------ + + def _append(self, heuristic: Heuristic) -> None: + path = os.path.join(self._dir, "heuristics.jsonl") + try: + with open(path, "a", encoding="utf-8") as f: + f.write(json.dumps(heuristic.to_dict(), ensure_ascii=False) + "\n") + except OSError as e: + logger.debug("Failed to append heuristic: %s", e) + + def _save_all(self) -> None: + path = os.path.join(self._dir, "heuristics.jsonl") + try: + with open(path, "w", encoding="utf-8") as f: + for h in self._heuristics.values(): + f.write(json.dumps(h.to_dict(), ensure_ascii=False) + "\n") + except OSError as e: + logger.debug("Failed to save heuristics: %s", e) + + def _load(self) -> None: + path = os.path.join(self._dir, "heuristics.jsonl") + if not os.path.exists(path): + return + try: + with open(path, "r", encoding="utf-8") as f: + for line in f: + line = line.strip() + if not line: + continue + try: + data = json.loads(line) + h = Heuristic.from_dict(data) + self._heuristics[h.id] = h + except (json.JSONDecodeError, KeyError): + continue + except OSError as e: + logger.debug("Failed to load heuristics: %s", e) diff --git a/dhee/core/meta_buddhi.py b/dhee/core/meta_buddhi.py new file mode 100644 index 0000000..a04e3fa --- /dev/null +++ b/dhee/core/meta_buddhi.py @@ -0,0 +1,432 @@ +"""MetaBuddhi — the improvement procedure that improves itself. + +Based on Meta's DGM-Hyperagents (arXiv:2603.19461): self-referential +meta-agents that modify their own improvement procedure. + +The DGM-H insight: the agent doesn't just improve at tasks — it improves +at improving. The meta-level feedback loop: + + 1. MetaBuddhi proposes a strategy change (e.g., increase keyword_weight) + 2. The system runs with the new strategy for N interactions + 3. Samskara signals measure whether retrieval/answer quality improved + 4. If improved → promote the strategy. If degraded → rollback. + 5. The RULES for proposing changes are themselves updated by outcomes. + +This is the self-referential loop: MetaBuddhi modifies the weights +that Buddhi uses, and the results modify how MetaBuddhi proposes changes. + +Strategies are stored as versioned JSON files — fully inspectable, +diffable, and rollback-safe. +""" + +from __future__ import annotations + +import json +import logging +import os +import random +import time +import uuid +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional, Tuple + +from dhee.core.strategy import RetrievalStrategy, StrategyStore + +logger = logging.getLogger(__name__) + +# Tunable knobs and their valid ranges +_TUNABLE_FIELDS = { + "semantic_weight": (0.3, 0.95), + "keyword_weight": (0.05, 0.7), + "recency_boost": (0.0, 0.2), + "strength_floor": (0.0, 0.3), + "contrastive_boost": (0.0, 0.4), + "heuristic_relevance_weight": (0.0, 0.3), + "insight_budget": (3, 20), + "memory_budget": (5, 30), +} + +# How many evaluations before judging a candidate +_MIN_EVAL_COUNT = 5 +# Minimum improvement to justify promotion +_PROMOTION_THRESHOLD = 0.03 + + +@dataclass +class ImprovementAttempt: + """A single proposed change to the retrieval strategy.""" + + id: str + strategy_id: str # the candidate strategy + parent_strategy_id: str # the strategy it mutated from + dimension: str # which field was changed + old_value: float + new_value: float + rationale: str + proposed_at: float + status: str = "evaluating" # evaluating | promoted | rolled_back | abandoned + eval_scores: List[float] = field(default_factory=list) + resolved_at: Optional[float] = None + + def to_dict(self) -> Dict[str, Any]: + return { + "id": self.id, + "strategy_id": self.strategy_id, + "parent_strategy_id": self.parent_strategy_id, + "dimension": self.dimension, + "old_value": self.old_value, + "new_value": self.new_value, + "rationale": self.rationale, + "proposed_at": self.proposed_at, + "status": self.status, + "eval_scores": self.eval_scores[-20:], + "resolved_at": self.resolved_at, + } + + +class MetaBuddhi: + """Self-referential cognition: the improvement procedure that improves itself. + + Operates on a simple loop: + propose → evaluate → promote/rollback → learn from the decision + + The learning happens implicitly: the vasana signals from Samskara + tell MetaBuddhi which dimensions are degrading, so it focuses + proposals on those dimensions. Successful proposals reinforce + the direction; failed ones reverse it. + """ + + def __init__( + self, + data_dir: Optional[str] = None, + strategy_store: Optional[StrategyStore] = None, + ): + self._dir = data_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "meta_buddhi" + ) + os.makedirs(self._dir, exist_ok=True) + + self._store = strategy_store or StrategyStore( + data_dir=os.path.join(self._dir, "strategies") + ) + self._attempts: Dict[str, ImprovementAttempt] = {} + self._pending_attempt: Optional[str] = None + self._load_attempts() + + @property + def strategy_store(self) -> StrategyStore: + return self._store + + def get_active_strategy(self) -> RetrievalStrategy: + return self._store.get_active() + + # ------------------------------------------------------------------ + # Propose + # ------------------------------------------------------------------ + + def propose_improvement( + self, + dimension: Optional[str] = None, + vasana_report: Optional[Dict[str, Any]] = None, + ) -> Optional[ImprovementAttempt]: + """Propose a strategy mutation based on current signals. + + If dimension is given, mutate that field. Otherwise, auto-select + the most degrading dimension from the vasana report. + + Returns None if there's already a pending evaluation. + """ + # Only one active evaluation at a time + if self._pending_attempt: + pending = self._attempts.get(self._pending_attempt) + if pending and pending.status == "evaluating": + return None + + active = self._store.get_active() + + # Pick dimension to improve + if not dimension: + dimension = self._select_dimension(vasana_report) + if not dimension or dimension not in _TUNABLE_FIELDS: + return None + + # Compute mutation + lo, hi = _TUNABLE_FIELDS[dimension] + current_val = getattr(active, dimension) + direction = self._mutation_direction(dimension, vasana_report) + step = (hi - lo) * 0.1 # 10% of range + new_val = current_val + direction * step + + # For integer fields + if isinstance(current_val, int): + new_val = int(round(new_val)) + lo, hi = int(lo), int(hi) + + new_val = max(lo, min(hi, new_val)) + + # Don't propose no-ops + if abs(new_val - current_val) < 1e-6: + return None + + # Create candidate strategy + candidate = RetrievalStrategy( + id=str(uuid.uuid4()), + version=active.version + 1, + name=f"{active.name}_v{active.version + 1}", + description=f"Mutated {dimension}: {current_val} → {new_val}", + parent_id=active.id, + status="candidate", + **{ + k: getattr(active, k) for k in _TUNABLE_FIELDS + if k != dimension + }, + **{dimension: new_val}, + ) + self._store.save(candidate) + + # Create attempt + rationale = self._build_rationale(dimension, current_val, new_val, vasana_report) + attempt = ImprovementAttempt( + id=str(uuid.uuid4()), + strategy_id=candidate.id, + parent_strategy_id=active.id, + dimension=dimension, + old_value=current_val, + new_value=new_val, + rationale=rationale, + proposed_at=time.time(), + ) + self._attempts[attempt.id] = attempt + self._pending_attempt = attempt.id + self._save_attempts() + + logger.info( + "MetaBuddhi proposed: %s %s → %s (%s)", + dimension, current_val, new_val, rationale, + ) + return attempt + + # ------------------------------------------------------------------ + # Evaluate + # ------------------------------------------------------------------ + + def record_evaluation(self, score: float) -> Optional[str]: + """Record an evaluation score for the pending improvement. + + Call this after each interaction while a candidate is being evaluated. + Returns the resolution status if the attempt has been resolved, + or None if still evaluating. + """ + if not self._pending_attempt: + return None + + attempt = self._attempts.get(self._pending_attempt) + if not attempt or attempt.status != "evaluating": + return None + + attempt.eval_scores.append(score) + + # Also track on the candidate strategy + candidate = self._store.get(attempt.strategy_id) + if candidate: + candidate.eval_scores.append(score) + candidate.eval_count += 1 + self._store.save(candidate) + + # Enough data to judge? + if len(attempt.eval_scores) >= _MIN_EVAL_COUNT: + return self._resolve_attempt(attempt) + + self._save_attempts() + return None + + def _resolve_attempt(self, attempt: ImprovementAttempt) -> str: + """Judge whether the improvement helped.""" + parent = self._store.get(attempt.parent_strategy_id) + parent_avg = parent.avg_score if parent and parent.eval_scores else 0.5 + candidate_avg = ( + sum(attempt.eval_scores) / len(attempt.eval_scores) + if attempt.eval_scores else 0.0 + ) + + delta = candidate_avg - parent_avg + + if delta >= _PROMOTION_THRESHOLD: + # Improvement confirmed — promote + self._store.promote(attempt.strategy_id) + attempt.status = "promoted" + logger.info( + "MetaBuddhi promoted strategy: %s (delta=+%.3f)", + attempt.dimension, delta, + ) + else: + # No improvement or regression — rollback + self._store.rollback(attempt.strategy_id) + attempt.status = "rolled_back" + logger.info( + "MetaBuddhi rolled back: %s (delta=%.3f)", + attempt.dimension, delta, + ) + + attempt.resolved_at = time.time() + self._pending_attempt = None + self._save_attempts() + return attempt.status + + # ------------------------------------------------------------------ + # Dimension selection (the meta-meta level) + # ------------------------------------------------------------------ + + def _select_dimension( + self, vasana_report: Optional[Dict[str, Any]] + ) -> Optional[str]: + """Pick the dimension most in need of improvement.""" + if not vasana_report: + # Random exploration + return random.choice(list(_TUNABLE_FIELDS.keys())) + + # Map vasana dimensions to strategy fields + vasana_to_strategy = { + "retrieval_precision": "semantic_weight", + "retrieval_recall": "keyword_weight", + "answer_quality": "insight_budget", + "fact_extraction": "memory_budget", + "dedup_quality": "strength_floor", + } + + # Find the most degrading vasana + worst_dim = None + worst_strength = 0.0 + for name, report in vasana_report.items(): + strength = report.get("strength", 0.0) if isinstance(report, dict) else 0.0 + if strength < worst_strength: + worst_strength = strength + worst_dim = name + + if worst_dim and worst_dim in vasana_to_strategy: + return vasana_to_strategy[worst_dim] + + return random.choice(list(_TUNABLE_FIELDS.keys())) + + def _mutation_direction( + self, + dimension: str, + vasana_report: Optional[Dict[str, Any]], + ) -> float: + """Decide whether to increase (+1) or decrease (-1) a dimension. + + Uses past attempt outcomes to learn which direction works. + """ + # Check history: which direction worked for this dimension? + ups, downs = 0, 0 + for attempt in self._attempts.values(): + if attempt.dimension != dimension: + continue + if attempt.status == "promoted": + if attempt.new_value > attempt.old_value: + ups += 1 + else: + downs += 1 + elif attempt.status == "rolled_back": + if attempt.new_value > attempt.old_value: + downs += 1 + else: + ups += 1 + + if ups > downs: + return 1.0 + elif downs > ups: + return -1.0 + # No history — random + return random.choice([-1.0, 1.0]) + + def _build_rationale( + self, + dimension: str, + old_val: Any, + new_val: Any, + vasana_report: Optional[Dict[str, Any]], + ) -> str: + """Build a human-readable rationale for the proposed change.""" + direction = "increase" if new_val > old_val else "decrease" + reason = "exploratory mutation" + if vasana_report: + degrading = [ + name for name, v in vasana_report.items() + if isinstance(v, dict) and v.get("strength", 0) < -0.1 + ] + if degrading: + reason = f"degrading vasanas: {', '.join(degrading[:3])}" + return f"{direction} {dimension} ({old_val} → {new_val}): {reason}" + + # ------------------------------------------------------------------ + # Status + # ------------------------------------------------------------------ + + def get_stats(self) -> Dict[str, Any]: + active = self._store.get_active() + return { + "active_strategy": active.to_dict() if active else None, + "pending_attempt": ( + self._attempts[self._pending_attempt].to_dict() + if self._pending_attempt and self._pending_attempt in self._attempts + else None + ), + "total_attempts": len(self._attempts), + "promoted": sum( + 1 for a in self._attempts.values() if a.status == "promoted" + ), + "rolled_back": sum( + 1 for a in self._attempts.values() if a.status == "rolled_back" + ), + "strategies_total": len(self._store.list_all()), + } + + # ------------------------------------------------------------------ + # Persistence + # ------------------------------------------------------------------ + + def _save_attempts(self) -> None: + path = os.path.join(self._dir, "attempts.jsonl") + try: + with open(path, "w", encoding="utf-8") as f: + for a in self._attempts.values(): + f.write(json.dumps(a.to_dict(), ensure_ascii=False) + "\n") + # Also save pending pointer + f.write(json.dumps({"_pending": self._pending_attempt}) + "\n") + except OSError as e: + logger.debug("Failed to save attempts: %s", e) + + def _load_attempts(self) -> None: + path = os.path.join(self._dir, "attempts.jsonl") + if not os.path.exists(path): + return + try: + with open(path, "r", encoding="utf-8") as f: + for line in f: + line = line.strip() + if not line: + continue + try: + data = json.loads(line) + if "_pending" in data: + self._pending_attempt = data["_pending"] + continue + attempt = ImprovementAttempt( + id=data["id"], + strategy_id=data["strategy_id"], + parent_strategy_id=data["parent_strategy_id"], + dimension=data["dimension"], + old_value=data["old_value"], + new_value=data["new_value"], + rationale=data.get("rationale", ""), + proposed_at=data.get("proposed_at", time.time()), + status=data.get("status", "evaluating"), + eval_scores=data.get("eval_scores", []), + resolved_at=data.get("resolved_at"), + ) + self._attempts[attempt.id] = attempt + except (KeyError, TypeError): + continue + except (OSError, json.JSONDecodeError) as e: + logger.debug("Failed to load attempts: %s", e) diff --git a/dhee/core/retrieval.py b/dhee/core/retrieval.py index 0ad07d6..68453ef 100644 --- a/dhee/core/retrieval.py +++ b/dhee/core/retrieval.py @@ -142,8 +142,9 @@ def hybrid_score( class HybridSearcher: """Helper class for hybrid search across memories.""" - def __init__(self, alpha: float = 0.7): + def __init__(self, alpha: float = 0.7, contrastive_boost: float = 0.0): self.alpha = alpha + self.contrastive_boost = contrastive_boost def score_memory( self, @@ -153,6 +154,7 @@ def score_memory( echo_keywords: Optional[List[str]] = None, echo_paraphrases: Optional[List[str]] = None, strength: float = 1.0, + contrastive_signal: float = 0.0, ) -> Dict[str, float]: keyword_score = calculate_keyword_score( query_terms=query_terms, @@ -163,9 +165,14 @@ def score_memory( hybrid = hybrid_score(semantic_similarity, keyword_score, self.alpha) + # Apply contrastive boost: results aligned with past successes score higher + if self.contrastive_boost > 0 and contrastive_signal > 0: + hybrid += self.contrastive_boost * contrastive_signal + return { "semantic_score": semantic_similarity, "keyword_score": keyword_score, "hybrid_score": hybrid, + "contrastive_signal": contrastive_signal, "composite_score": composite_score(hybrid, strength), } diff --git a/dhee/core/samskara.py b/dhee/core/samskara.py index 499079d..6cc0bd2 100644 --- a/dhee/core/samskara.py +++ b/dhee/core/samskara.py @@ -494,6 +494,39 @@ def _load_state(self) -> None: except (OSError, json.JSONDecodeError): pass + def get_training_data(self) -> Dict[str, Any]: + """Export accumulated data formatted for BuddhiMini training pipeline. + + Returns SFT examples from session samskaras and DPO pairs from corrections. + Called by BuddhiMini.train_cycle() to feed the progressive trainer. + """ + sft_samples = [] + for s in self._session_samskaras: + if s.input_text and s.output_text: + sample = { + "input": f"[{s.type.value.upper()}] {s.input_text}", + "output": s.output_text, + "type": s.type.value, + "valence": s.valence.value, + } + if s.corrected_text: + sample["corrected"] = s.corrected_text + sft_samples.append(sample) + + return { + "sft_samples": sft_samples, + "dpo_pairs": list(self._dpo_pairs), + "vasana_report": { + name: {"strength": v.strength, "count": v.count} + for name, v in self.vasanas.items() + }, + "degrading_dimensions": [ + name for name, v in self.vasanas.items() + if v.is_degrading + ], + "total_samskaras": self._total_samskaras, + } + def flush(self) -> None: """Persist current state. Call periodically or on shutdown.""" self._save_state() diff --git a/dhee/core/strategy.py b/dhee/core/strategy.py new file mode 100644 index 0000000..001346f --- /dev/null +++ b/dhee/core/strategy.py @@ -0,0 +1,232 @@ +"""Versioned retrieval strategies — inspectable, evolvable, rollback-safe. + +Every strategy is a JSON file on disk (per DGM-H: "make everything a file"). +MetaBuddhi proposes mutations; strategies that improve metrics get promoted; +those that degrade get rolled back. + +A strategy controls scoring weights used by HybridSearcher and Buddhi's +HyperContext assembly. Changing a strategy changes HOW the system retrieves +and prioritizes information — the meta-cognitive lever. +""" + +from __future__ import annotations + +import json +import logging +import os +import time +import uuid +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional + +logger = logging.getLogger(__name__) + + +@dataclass +class RetrievalStrategy: + """A versioned set of scoring weights and retrieval parameters. + + Fields mirror the tunable knobs in Dhee's retrieval pipeline: + - semantic_weight / keyword_weight: HybridSearcher alpha split + - recency_boost: bonus for recently accessed memories + - strength_floor: minimum memory strength to surface + - contrastive_boost: bonus when contrastive evidence supports a result + - heuristic_relevance_weight: how much heuristic matches influence ranking + - insight_budget: max insights to include in HyperContext + - memory_budget: max memories to include in HyperContext + """ + + id: str + version: int + name: str + description: str + + # Retrieval scoring weights + semantic_weight: float = 0.7 + keyword_weight: float = 0.3 + recency_boost: float = 0.05 + strength_floor: float = 0.1 + contrastive_boost: float = 0.15 + heuristic_relevance_weight: float = 0.1 + + # HyperContext budgets + insight_budget: int = 10 + memory_budget: int = 10 + warning_budget: int = 5 + + # Lifecycle + created_at: float = field(default_factory=time.time) + parent_id: Optional[str] = None # which strategy this mutated from + status: str = "active" # active | candidate | retired | rolled_back + + # Performance tracking (populated by MetaBuddhi) + eval_scores: List[float] = field(default_factory=list) + eval_count: int = 0 + + def to_dict(self) -> Dict[str, Any]: + return { + "id": self.id, + "version": self.version, + "name": self.name, + "description": self.description, + "semantic_weight": self.semantic_weight, + "keyword_weight": self.keyword_weight, + "recency_boost": self.recency_boost, + "strength_floor": self.strength_floor, + "contrastive_boost": self.contrastive_boost, + "heuristic_relevance_weight": self.heuristic_relevance_weight, + "insight_budget": self.insight_budget, + "memory_budget": self.memory_budget, + "warning_budget": self.warning_budget, + "created_at": self.created_at, + "parent_id": self.parent_id, + "status": self.status, + "eval_scores": self.eval_scores[-20:], + "eval_count": self.eval_count, + } + + @classmethod + def from_dict(cls, d: Dict[str, Any]) -> RetrievalStrategy: + return cls( + id=d["id"], + version=d.get("version", 1), + name=d.get("name", "unnamed"), + description=d.get("description", ""), + semantic_weight=d.get("semantic_weight", 0.7), + keyword_weight=d.get("keyword_weight", 0.3), + recency_boost=d.get("recency_boost", 0.05), + strength_floor=d.get("strength_floor", 0.1), + contrastive_boost=d.get("contrastive_boost", 0.15), + heuristic_relevance_weight=d.get("heuristic_relevance_weight", 0.1), + insight_budget=d.get("insight_budget", 10), + memory_budget=d.get("memory_budget", 10), + warning_budget=d.get("warning_budget", 5), + created_at=d.get("created_at", time.time()), + parent_id=d.get("parent_id"), + status=d.get("status", "active"), + eval_scores=d.get("eval_scores", []), + eval_count=d.get("eval_count", 0), + ) + + @property + def avg_score(self) -> float: + if not self.eval_scores: + return 0.0 + return sum(self.eval_scores) / len(self.eval_scores) + + +class StrategyStore: + """Manages versioned strategies on disk. + + Each strategy is a JSON file: strategies/{id}.json + One is marked active at a time. History is preserved for rollback. + """ + + def __init__(self, data_dir: Optional[str] = None): + self._dir = data_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "meta_buddhi", "strategies" + ) + os.makedirs(self._dir, exist_ok=True) + self._active_id: Optional[str] = None + self._strategies: Dict[str, RetrievalStrategy] = {} + self._load() + + def get_active(self) -> RetrievalStrategy: + """Get the active strategy. Creates default if none exists.""" + if self._active_id and self._active_id in self._strategies: + return self._strategies[self._active_id] + return self._ensure_default() + + def save(self, strategy: RetrievalStrategy) -> None: + """Persist a strategy to disk.""" + self._strategies[strategy.id] = strategy + path = os.path.join(self._dir, f"{strategy.id}.json") + try: + with open(path, "w", encoding="utf-8") as f: + json.dump(strategy.to_dict(), f, indent=2) + except OSError as e: + logger.debug("Failed to save strategy %s: %s", strategy.id, e) + self._save_index() + + def promote(self, strategy_id: str) -> bool: + """Make a candidate strategy the active one.""" + strategy = self._strategies.get(strategy_id) + if not strategy: + return False + # Retire the current active + if self._active_id and self._active_id in self._strategies: + old = self._strategies[self._active_id] + old.status = "retired" + self.save(old) + # Activate the new one + strategy.status = "active" + self._active_id = strategy.id + self.save(strategy) + return True + + def rollback(self, strategy_id: str) -> Optional[RetrievalStrategy]: + """Roll back a strategy to its parent.""" + strategy = self._strategies.get(strategy_id) + if not strategy or not strategy.parent_id: + return None + parent = self._strategies.get(strategy.parent_id) + if not parent: + return None + strategy.status = "rolled_back" + self.save(strategy) + self.promote(parent.id) + return parent + + def list_all(self) -> List[RetrievalStrategy]: + return list(self._strategies.values()) + + def get(self, strategy_id: str) -> Optional[RetrievalStrategy]: + return self._strategies.get(strategy_id) + + def _ensure_default(self) -> RetrievalStrategy: + """Create the default strategy if none exist.""" + default = RetrievalStrategy( + id=str(uuid.uuid4()), + version=1, + name="default", + description="Balanced default retrieval strategy", + status="active", + ) + self._active_id = default.id + self.save(default) + return default + + def _save_index(self) -> None: + path = os.path.join(self._dir, "_index.json") + try: + with open(path, "w", encoding="utf-8") as f: + json.dump({"active_id": self._active_id}, f) + except OSError: + pass + + def _load(self) -> None: + # Load index + index_path = os.path.join(self._dir, "_index.json") + if os.path.exists(index_path): + try: + with open(index_path, "r", encoding="utf-8") as f: + idx = json.load(f) + self._active_id = idx.get("active_id") + except (OSError, json.JSONDecodeError): + pass + + # Load all strategy files + try: + for fname in os.listdir(self._dir): + if not fname.endswith(".json") or fname.startswith("_"): + continue + fpath = os.path.join(self._dir, fname) + try: + with open(fpath, "r", encoding="utf-8") as f: + data = json.load(f) + s = RetrievalStrategy.from_dict(data) + self._strategies[s.id] = s + except (OSError, json.JSONDecodeError, KeyError): + continue + except OSError: + pass diff --git a/dhee/edge/__init__.py b/dhee/edge/__init__.py new file mode 100644 index 0000000..2e4fc3b --- /dev/null +++ b/dhee/edge/__init__.py @@ -0,0 +1,6 @@ +"""Dhee Edge — minimal-footprint cognition for hardware/humanoid deployment.""" + +from dhee.edge.edge_plugin import DheeEdge +from dhee.edge.edge_trainer import EdgeTrainer + +__all__ = ["DheeEdge", "EdgeTrainer"] diff --git a/dhee/edge/edge_plugin.py b/dhee/edge/edge_plugin.py new file mode 100644 index 0000000..2777aa2 --- /dev/null +++ b/dhee/edge/edge_plugin.py @@ -0,0 +1,322 @@ +"""DheeEdge — minimal cognition plugin for edge/hardware deployment. + +Designed for humanoid robots, IoT devices, and AI hardware products. +All computation runs locally — no cloud API calls, no internet required. + +Constraints: + - LLM: DheeModel (GGUF Q4, ~1.5GB) or mock fallback + - Embedder: ONNX MiniLM (22MB) or hash-based fallback + - Vector store: sqlite_vec (local file) + - RAM: <500MB working set + - No external API calls ever + +Adds embodiment hooks for hardware integration: + - on_sensor_input() — process sensor data into episodic memory + - on_action_result() — record action outcomes for environment learning + - predict_environment() — predict next state from memory patterns + +Usage: + from dhee.edge import DheeEdge + + d = DheeEdge(data_dir="/data/dhee") + d.remember("User prefers quiet mode after 10pm") + d.on_sensor_input("microphone", {"volume_db": 85, "duration": 3.0}) + d.on_action_result("reduce_volume", success=True, env_state={"volume_db": 40}) +""" + +from __future__ import annotations + +import json +import logging +import os +import time +from pathlib import Path +from typing import Any, Dict, List, Optional, Union + +from dhee.adapters.base import DheePlugin + +logger = logging.getLogger(__name__) + + +class DheeEdge(DheePlugin): + """Minimal cognition plugin for edge deployment. + + Forces all-offline providers. No API calls, no internet. + Extends DheePlugin with embodiment hooks for hardware. + + Args: + data_dir: Storage directory (required for edge — no temp dirs). + model_path: Path to GGUF model file for local LLM inference. + user_id: Default user ID. + """ + + def __init__( + self, + data_dir: Union[str, Path], + model_path: Optional[str] = None, + user_id: str = "default", + ): + # Force offline — never make API calls. + # Try persistent storage first; fall back to in-memory if + # sqlite_vec extension isn't available on this platform. + try: + super().__init__( + data_dir=data_dir, + provider="mock", + user_id=user_id, + in_memory=False, + offline=True, + ) + except (AttributeError, OSError) as e: + logger.debug("Persistent storage unavailable (%s), using in-memory", e) + super().__init__( + data_dir=data_dir, + provider="mock", + user_id=user_id, + in_memory=True, + offline=True, + ) + + # Embodiment state + self._sensor_history: List[Dict[str, Any]] = [] + self._action_history: List[Dict[str, Any]] = [] + self._environment_model: Dict[str, Any] = {} + + # Try to upgrade to local GGUF model + if model_path: + self._try_load_local_model(model_path) + + def _try_load_local_model(self, model_path: str) -> None: + """Attempt to load a local GGUF model for on-device LLM inference.""" + if not os.path.exists(model_path): + logger.debug("GGUF model not found at %s, using mock LLM", model_path) + return + try: + from dhee.llms.dhee import DheeLLM + self._engram._memory.llm = DheeLLM( + config={"model_path": model_path, "backend": "gguf"} + ) + logger.info("Loaded local GGUF model: %s", model_path) + except ImportError: + logger.debug("llama-cpp-python not available, using mock LLM") + except Exception as e: + logger.debug("Failed to load GGUF model: %s", e) + + # ------------------------------------------------------------------ + # Embodiment hooks (from Self-evolving Embodied AI, arXiv:2602.04411) + # ------------------------------------------------------------------ + + def on_sensor_input( + self, + sensor_type: str, + data: Dict[str, Any], + user_id: Optional[str] = None, + ) -> Dict[str, Any]: + """Process sensor data into episodic memory. + + Converts raw sensor readings into natural language memories that + can be recalled later. Tracks sensor patterns for environment + prediction. + + Args: + sensor_type: Type of sensor (e.g., "microphone", "camera", "imu") + data: Sensor data dict with readings and metadata + user_id: Override default user_id + + Returns: + {"stored": bool, "id": str, "description": str} + """ + uid = user_id or self._user_id + timestamp = data.get("timestamp", time.time()) + + # Build natural language description from sensor data + description = self._describe_sensor_data(sensor_type, data) + + # Store as memory + result = self.remember( + content=description, + user_id=uid, + metadata={ + "source": "sensor", + "sensor_type": sensor_type, + "timestamp": timestamp, + "raw_data": data, + }, + ) + + # Track in sensor history (bounded) + record = { + "sensor_type": sensor_type, + "data": data, + "timestamp": timestamp, + "description": description, + } + self._sensor_history.append(record) + if len(self._sensor_history) > 500: + self._sensor_history = self._sensor_history[-500:] + + # Update environment model + self._update_environment_model(sensor_type, data) + + result["description"] = description + return result + + def on_action_result( + self, + action: str, + success: bool, + env_state: Optional[Dict[str, Any]] = None, + user_id: Optional[str] = None, + ) -> Dict[str, Any]: + """Record action outcomes for environment self-prediction. + + Builds a causal model: action + context → outcome. Over time, + the system learns which actions work in which states. + + Args: + action: What the agent did (e.g., "reduce_volume", "move_forward") + success: Whether the action achieved its goal + env_state: Environment state after action + user_id: Override default user_id + """ + uid = user_id or self._user_id + + # Store as memory with outcome + outcome_word = "succeeded" if success else "failed" + content = f"Action '{action}' {outcome_word}" + if env_state: + state_summary = ", ".join(f"{k}={v}" for k, v in list(env_state.items())[:5]) + content += f". Environment state: {state_summary}" + + result = self.remember( + content=content, + user_id=uid, + metadata={ + "source": "action_result", + "action": action, + "success": success, + "env_state": env_state, + }, + ) + + # Track action history + record = { + "action": action, + "success": success, + "env_state": env_state, + "timestamp": time.time(), + } + self._action_history.append(record) + if len(self._action_history) > 500: + self._action_history = self._action_history[-500:] + + # Record outcome for performance tracking + task_type = f"action_{action}" + self._buddhi.record_outcome( + user_id=uid, + task_type=task_type, + score=1.0 if success else 0.0, + ) + + return result + + def predict_environment( + self, + current_state: Dict[str, Any], + proposed_action: Optional[str] = None, + ) -> Dict[str, Any]: + """Predict next environment state from memory patterns. + + Uses action history to estimate what will happen if a given + action is taken in the current state. + + Args: + current_state: Current environment state dict + proposed_action: Action being considered (optional) + + Returns: + {"prediction": str, "confidence": float, "similar_outcomes": list} + """ + # Build query from current state + proposed action + state_desc = ", ".join(f"{k}={v}" for k, v in list(current_state.items())[:5]) + query = f"environment state: {state_desc}" + if proposed_action: + query += f", action: {proposed_action}" + + # Search for similar past situations + similar = self.recall(query=query, limit=5) + + # Compute confidence from action history + confidence = 0.0 + outcomes = [] + if proposed_action and self._action_history: + matching = [ + a for a in self._action_history + if a["action"] == proposed_action + ] + if matching: + success_rate = sum(1 for a in matching if a["success"]) / len(matching) + confidence = success_rate + outcomes = matching[-3:] # last 3 similar actions + + # Simple prediction based on success rate + prediction = "unknown" + if confidence > 0.7: + prediction = f"Action '{proposed_action}' is likely to succeed (confidence: {confidence:.0%})" + elif confidence > 0.3: + prediction = f"Action '{proposed_action}' has mixed results (confidence: {confidence:.0%})" + elif confidence > 0 and proposed_action: + prediction = f"Action '{proposed_action}' has often failed (confidence: {confidence:.0%})" + + return { + "prediction": prediction, + "confidence": round(confidence, 3), + "similar_memories": similar[:3], + "recent_outcomes": [ + {"action": o["action"], "success": o["success"]} + for o in outcomes + ], + } + + def adapt_embodiment( + self, + capabilities: Dict[str, Any], + user_id: Optional[str] = None, + ) -> Dict[str, Any]: + """Update self-model when hardware capabilities change. + + Call when sensors are added/removed, actuators change, or the + physical form factor is modified. + + Args: + capabilities: New capability dict (e.g., {"has_camera": True, "arm_reach_cm": 60}) + """ + uid = user_id or self._user_id + cap_desc = ", ".join(f"{k}: {v}" for k, v in capabilities.items()) + content = f"Embodiment update: {cap_desc}" + return self.remember( + content=content, + user_id=uid, + metadata={"source": "embodiment_update", "capabilities": capabilities}, + ) + + # ------------------------------------------------------------------ + # Internal helpers + # ------------------------------------------------------------------ + + def _describe_sensor_data(self, sensor_type: str, data: Dict[str, Any]) -> str: + """Convert raw sensor data to natural language for memory storage.""" + readings = ", ".join( + f"{k}={v}" for k, v in data.items() + if k != "timestamp" and not isinstance(v, (dict, list)) + ) + return f"Sensor[{sensor_type}]: {readings}" + + def _update_environment_model( + self, sensor_type: str, data: Dict[str, Any], + ) -> None: + """Update the running environment model with new sensor data.""" + self._environment_model[sensor_type] = { + "last_reading": data, + "last_updated": time.time(), + } diff --git a/dhee/edge/edge_trainer.py b/dhee/edge/edge_trainer.py new file mode 100644 index 0000000..572bd9c --- /dev/null +++ b/dhee/edge/edge_trainer.py @@ -0,0 +1,508 @@ +"""EdgeTrainer — on-device micro-training for edge deployments. + +Runs minimal LoRA fine-tuning directly on edge hardware (ARM CPU, low-RAM +devices). Designed for DheeEdge scenarios where the model needs to adapt +to its specific user/environment without cloud connectivity. + +Constraints: + - CPU-only training (no CUDA required) + - <2GB peak RAM during training + - LoRA rank 4-8 (tiny adapter, ~2MB) + - Micro-batches of 1-4 samples + - 10-50 gradient steps per cycle (not epochs) + +Training data sources (all local): + - Samskara signals from SamskaraCollector.get_training_data() + - Action outcome pairs from DheeEdge._action_history + - Sensor pattern correlations + +Architecture: + EdgeTrainer does NOT require torch/transformers at init. It checks + for their availability lazily. On devices without PyTorch, it logs + a warning and becomes a no-op. + +Usage: + from dhee.edge.edge_trainer import EdgeTrainer + + trainer = EdgeTrainer( + model_path="/data/models/dhee-2b-q4.gguf", + adapter_dir="/data/dhee/adapters", + ) + + # Check if training is possible on this device + if trainer.can_train: + result = trainer.micro_train(training_data) +""" + +from __future__ import annotations + +import json +import logging +import os +import time +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional + +logger = logging.getLogger(__name__) + + +@dataclass +class MicroTrainResult: + """Result of a micro-training cycle.""" + + success: bool + steps_completed: int = 0 + samples_used: int = 0 + loss_start: float = 0.0 + loss_end: float = 0.0 + adapter_path: Optional[str] = None + duration_seconds: float = 0.0 + error: Optional[str] = None + + def to_dict(self) -> Dict[str, Any]: + return { + "success": self.success, + "steps_completed": self.steps_completed, + "samples_used": self.samples_used, + "loss_start": round(self.loss_start, 4), + "loss_end": round(self.loss_end, 4), + "adapter_path": self.adapter_path, + "duration_seconds": round(self.duration_seconds, 2), + "error": self.error, + } + + +@dataclass +class EdgeTrainingConfig: + """Configuration for edge micro-training.""" + + lora_rank: int = 4 + lora_alpha: int = 8 + learning_rate: float = 2e-4 + max_steps: int = 30 + micro_batch_size: int = 2 + max_seq_len: int = 256 + gradient_accumulation_steps: int = 2 + warmup_steps: int = 3 + weight_decay: float = 0.01 + max_samples: int = 100 # Limit training data + + +class EdgeTrainer: + """On-device micro-training for edge deployments. + + Performs minimal LoRA fine-tuning on CPU with tight resource budgets. + Training is designed to be interruptible — partial progress is saved. + """ + + def __init__( + self, + model_path: Optional[str] = None, + adapter_dir: Optional[str] = None, + config: Optional[EdgeTrainingConfig] = None, + ): + """ + Args: + model_path: Path to the base model (GGUF or safetensors). + adapter_dir: Directory to save/load LoRA adapters. + config: Training hyperparameters. + """ + self._model_path = model_path + self._adapter_dir = adapter_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "edge_adapters", + ) + self.config = config or EdgeTrainingConfig() + self._training_history: List[Dict[str, Any]] = [] + self._torch_available: Optional[bool] = None + + @property + def can_train(self) -> bool: + """Check if training is possible on this device.""" + if self._torch_available is None: + try: + import torch # noqa: F401 + self._torch_available = True + except ImportError: + self._torch_available = False + logger.info( + "PyTorch not available — edge training disabled. " + "Install with: pip install torch --index-url https://download.pytorch.org/whl/cpu" + ) + return self._torch_available and self._model_path is not None + + def micro_train( + self, + training_data: Dict[str, Any], + samskara_signals: Optional[Dict[str, Any]] = None, + ) -> MicroTrainResult: + """Run a micro-training cycle. + + Args: + training_data: Dict with keys: + - sft_samples: List of {"input": str, "output": str} dicts + - dpo_pairs: List of {"chosen": str, "rejected": str} dicts (optional) + samskara_signals: Optional vasana report for sample weighting. + + Returns: + MicroTrainResult with training metrics. + """ + if not self.can_train: + return MicroTrainResult( + success=False, + error="Training not available (missing PyTorch or model)", + ) + + start_time = time.time() + + # Prepare training samples + sft_samples = training_data.get("sft_samples", []) + if not sft_samples: + return MicroTrainResult( + success=False, + error="No training samples provided", + ) + + # Limit to max_samples + samples = sft_samples[:self.config.max_samples] + + # Weight samples by vasana if available + if samskara_signals: + samples = self._weight_samples(samples, samskara_signals) + + try: + result = self._run_lora_training(samples) + result.duration_seconds = time.time() - start_time + + # Record in history + self._training_history.append({ + "timestamp": time.time(), + "result": result.to_dict(), + }) + + # Persist training log + self._save_training_log() + + return result + except Exception as e: + logger.warning("Micro-training failed: %s", e) + return MicroTrainResult( + success=False, + duration_seconds=time.time() - start_time, + error=str(e), + ) + + def _run_lora_training(self, samples: List[Dict]) -> MicroTrainResult: + """Run LoRA fine-tuning with PyTorch. + + This is CPU-optimized: + - float32 (no mixed precision on CPU) + - Gradient checkpointing for memory efficiency + - Small LoRA rank (4) = tiny trainable parameter count + """ + import torch + from torch.utils.data import DataLoader, Dataset + + class TextDataset(Dataset): + def __init__(self, data: List[Dict]): + self.data = data + + def __len__(self): + return len(self.data) + + def __getitem__(self, idx): + item = self.data[idx] + return item.get("input", ""), item.get("output", "") + + # Check if we can load the model for training + # For GGUF models, we need llama-cpp-python for inference but + # can't fine-tune them directly. We look for a safetensors/HF model. + model, tokenizer = self._load_model_for_training() + if model is None: + # Fallback: save training data for later batch processing + return self._save_for_deferred_training(samples) + + dataset = TextDataset(samples) + loader = DataLoader( + dataset, + batch_size=self.config.micro_batch_size, + shuffle=True, + ) + + # Apply LoRA + model = self._apply_lora(model) + model.train() + + # Optimizer + trainable_params = [p for p in model.parameters() if p.requires_grad] + optimizer = torch.optim.AdamW( + trainable_params, + lr=self.config.learning_rate, + weight_decay=self.config.weight_decay, + ) + + # Training loop + total_steps = 0 + losses = [] + accum_loss = 0.0 + + for step_idx in range(self.config.max_steps): + for batch_inputs, batch_outputs in loader: + # Tokenize + combined = [ + f"{inp} {out}" for inp, out in zip(batch_inputs, batch_outputs) + ] + encodings = tokenizer( + combined, + return_tensors="pt", + max_length=self.config.max_seq_len, + truncation=True, + padding=True, + ) + + # Forward pass + outputs = model( + input_ids=encodings["input_ids"], + attention_mask=encodings["attention_mask"], + labels=encodings["input_ids"], + ) + loss = outputs.loss / self.config.gradient_accumulation_steps + loss.backward() + accum_loss += loss.item() + + total_steps += 1 + + if total_steps % self.config.gradient_accumulation_steps == 0: + torch.nn.utils.clip_grad_norm_(trainable_params, 1.0) + optimizer.step() + optimizer.zero_grad() + losses.append(accum_loss) + accum_loss = 0.0 + + if total_steps >= self.config.max_steps: + break + + if total_steps >= self.config.max_steps: + break + + # Save adapter + adapter_path = self._save_adapter(model) + + return MicroTrainResult( + success=True, + steps_completed=total_steps, + samples_used=len(samples), + loss_start=losses[0] if losses else 0.0, + loss_end=losses[-1] if losses else 0.0, + adapter_path=adapter_path, + ) + + def _load_model_for_training(self): + """Load model + tokenizer for training. + + Returns (model, tokenizer) or (None, None) if not available. + """ + if not self._model_path: + return None, None + + # GGUF files can't be fine-tuned directly + if self._model_path.endswith(".gguf"): + # Check for a companion safetensors model + base_dir = os.path.dirname(self._model_path) + safetensors_path = os.path.join(base_dir, "model.safetensors") + config_path = os.path.join(base_dir, "config.json") + if not os.path.exists(config_path): + logger.info( + "GGUF model cannot be fine-tuned directly. " + "Saving training data for deferred processing." + ) + return None, None + model_dir = base_dir + else: + model_dir = self._model_path + + try: + from transformers import AutoModelForCausalLM, AutoTokenizer + tokenizer = AutoTokenizer.from_pretrained( + model_dir, trust_remote_code=True, + ) + if tokenizer.pad_token is None: + tokenizer.pad_token = tokenizer.eos_token + + model = AutoModelForCausalLM.from_pretrained( + model_dir, + torch_dtype="auto", + trust_remote_code=True, + ) + return model, tokenizer + except Exception as e: + logger.info("Model loading failed: %s", e) + return None, None + + def _apply_lora(self, model): + """Apply LoRA adapters to the model.""" + try: + from peft import LoraConfig, get_peft_model, TaskType + lora_config = LoraConfig( + task_type=TaskType.CAUSAL_LM, + r=self.config.lora_rank, + lora_alpha=self.config.lora_alpha, + lora_dropout=0.0, # No dropout for micro-training + target_modules=["q_proj", "v_proj"], # Minimal target + ) + model = get_peft_model(model, lora_config) + trainable = sum(p.numel() for p in model.parameters() if p.requires_grad) + total = sum(p.numel() for p in model.parameters()) + logger.info( + "LoRA applied: %d trainable / %d total params (%.2f%%)", + trainable, total, 100 * trainable / total, + ) + return model + except ImportError: + logger.warning( + "peft not available — training all parameters (not recommended " + "for edge). Install: pip install peft" + ) + return model + + def _save_adapter(self, model) -> str: + """Save the LoRA adapter to disk.""" + os.makedirs(self._adapter_dir, exist_ok=True) + adapter_name = f"adapter_{int(time.time())}" + adapter_path = os.path.join(self._adapter_dir, adapter_name) + + try: + if hasattr(model, "save_pretrained"): + model.save_pretrained(adapter_path) + else: + # Fallback: save state dict + import torch + torch.save( + {k: v for k, v in model.state_dict().items() if "lora" in k}, + os.path.join(adapter_path, "lora_weights.pt"), + ) + except Exception as e: + logger.warning("Adapter save failed: %s", e) + adapter_path = "" + + return adapter_path + + def _save_for_deferred_training( + self, samples: List[Dict], + ) -> MicroTrainResult: + """Save training data to disk for later batch processing. + + Used when the model format doesn't support direct fine-tuning + (e.g., GGUF without a companion HF model). + """ + os.makedirs(self._adapter_dir, exist_ok=True) + deferred_path = os.path.join( + self._adapter_dir, f"deferred_{int(time.time())}.jsonl", + ) + + with open(deferred_path, "w", encoding="utf-8") as f: + for sample in samples: + f.write(json.dumps(sample, ensure_ascii=False) + "\n") + + return MicroTrainResult( + success=True, + steps_completed=0, + samples_used=len(samples), + adapter_path=deferred_path, + error="deferred: saved training data for batch processing", + ) + + def _weight_samples( + self, + samples: List[Dict], + samskara_signals: Dict[str, Any], + ) -> List[Dict]: + """Weight training samples based on vasana degradation signals. + + Samples from degrading dimensions get 2x representation. + """ + degrading = set(samskara_signals.get("degrading_dimensions", [])) + if not degrading: + return samples + + weighted = [] + for sample in samples: + weighted.append(sample) + sample_type = sample.get("type", "") + if sample_type in degrading: + weighted.append(sample) # Duplicate for emphasis + + return weighted + + def _save_training_log(self) -> None: + """Persist training history to disk.""" + os.makedirs(self._adapter_dir, exist_ok=True) + log_path = os.path.join(self._adapter_dir, "training_log.jsonl") + try: + with open(log_path, "a", encoding="utf-8") as f: + entry = self._training_history[-1] + f.write(json.dumps(entry, ensure_ascii=False) + "\n") + except OSError: + pass + + # ------------------------------------------------------------------ + # Integration with DheeEdge + # ------------------------------------------------------------------ + + def train_from_edge( + self, + edge_plugin: Any, + samskara: Optional[Any] = None, + ) -> MicroTrainResult: + """Convenience: collect training data from a DheeEdge instance and train. + + Gathers: + - Action outcome pairs as SFT samples + - Samskara signals for weighting + """ + # Collect action-based SFT samples + sft_samples = [] + action_history = getattr(edge_plugin, "_action_history", []) + for record in action_history[-self.config.max_samples:]: + action = record.get("action", "") + success = record.get("success", False) + env = record.get("env_state", {}) + + env_desc = ", ".join(f"{k}={v}" for k, v in list(env.items())[:5]) if env else "unknown" + sft_samples.append({ + "input": f"[ACTION] state: {env_desc}, action: {action}", + "output": f"{'success' if success else 'failure'}", + "type": "action_prediction", + }) + + # Add samskara training data if available + samskara_data = {} + if samskara: + try: + samskara_data = samskara.get_training_data() + sft_samples.extend(samskara_data.get("sft_samples", [])) + except Exception: + pass + + if not sft_samples: + return MicroTrainResult( + success=False, error="No training data from edge", + ) + + return self.micro_train( + training_data={"sft_samples": sft_samples}, + samskara_signals=samskara_data, + ) + + def get_status(self) -> Dict[str, Any]: + """Get trainer status.""" + return { + "can_train": self.can_train, + "model_path": self._model_path, + "adapter_dir": self._adapter_dir, + "training_cycles": len(self._training_history), + "config": { + "lora_rank": self.config.lora_rank, + "max_steps": self.config.max_steps, + "learning_rate": self.config.learning_rate, + }, + } diff --git a/dhee/hive/__init__.py b/dhee/hive/__init__.py new file mode 100644 index 0000000..a35f837 --- /dev/null +++ b/dhee/hive/__init__.py @@ -0,0 +1,9 @@ +"""Dhee Hive — multi-agent shared cognition layer. + +Built on top of engram-bus for real-time agent-to-agent communication, +with CRDT-based sync for offline/edge scenarios. +""" + +from dhee.hive.hive_memory import HiveMemory + +__all__ = ["HiveMemory"] diff --git a/dhee/hive/hive_memory.py b/dhee/hive/hive_memory.py new file mode 100644 index 0000000..26453e2 --- /dev/null +++ b/dhee/hive/hive_memory.py @@ -0,0 +1,526 @@ +"""HiveMemory — multi-agent shared cognition on top of engram-bus. + +Enables multiple DheePlugin instances (across agents, processes, or machines) +to share and evolve a collective knowledge base: + + - **Shared Insights**: Cross-agent discoveries and patterns. + - **Shared Heuristics**: Abstract reasoning rules mined from any agent's trajectories. + - **Shared Skills**: Proven skills that any agent can adopt. + - **Collective Signals**: Aggregated samskara-like signals for hive-level evolution. + +Architecture: + Each agent runs a local DheePlugin. HiveMemory sits alongside it and + periodically publishes local discoveries to the bus, and subscribes to + discoveries from other agents. A quality gate ensures only validated + knowledge propagates. + + ┌──────────┐ bus.publish() ┌─────────────┐ + │ Agent A │ ────────────────────▶ │ engram-bus │ + │ DheePlugin│ ◀──────────────────── │ (pub/sub + │ + │ + Hive │ bus.subscribe() │ KV store) │ + └──────────┘ └──────┬──────┘ + │ + ┌──────────┐ │ + │ Agent B │ ◀───────────────────────────┘ + │ DheePlugin│ + │ + Hive │ + └──────────┘ +""" + +from __future__ import annotations + +import json +import logging +import time +import threading +from dataclasses import asdict, dataclass, field +from datetime import datetime, timezone +from typing import Any, Callable, Dict, List, Optional, Set + +logger = logging.getLogger(__name__) + + +def _now_iso() -> str: + return datetime.now(timezone.utc).isoformat() + + +# --------------------------------------------------------------------------- +# Shared knowledge item types +# --------------------------------------------------------------------------- + +@dataclass +class SharedItem: + """A piece of knowledge shared on the hive.""" + + id: str + kind: str # "insight" | "heuristic" | "skill" | "signal" + content: Dict[str, Any] + source_agent: str + timestamp: str = field(default_factory=_now_iso) + confidence: float = 0.5 + votes_up: int = 0 + votes_down: int = 0 + adopted_by: List[str] = field(default_factory=list) + + def to_dict(self) -> Dict[str, Any]: + return { + "id": self.id, + "kind": self.kind, + "content": self.content, + "source_agent": self.source_agent, + "timestamp": self.timestamp, + "confidence": self.confidence, + "votes_up": self.votes_up, + "votes_down": self.votes_down, + "adopted_by": self.adopted_by, + } + + @classmethod + def from_dict(cls, data: Dict[str, Any]) -> "SharedItem": + return cls( + id=data["id"], + kind=data["kind"], + content=data.get("content", {}), + source_agent=data.get("source_agent", "unknown"), + timestamp=data.get("timestamp", _now_iso()), + confidence=data.get("confidence", 0.5), + votes_up=data.get("votes_up", 0), + votes_down=data.get("votes_down", 0), + adopted_by=data.get("adopted_by", []), + ) + + @property + def quality_score(self) -> float: + """Wilson score lower bound — conservative estimate of true quality.""" + n = self.votes_up + self.votes_down + if n == 0: + return self.confidence + p = self.votes_up / n + # Wilson score interval (simplified) + z = 1.96 # 95% confidence + denominator = 1 + z * z / n + centre = p + z * z / (2 * n) + spread = z * ((p * (1 - p) + z * z / (4 * n)) / n) ** 0.5 + return (centre - spread) / denominator + + +# --------------------------------------------------------------------------- +# Topics +# --------------------------------------------------------------------------- + +_TOPIC_SHARE = "dhee.hive.share" +_TOPIC_VOTE = "dhee.hive.vote" +_TOPIC_ADOPT = "dhee.hive.adopt" +_TOPIC_SYNC_REQUEST = "dhee.hive.sync.request" +_TOPIC_SYNC_RESPONSE = "dhee.hive.sync.response" +_NS_HIVE = "dhee_hive" + + +# --------------------------------------------------------------------------- +# HiveMemory +# --------------------------------------------------------------------------- + +class HiveMemory: + """Multi-agent shared cognition layer. + + Wraps an engram-bus instance to provide structured knowledge sharing + with quality gating and adoption tracking. + """ + + def __init__( + self, + agent_id: str, + bus: Any = None, + min_confidence_to_share: float = 0.4, + min_quality_to_adopt: float = 0.3, + auto_subscribe: bool = True, + ): + """ + Args: + agent_id: This agent's identifier on the hive. + bus: An engram_bus.Bus instance. If None, creates an in-memory bus. + min_confidence_to_share: Minimum confidence to publish to hive. + min_quality_to_adopt: Minimum quality_score to auto-adopt shared items. + auto_subscribe: Whether to subscribe to hive topics on init. + """ + self.agent_id = agent_id + self._min_share = min_confidence_to_share + self._min_adopt = min_quality_to_adopt + + # Local store of hive items (id -> SharedItem) + self._items: Dict[str, SharedItem] = {} + self._lock = threading.RLock() + self._on_receive_callbacks: List[Callable] = [] + + # Bus connection + if bus is None: + try: + from engram_bus import Bus + bus = Bus() + except ImportError: + logger.warning("engram-bus not available, hive runs in local-only mode") + bus = None + + self._bus = bus + + if bus and auto_subscribe: + self._subscribe() + + def _subscribe(self) -> None: + """Subscribe to hive topics on the bus.""" + if not self._bus: + return + self._bus.subscribe(_TOPIC_SHARE, self._on_share_received, agent=self.agent_id) + self._bus.subscribe(_TOPIC_VOTE, self._on_vote_received, agent=self.agent_id) + self._bus.subscribe(_TOPIC_ADOPT, self._on_adopt_received, agent=self.agent_id) + self._bus.subscribe( + _TOPIC_SYNC_REQUEST, self._on_sync_request, agent=self.agent_id, + ) + self._bus.register(self.agent_id, metadata={"type": "dhee_hive_member"}) + + # ------------------------------------------------------------------ + # Publishing + # ------------------------------------------------------------------ + + def share_insight( + self, + insight_id: str, + content: Dict[str, Any], + confidence: float = 0.5, + ) -> Optional[SharedItem]: + """Share an insight (from Buddhi reflect) with the hive.""" + return self._publish_item( + item_id=f"insight:{self.agent_id}:{insight_id}", + kind="insight", + content=content, + confidence=confidence, + ) + + def share_heuristic( + self, + heuristic_id: str, + content: Dict[str, Any], + confidence: float = 0.5, + ) -> Optional[SharedItem]: + """Share a distilled heuristic with the hive.""" + return self._publish_item( + item_id=f"heuristic:{self.agent_id}:{heuristic_id}", + kind="heuristic", + content=content, + confidence=confidence, + ) + + def share_skill( + self, + skill_id: str, + content: Dict[str, Any], + confidence: float = 0.5, + ) -> Optional[SharedItem]: + """Share a proven skill with the hive.""" + return self._publish_item( + item_id=f"skill:{self.agent_id}:{skill_id}", + kind="skill", + content=content, + confidence=confidence, + ) + + def share_signal( + self, + signal_type: str, + data: Dict[str, Any], + ) -> Optional[SharedItem]: + """Share an aggregated signal (e.g., vasana shift) with the hive.""" + return self._publish_item( + item_id=f"signal:{self.agent_id}:{signal_type}:{int(time.time())}", + kind="signal", + content={"signal_type": signal_type, **data}, + confidence=0.5, + ) + + def _publish_item( + self, + item_id: str, + kind: str, + content: Dict[str, Any], + confidence: float, + ) -> Optional[SharedItem]: + if confidence < self._min_share: + logger.debug( + "Not sharing %s (confidence %.2f < %.2f)", + item_id, confidence, self._min_share, + ) + return None + + item = SharedItem( + id=item_id, + kind=kind, + content=content, + source_agent=self.agent_id, + confidence=confidence, + ) + + with self._lock: + self._items[item.id] = item + + if self._bus: + self._bus.publish(_TOPIC_SHARE, item.to_dict(), agent=self.agent_id) + # Also store in bus KV for late joiners + self._bus.put( + f"hive:{item.id}", + json.dumps(item.to_dict()), + agent=self.agent_id, + namespace=_NS_HIVE, + ) + + return item + + # ------------------------------------------------------------------ + # Voting + # ------------------------------------------------------------------ + + def vote(self, item_id: str, upvote: bool = True) -> None: + """Vote on a shared item's quality.""" + with self._lock: + item = self._items.get(item_id) + if item: + if upvote: + item.votes_up += 1 + else: + item.votes_down += 1 + + if self._bus: + self._bus.publish( + _TOPIC_VOTE, + {"item_id": item_id, "upvote": upvote, "voter": self.agent_id}, + agent=self.agent_id, + ) + + def adopt(self, item_id: str) -> Optional[SharedItem]: + """Mark a shared item as adopted by this agent.""" + with self._lock: + item = self._items.get(item_id) + if not item: + return None + if self.agent_id not in item.adopted_by: + item.adopted_by.append(self.agent_id) + + if self._bus: + self._bus.publish( + _TOPIC_ADOPT, + {"item_id": item_id, "adopter": self.agent_id}, + agent=self.agent_id, + ) + return item + + # ------------------------------------------------------------------ + # Querying + # ------------------------------------------------------------------ + + def get_shared( + self, + kind: Optional[str] = None, + min_quality: Optional[float] = None, + limit: int = 20, + ) -> List[SharedItem]: + """Get shared items, optionally filtered by kind and quality.""" + min_q = min_quality if min_quality is not None else 0.0 + + with self._lock: + items = list(self._items.values()) + + if kind: + items = [i for i in items if i.kind == kind] + items = [i for i in items if i.quality_score >= min_q] + items.sort(key=lambda i: i.quality_score, reverse=True) + return items[:limit] + + def get_adoptable(self, limit: int = 10) -> List[SharedItem]: + """Get high-quality items not yet adopted by this agent.""" + with self._lock: + candidates = [ + item for item in self._items.values() + if self.agent_id not in item.adopted_by + and item.source_agent != self.agent_id + and item.quality_score >= self._min_adopt + ] + candidates.sort(key=lambda i: i.quality_score, reverse=True) + return candidates[:limit] + + def get_hive_stats(self) -> Dict[str, Any]: + """Get statistics about the hive.""" + with self._lock: + items = list(self._items.values()) + + by_kind: Dict[str, int] = {} + by_agent: Dict[str, int] = {} + total_votes = 0 + + for item in items: + by_kind[item.kind] = by_kind.get(item.kind, 0) + 1 + by_agent[item.source_agent] = by_agent.get(item.source_agent, 0) + 1 + total_votes += item.votes_up + item.votes_down + + return { + "total_items": len(items), + "by_kind": by_kind, + "by_agent": by_agent, + "total_votes": total_votes, + "avg_quality": ( + sum(i.quality_score for i in items) / len(items) + if items else 0.0 + ), + } + + # ------------------------------------------------------------------ + # Bus callbacks + # ------------------------------------------------------------------ + + def _on_share_received( + self, topic: str, data: Any, sender_agent: Optional[str], + ) -> None: + """Handle incoming shared item from another agent.""" + if sender_agent == self.agent_id: + return # Ignore own messages + + try: + item = SharedItem.from_dict(data) + except (TypeError, KeyError) as e: + logger.debug("Invalid shared item: %s", e) + return + + with self._lock: + if item.id not in self._items: + self._items[item.id] = item + + for cb in self._on_receive_callbacks: + try: + cb(item) + except Exception as e: + logger.debug("Hive receive callback error: %s", e) + + def _on_vote_received( + self, topic: str, data: Any, sender_agent: Optional[str], + ) -> None: + """Handle incoming vote from another agent.""" + if sender_agent == self.agent_id: + return + + item_id = data.get("item_id") + upvote = data.get("upvote", True) + + with self._lock: + item = self._items.get(item_id) + if item: + if upvote: + item.votes_up += 1 + else: + item.votes_down += 1 + + def _on_adopt_received( + self, topic: str, data: Any, sender_agent: Optional[str], + ) -> None: + """Handle adoption notification from another agent.""" + item_id = data.get("item_id") + adopter = data.get("adopter") + + with self._lock: + item = self._items.get(item_id) + if item and adopter and adopter not in item.adopted_by: + item.adopted_by.append(adopter) + + def _on_sync_request( + self, topic: str, data: Any, sender_agent: Optional[str], + ) -> None: + """Respond to sync request from another agent (e.g., edge coming online).""" + if sender_agent == self.agent_id or not self._bus: + return + + # Send all our items as a sync response + with self._lock: + payload = {item_id: item.to_dict() for item_id, item in self._items.items()} + + self._bus.publish( + _TOPIC_SYNC_RESPONSE, + {"items": payload, "responder": self.agent_id}, + agent=self.agent_id, + ) + + # ------------------------------------------------------------------ + # Sync (pull-based) + # ------------------------------------------------------------------ + + def request_sync(self) -> None: + """Request a full sync from other hive members (e.g., after coming online).""" + if not self._bus: + return + self._bus.subscribe( + _TOPIC_SYNC_RESPONSE, self._on_sync_response, agent=self.agent_id, + ) + self._bus.publish( + _TOPIC_SYNC_REQUEST, + {"requester": self.agent_id}, + agent=self.agent_id, + ) + + def _on_sync_response( + self, topic: str, data: Any, sender_agent: Optional[str], + ) -> None: + """Handle sync response — merge received items.""" + items_data = data.get("items", {}) + with self._lock: + for item_id, item_dict in items_data.items(): + if item_id not in self._items: + try: + self._items[item_id] = SharedItem.from_dict(item_dict) + except (TypeError, KeyError): + pass + + # ------------------------------------------------------------------ + # Callbacks + # ------------------------------------------------------------------ + + def on_receive(self, callback: Callable[[SharedItem], None]) -> None: + """Register a callback for when new items arrive from the hive.""" + self._on_receive_callbacks.append(callback) + + # ------------------------------------------------------------------ + # Export for DheePlugin integration + # ------------------------------------------------------------------ + + def get_context_block(self, limit: int = 5) -> Dict[str, Any]: + """Get hive knowledge formatted for HyperContext injection.""" + insights = self.get_shared(kind="insight", limit=limit) + heuristics = self.get_shared(kind="heuristic", limit=limit) + skills = self.get_shared(kind="skill", limit=limit) + + return { + "hive_insights": [ + { + "source": i.source_agent, + "content": i.content, + "quality": round(i.quality_score, 2), + } + for i in insights + ], + "hive_heuristics": [ + { + "source": h.source_agent, + "content": h.content, + "quality": round(h.quality_score, 2), + } + for h in heuristics + ], + "hive_skills": [ + { + "source": s.source_agent, + "name": s.content.get("name", s.id), + "quality": round(s.quality_score, 2), + } + for s in skills + ], + } + + def close(self) -> None: + """Unsubscribe and clean up.""" + # Bus cleanup is handled by the bus owner + self._on_receive_callbacks.clear() diff --git a/dhee/hive/sync.py b/dhee/hive/sync.py new file mode 100644 index 0000000..1a0e546 --- /dev/null +++ b/dhee/hive/sync.py @@ -0,0 +1,436 @@ +"""CRDT-based sync protocol for offline/edge Dhee nodes. + +When a DheeEdge instance operates offline (e.g., a humanoid robot in a +warehouse with no connectivity), it accumulates local hive items. On +reconnection, it needs to merge with the central hive without conflicts. + +This module implements: + 1. **LWW-Register** (Last-Writer-Wins) for individual shared items. + 2. **G-Counter** for vote counts (grow-only, merge = max per node). + 3. **OR-Set** (Observed-Remove) for adoption lists. + 4. **SyncEnvelope** — wire format for shipping CRDT state between nodes. + +Usage: + # On edge device: + state = CRDTState(node_id="edge-1") + state.set_item(shared_item) + state.increment_votes_up("item:123") + envelope = state.export_envelope() + + # Ship envelope (HTTP, BLE, serial, file drop — whatever works) + + # On hub: + hub_state = CRDTState(node_id="hub") + hub_state.merge(envelope) +""" + +from __future__ import annotations + +import json +import logging +import os +import time +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional, Set, Tuple + +logger = logging.getLogger(__name__) + + +def _hlc_now(node_id: str) -> str: + """Hybrid Logical Clock timestamp: |. + + Provides a globally-unique, monotonically-increasing timestamp even + if wall clocks disagree between nodes. Uses '|' separator so node_ids + can contain dashes. + """ + return f"{int(time.time() * 1000)}|{node_id}" + + +def _hlc_compare(a: str, b: str) -> int: + """Compare two HLC timestamps. Returns -1, 0, or 1.""" + a_ms, a_node = a.split("|", 1) + b_ms, b_node = b.split("|", 1) + a_int, b_int = int(a_ms), int(b_ms) + if a_int != b_int: + return -1 if a_int < b_int else 1 + if a_node < b_node: + return -1 + if a_node > b_node: + return 1 + return 0 + + +# --------------------------------------------------------------------------- +# LWW-Register: per-item state +# --------------------------------------------------------------------------- + +@dataclass +class LWWRegister: + """Last-Writer-Wins Register for a shared item's content.""" + + value: Dict[str, Any] + timestamp: str # HLC timestamp + node_id: str + + def merge(self, other: "LWWRegister") -> "LWWRegister": + """Merge two registers — latest timestamp wins.""" + if _hlc_compare(self.timestamp, other.timestamp) >= 0: + return self + return other + + def to_dict(self) -> Dict[str, Any]: + return { + "value": self.value, + "timestamp": self.timestamp, + "node_id": self.node_id, + } + + @classmethod + def from_dict(cls, data: Dict[str, Any]) -> "LWWRegister": + return cls( + value=data["value"], + timestamp=data["timestamp"], + node_id=data["node_id"], + ) + + +# --------------------------------------------------------------------------- +# G-Counter: grow-only counter per node +# --------------------------------------------------------------------------- + +@dataclass +class GCounter: + """Grow-only counter — each node has its own monotonic count.""" + + counts: Dict[str, int] = field(default_factory=dict) # node_id -> count + + @property + def value(self) -> int: + return sum(self.counts.values()) + + def increment(self, node_id: str, amount: int = 1) -> None: + self.counts[node_id] = self.counts.get(node_id, 0) + amount + + def merge(self, other: "GCounter") -> "GCounter": + """Merge = max of each node's count.""" + all_nodes = set(self.counts) | set(other.counts) + merged = GCounter() + for node in all_nodes: + merged.counts[node] = max( + self.counts.get(node, 0), + other.counts.get(node, 0), + ) + return merged + + def to_dict(self) -> Dict[str, int]: + return dict(self.counts) + + @classmethod + def from_dict(cls, data: Dict[str, int]) -> "GCounter": + return cls(counts=dict(data)) + + +# --------------------------------------------------------------------------- +# OR-Set: Observed-Remove Set (for adoption lists) +# --------------------------------------------------------------------------- + +@dataclass +class ORSet: + """Observed-Remove Set — supports both add and remove with convergence. + + Each element is tagged with a unique (node_id, seq) pair. Removes + only remove the tags that were observed, so concurrent adds win. + """ + + # element -> set of (node_id, seq) tags + _elements: Dict[str, Set[Tuple[str, int]]] = field(default_factory=lambda: {}) + _tombstones: Set[Tuple[str, int]] = field(default_factory=set) + _seq: int = 0 + + def add(self, element: str, node_id: str) -> None: + self._seq += 1 + tag = (node_id, self._seq) + if element not in self._elements: + self._elements[element] = set() + self._elements[element].add(tag) + + def remove(self, element: str) -> None: + tags = self._elements.pop(element, set()) + self._tombstones.update(tags) + + @property + def elements(self) -> Set[str]: + return { + elem for elem, tags in self._elements.items() + if tags - self._tombstones + } + + def merge(self, other: "ORSet") -> "ORSet": + """Merge two OR-Sets.""" + merged = ORSet() + merged._seq = max(self._seq, other._seq) + merged._tombstones = self._tombstones | other._tombstones + + all_elements = set(self._elements) | set(other._elements) + for elem in all_elements: + tags_a = self._elements.get(elem, set()) + tags_b = other._elements.get(elem, set()) + # Union of live tags minus all tombstones + live_tags = (tags_a | tags_b) - merged._tombstones + if live_tags: + merged._elements[elem] = live_tags + + return merged + + def to_dict(self) -> Dict[str, Any]: + return { + "elements": { + elem: [list(t) for t in tags] + for elem, tags in self._elements.items() + }, + "tombstones": [list(t) for t in self._tombstones], + "seq": self._seq, + } + + @classmethod + def from_dict(cls, data: Dict[str, Any]) -> "ORSet": + s = cls() + s._seq = data.get("seq", 0) + s._tombstones = {tuple(t) for t in data.get("tombstones", [])} + s._elements = { + elem: {tuple(t) for t in tags} + for elem, tags in data.get("elements", {}).items() + } + return s + + +# --------------------------------------------------------------------------- +# Per-item CRDT state +# --------------------------------------------------------------------------- + +@dataclass +class ItemCRDT: + """CRDT state for a single shared hive item.""" + + item_id: str + content: LWWRegister # The item payload + votes_up: GCounter = field(default_factory=GCounter) + votes_down: GCounter = field(default_factory=GCounter) + adopted_by: ORSet = field(default_factory=ORSet) + + def merge(self, other: "ItemCRDT") -> "ItemCRDT": + assert self.item_id == other.item_id + return ItemCRDT( + item_id=self.item_id, + content=self.content.merge(other.content), + votes_up=self.votes_up.merge(other.votes_up), + votes_down=self.votes_down.merge(other.votes_down), + adopted_by=self.adopted_by.merge(other.adopted_by), + ) + + def to_dict(self) -> Dict[str, Any]: + return { + "item_id": self.item_id, + "content": self.content.to_dict(), + "votes_up": self.votes_up.to_dict(), + "votes_down": self.votes_down.to_dict(), + "adopted_by": self.adopted_by.to_dict(), + } + + @classmethod + def from_dict(cls, data: Dict[str, Any]) -> "ItemCRDT": + return cls( + item_id=data["item_id"], + content=LWWRegister.from_dict(data["content"]), + votes_up=GCounter.from_dict(data.get("votes_up", {})), + votes_down=GCounter.from_dict(data.get("votes_down", {})), + adopted_by=ORSet.from_dict(data.get("adopted_by", {})), + ) + + +# --------------------------------------------------------------------------- +# SyncEnvelope — wire format +# --------------------------------------------------------------------------- + +@dataclass +class SyncEnvelope: + """Wire format for CRDT state exchange between nodes.""" + + source_node: str + timestamp: str # HLC + items: Dict[str, Dict[str, Any]] # item_id -> ItemCRDT.to_dict() + + def to_bytes(self) -> bytes: + return json.dumps({ + "source_node": self.source_node, + "timestamp": self.timestamp, + "items": self.items, + }, ensure_ascii=False).encode("utf-8") + + @classmethod + def from_bytes(cls, data: bytes) -> "SyncEnvelope": + d = json.loads(data.decode("utf-8")) + return cls( + source_node=d["source_node"], + timestamp=d["timestamp"], + items=d.get("items", {}), + ) + + def to_dict(self) -> Dict[str, Any]: + return { + "source_node": self.source_node, + "timestamp": self.timestamp, + "items": self.items, + } + + @classmethod + def from_dict(cls, data: Dict[str, Any]) -> "SyncEnvelope": + return cls( + source_node=data["source_node"], + timestamp=data["timestamp"], + items=data.get("items", {}), + ) + + +# --------------------------------------------------------------------------- +# CRDTState — per-node state manager +# --------------------------------------------------------------------------- + +class CRDTState: + """Manages CRDT state for a single node. + + Each node maintains its own CRDTState. On sync, nodes exchange + SyncEnvelopes and merge them — convergence is guaranteed by the + CRDT merge semantics (commutative, associative, idempotent). + """ + + def __init__(self, node_id: str, persist_path: Optional[str] = None): + self.node_id = node_id + self._items: Dict[str, ItemCRDT] = {} + self._persist_path = persist_path + if persist_path: + self._load() + + def set_item(self, item_id: str, content: Dict[str, Any]) -> None: + """Set or update an item's content (LWW).""" + ts = _hlc_now(self.node_id) + register = LWWRegister(value=content, timestamp=ts, node_id=self.node_id) + + if item_id in self._items: + self._items[item_id].content = self._items[item_id].content.merge(register) + else: + self._items[item_id] = ItemCRDT(item_id=item_id, content=register) + + self._auto_persist() + + def increment_votes_up(self, item_id: str, amount: int = 1) -> None: + if item_id in self._items: + self._items[item_id].votes_up.increment(self.node_id, amount) + self._auto_persist() + + def increment_votes_down(self, item_id: str, amount: int = 1) -> None: + if item_id in self._items: + self._items[item_id].votes_down.increment(self.node_id, amount) + self._auto_persist() + + def add_adopter(self, item_id: str, adopter: str) -> None: + if item_id in self._items: + self._items[item_id].adopted_by.add(adopter, self.node_id) + self._auto_persist() + + def export_envelope(self) -> SyncEnvelope: + """Export current state as a sync envelope.""" + return SyncEnvelope( + source_node=self.node_id, + timestamp=_hlc_now(self.node_id), + items={ + item_id: crdt.to_dict() + for item_id, crdt in self._items.items() + }, + ) + + def merge(self, envelope: SyncEnvelope) -> int: + """Merge a received envelope into local state. + + Returns number of items updated. + """ + updated = 0 + for item_id, item_dict in envelope.items.items(): + try: + remote = ItemCRDT.from_dict(item_dict) + except (TypeError, KeyError) as e: + logger.debug("Skipping malformed item %s: %s", item_id, e) + continue + + if item_id in self._items: + merged = self._items[item_id].merge(remote) + # Check if anything actually changed + if merged.to_dict() != self._items[item_id].to_dict(): + self._items[item_id] = merged + updated += 1 + else: + self._items[item_id] = remote + updated += 1 + + if updated: + self._auto_persist() + return updated + + def get_item(self, item_id: str) -> Optional[Dict[str, Any]]: + """Get resolved state of an item (content + vote totals + adopters).""" + crdt = self._items.get(item_id) + if not crdt: + return None + return { + "item_id": item_id, + "content": crdt.content.value, + "votes_up": crdt.votes_up.value, + "votes_down": crdt.votes_down.value, + "adopted_by": sorted(crdt.adopted_by.elements), + "last_updated": crdt.content.timestamp, + } + + def list_items(self) -> List[Dict[str, Any]]: + """List all items with resolved state.""" + return [ + self.get_item(item_id) + for item_id in sorted(self._items) + ] + + @property + def item_count(self) -> int: + return len(self._items) + + # ── Persistence ── + + def _auto_persist(self) -> None: + if self._persist_path: + self.save() + + def save(self, path: Optional[str] = None) -> None: + path = path or self._persist_path + if not path: + return + os.makedirs(os.path.dirname(path) or ".", exist_ok=True) + tmp = path + ".tmp" + data = { + "node_id": self.node_id, + "items": { + item_id: crdt.to_dict() + for item_id, crdt in self._items.items() + }, + } + with open(tmp, "w", encoding="utf-8") as f: + json.dump(data, f, ensure_ascii=False) + os.replace(tmp, path) + + def _load(self) -> None: + if not self._persist_path or not os.path.exists(self._persist_path): + return + try: + with open(self._persist_path, "r", encoding="utf-8") as f: + data = json.load(f) + for item_id, item_dict in data.get("items", {}).items(): + self._items[item_id] = ItemCRDT.from_dict(item_dict) + except (OSError, json.JSONDecodeError, KeyError) as e: + logger.warning("Failed to load CRDT state: %s", e) diff --git a/dhee/llms/dhee.py b/dhee/llms/dhee.py index 3b7e533..a7b5cc9 100644 --- a/dhee/llms/dhee.py +++ b/dhee/llms/dhee.py @@ -249,6 +249,9 @@ def generate_with_task(self, task: str, content: str) -> str: - [DECOMPOSE]: complex question -> sub-questions - [CONTEXT]: text -> ContextAnchor - [SCENE]: text -> SceneSnapshot + - [MEMORY_OP]: context -> optimal memory operation + - [HEURISTIC]: trajectory summary -> abstract reasoning pattern + - [RETRIEVAL_JUDGE]: query + results -> sufficiency score 0.0-1.0 """ prompt = f"[{task.upper()}]\n{content}" return self.generate(prompt) @@ -281,6 +284,24 @@ def extract_scene(self, text: str) -> str: """[SCENE] task: extract scene snapshot.""" return self.generate_with_task("SCENE", text) + # --- BuddhiMini task heads (added for self-evolution) --- + + def classify_memory_op(self, context: str) -> str: + """[MEMORY_OP] task: predict optimal memory operation for context. + + Returns: store | retrieve | update | summarize | discard | none + """ + return self.generate_with_task("MEMORY_OP", context) + + def generate_heuristic(self, trajectory_summary: str) -> str: + """[HEURISTIC] task: distill abstract reasoning pattern from trajectory.""" + return self.generate_with_task("HEURISTIC", trajectory_summary) + + def judge_retrieval(self, query: str, results_text: str) -> str: + """[RETRIEVAL_JUDGE] task: score retrieval sufficiency 0.0-1.0.""" + prompt = f"Query: {query}\nResults:\n{results_text}" + return self.generate_with_task("RETRIEVAL_JUDGE", prompt) + @property def backend(self) -> str: return self._backend diff --git a/dhee/mcp_slim.py b/dhee/mcp_slim.py index 142c3fe..65dc249 100644 --- a/dhee/mcp_slim.py +++ b/dhee/mcp_slim.py @@ -28,33 +28,24 @@ logger = logging.getLogger(__name__) # --------------------------------------------------------------------------- -# Lazy singletons +# Lazy singleton — DheePlugin wraps Engram + Buddhi # --------------------------------------------------------------------------- -_memory = None -_buddhi = None +_plugin = None -def _get_memory(): - """Create memory instance with deferred enrichment (0 LLM on hot path).""" - global _memory - if _memory is None: - from dhee.mcp_server import get_memory_instance - _memory = get_memory_instance() - # Enable deferred enrichment: 0 LLM calls at ingestion, - # batch-enrich later at checkpoint time for retrieval quality. - if hasattr(_memory, "config") and hasattr(_memory.config, "enrichment"): - _memory.config.enrichment.defer_enrichment = True - _memory.config.enrichment.enable_unified = True - return _memory - - -def _get_buddhi(): - global _buddhi - if _buddhi is None: - from dhee.core.buddhi import Buddhi - _buddhi = Buddhi() - return _buddhi +def _get_plugin(): + """Create the DheePlugin singleton. Wraps Engram + Buddhi.""" + global _plugin + if _plugin is None: + from dhee.adapters.base import DheePlugin + _plugin = DheePlugin() + # Enable deferred enrichment on the underlying memory + memory = _plugin._engram._memory + if hasattr(memory, "config") and hasattr(memory.config, "enrichment"): + memory.config.enrichment.defer_enrichment = True + memory.config.enrichment.enable_unified = True + return _plugin # --------------------------------------------------------------------------- @@ -224,56 +215,32 @@ def _get_buddhi(): # --------------------------------------------------------------------------- def _handle_remember(args: Dict[str, Any]) -> Dict[str, Any]: - """Store a memory. 0 LLM calls on hot path, 1 embed. Enrichment deferred.""" - memory = _get_memory() + """Store a memory. Delegates to DheePlugin.remember().""" content = args.get("content", "") if not content: return {"error": "content is required"} - - user_id = args.get("user_id", "default") - - # infer=False: agent explicitly stated the fact, no need to re-extract. - # defer_enrichment (set in _get_memory): echo/keywords added at checkpoint. - result = memory.add( - messages=content, - user_id=user_id, - agent_id="agent", - source_app="dhee-mcp", - infer=False, + return _get_plugin().remember( + content=content, + user_id=args.get("user_id", "default"), ) - # Buddhi: detect intentions in the content - buddhi = _get_buddhi() - intention = buddhi.on_memory_stored(content=content, user_id=user_id) - - response: Dict[str, Any] = {"stored": True} - if isinstance(result, dict): - results = result.get("results", []) - if results: - response["id"] = results[0].get("id") - if intention: - response["detected_intention"] = intention.to_dict() - return response - def _handle_recall(args: Dict[str, Any]) -> Dict[str, Any]: """Search memory. 0 LLM calls, 1 embed.""" - memory = _get_memory() query = args.get("query", "") if not query: return {"error": "query is required"} + plugin = _get_plugin() user_id = args.get("user_id", "default") limit = min(max(1, int(args.get("limit", 5))), 20) - result = memory.search( - query=query, - user_id=user_id, - limit=limit, + # Use raw memory search to get proactive signals alongside results + raw_result = plugin._engram._memory.search( + query=query, user_id=user_id, limit=limit, ) - results = result.get("results", []) + results = raw_result.get("results", []) if isinstance(raw_result, dict) else [] - # Compact output — only what the agent needs memories = [ { "id": r.get("id"), @@ -286,7 +253,7 @@ def _handle_recall(args: Dict[str, Any]) -> Dict[str, Any]: response: Dict[str, Any] = {"memories": memories, "count": len(memories)} # Attach Buddhi proactive signals if any - buddhi_signals = result.get("buddhi") + buddhi_signals = raw_result.get("buddhi") if isinstance(raw_result, dict) else None if buddhi_signals: response["proactive"] = buddhi_signals @@ -294,103 +261,36 @@ def _handle_recall(args: Dict[str, Any]) -> Dict[str, Any]: def _handle_context(args: Dict[str, Any]) -> Dict[str, Any]: - """HyperAgent bootstrap. Buddhi-powered.""" - memory = _get_memory() - buddhi = _get_buddhi() - user_id = args.get("user_id", "default") - task_description = args.get("task_description") - - hyper_ctx = buddhi.get_hyper_context( - user_id=user_id, - task_description=task_description, - memory=memory, + """HyperAgent bootstrap. Delegates to DheePlugin.context().""" + return _get_plugin().context( + task_description=args.get("task_description"), + user_id=args.get("user_id", "default"), ) - return hyper_ctx.to_dict() def _handle_checkpoint(args: Dict[str, Any]) -> Dict[str, Any]: - """Session lifecycle — save digest + enrich + outcome + reflect + intention.""" + """Session lifecycle. Delegates to DheePlugin.checkpoint().""" summary = args.get("summary", "") if not summary: return {"error": "summary is required"} - user_id = args.get("user_id", "default") - agent_id = args.get("agent_id", "agent") - result: Dict[str, Any] = {} - - # 1. Save session digest (for handoff) - try: - from dhee.core.kernel import save_session_digest - digest = save_session_digest( - task_summary=summary, - agent_id=agent_id, - repo=args.get("repo"), - status=args.get("status", "paused"), - decisions_made=args.get("decisions"), - files_touched=args.get("files_touched"), - todos_remaining=args.get("todos"), - ) - result["session_saved"] = True - if isinstance(digest, dict): - result["session_id"] = digest.get("session_id") - except Exception as e: - logger.debug("Session save skipped: %s", e) - result["session_saved"] = False - - # 2. Batch-enrich deferred memories (1 LLM call per ~10 memories) - # This is where retrieval quality gets added — echo paraphrases, keywords, - # categories — all in one batched LLM call. Not on the hot path. - memory = _get_memory() - if hasattr(memory, "enrich_pending"): - try: - enrich_result = memory.enrich_pending( - user_id=user_id, batch_size=10, max_batches=5, - ) - enriched = enrich_result.get("enriched_count", 0) - if enriched > 0: - result["memories_enriched"] = enriched - except Exception as e: - logger.debug("Batch enrichment skipped: %s", e) - - buddhi = _get_buddhi() - - # 3. Record outcome (for performance tracking) - task_type = args.get("task_type") - outcome_score = args.get("outcome_score") - if task_type and outcome_score is not None: - score = max(0.0, min(1.0, float(outcome_score))) - insight = buddhi.record_outcome( - user_id=user_id, task_type=task_type, score=score, - ) - result["outcome_recorded"] = True - if insight: - result["auto_insight"] = insight.to_dict() - - # 4. Reflect (for insight synthesis) - what_worked = args.get("what_worked") - what_failed = args.get("what_failed") - key_decision = args.get("key_decision") - if any([what_worked, what_failed, key_decision]): - reflections = buddhi.reflect( - user_id=user_id, - task_type=task_type or "general", - what_worked=what_worked, - what_failed=what_failed, - key_decision=key_decision, - ) - result["insights_created"] = len(reflections) - - # 5. Store intention (for prospective memory) - remember_to = args.get("remember_to") - if remember_to: - intention = buddhi.store_intention( - user_id=user_id, - description=remember_to, - trigger_keywords=args.get("trigger_keywords"), - ) - result["intention_stored"] = intention.to_dict() - - return result + return _get_plugin().checkpoint( + summary=summary, + task_type=args.get("task_type"), + outcome_score=args.get("outcome_score"), + what_worked=args.get("what_worked"), + what_failed=args.get("what_failed"), + key_decision=args.get("key_decision"), + remember_to=args.get("remember_to"), + trigger_keywords=args.get("trigger_keywords"), + status=args.get("status", "paused"), + decisions=args.get("decisions"), + todos=args.get("todos"), + files_touched=args.get("files_touched"), + repo=args.get("repo"), + user_id=args.get("user_id", "default"), + agent_id=args.get("agent_id", "agent"), + ) HANDLERS = { diff --git a/dhee/mini/__init__.py b/dhee/mini/__init__.py new file mode 100644 index 0000000..df441ba --- /dev/null +++ b/dhee/mini/__init__.py @@ -0,0 +1,6 @@ +"""Dhee Mini — small trainable model for self-evolving cognition.""" + +from dhee.mini.buddhi_mini import BuddhiMini +from dhee.mini.trace_segmenter import TraceSegmenter, TrainingSpan, SpanType + +__all__ = ["BuddhiMini", "TraceSegmenter", "TrainingSpan", "SpanType"] diff --git a/dhee/mini/buddhi_mini.py b/dhee/mini/buddhi_mini.py new file mode 100644 index 0000000..8fa8cd7 --- /dev/null +++ b/dhee/mini/buddhi_mini.py @@ -0,0 +1,414 @@ +"""BuddhiMini — small trainable model for self-evolving cognition. + +NOT a separate model from DheeModel. This is DheeModel with 3 new task +heads + a trace-driven data pipeline that produces better training data. + +The self-evolution loop: + 1. Agent uses Dhee (remember/recall/context/checkpoint) + 2. Samskara collects 12 signal types per operation + 3. TraceSegmenter splits trajectories into [REASON]/[ACT]/[MEMORY_OP] + 4. When signals reach critical mass → Nididhyasana triggers + 5. ProgressiveTrainer runs: SFT → DPO → RL + 6. DheeModel updates weights (LoRA merge or GGUF export) + 7. Hot-swapped without restart + +Research basis: + - Structured Agent Distillation (arXiv:2505.13820): span-specific losses + - AgeMem (arXiv:2601.01885): memory ops as RL-optimized tool calls + - EvolveR (arXiv:2510.16079): offline distillation → online retrieval + +New task heads (added to DheeModel's existing 6): + [MEMORY_OP] — predict optimal memory operation for context + [HEURISTIC] — generate abstract heuristic from trajectory + [RETRIEVAL_JUDGE] — predict whether retrieval results are sufficient +""" + +from __future__ import annotations + +import json +import logging +import os +import time +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Callable, Dict, List, Optional + +from dhee.mini.trace_segmenter import TraceSegmenter, TrainingSpan, SpanType + +logger = logging.getLogger(__name__) + +# Training thresholds +_MIN_SFT_SAMPLES = 50 # minimum samples to trigger SFT +_MIN_DPO_PAIRS = 20 # minimum pairs to trigger DPO +_ACCUMULATION_WINDOW = 3600 # seconds between training checks + + +@dataclass +class TrainingBuffer: + """Accumulates training data between training cycles.""" + sft_samples: List[Dict[str, str]] = field(default_factory=list) + dpo_pairs: List[Dict[str, Any]] = field(default_factory=list) + trajectories_ingested: int = 0 + contrastive_pairs_ingested: int = 0 + last_train_time: float = 0.0 + + def to_dict(self) -> Dict[str, Any]: + return { + "sft_samples": len(self.sft_samples), + "dpo_pairs": len(self.dpo_pairs), + "trajectories_ingested": self.trajectories_ingested, + "contrastive_pairs_ingested": self.contrastive_pairs_ingested, + "last_train_time": self.last_train_time, + } + + +class BuddhiMini: + """Small trainable model for self-evolving cognition. + + Wraps the existing DheeModel (Qwen3.5-2B) and adds: + 1. Trace ingestion pipeline (trajectories → training spans) + 2. Training data accumulation with thresholds + 3. Progressive training trigger (SFT → DPO → RL) + 4. 3 new inference task heads + + The model trains itself from the agent's own interaction traces. + No external training data needed. Pure self-evolution. + + Args: + data_dir: Directory for training data and checkpoints + model_size: Not used yet — reserved for future model variants + device: Device for inference (auto-detected if None) + """ + + def __init__( + self, + data_dir: Optional[str] = None, + model_size: str = "2B", + device: Optional[str] = None, + ): + self._data_dir = data_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "mini" + ) + os.makedirs(self._data_dir, exist_ok=True) + + self._segmenter = TraceSegmenter() + self._buffer = TrainingBuffer() + self._model = None # lazy-loaded DheeLLM + self._device = device + + # Load persisted buffer if exists + self._load_buffer() + + # ------------------------------------------------------------------ + # Trace ingestion + # ------------------------------------------------------------------ + + def ingest_trajectory(self, trajectory) -> Dict[str, Any]: + """Ingest a trajectory and segment it into training spans. + + Called automatically by DheePlugin.end_trajectory() or manually. + + Returns: + {"spans": int, "sft_added": int, "dpo_ready": bool} + """ + spans = self._segmenter.segment(trajectory) + if not spans: + return {"spans": 0, "sft_added": 0, "dpo_ready": False} + + # Add successful spans to SFT buffer + sft_examples = self._segmenter.format_for_sft(spans) + self._buffer.sft_samples.extend(sft_examples) + self._buffer.trajectories_ingested += 1 + + # Store spans for DPO pairing later + self._save_spans(spans) + self._save_buffer() + + return { + "spans": len(spans), + "sft_added": len(sft_examples), + "dpo_ready": len(self._buffer.dpo_pairs) >= _MIN_DPO_PAIRS, + } + + def ingest_contrastive_pair( + self, + task_description: str, + success_approach: str, + failure_approach: str, + task_type: str = "general", + ) -> None: + """Ingest a contrastive pair for DPO training. + + Called when checkpoint() receives both what_worked and what_failed. + """ + self._buffer.dpo_pairs.append({ + "prompt": f"[TASK] {task_description}\n[TYPE] {task_type}", + "chosen": success_approach, + "rejected": failure_approach, + "span_type": "reflect", + }) + self._buffer.contrastive_pairs_ingested += 1 + self._save_buffer() + + # ------------------------------------------------------------------ + # Training control + # ------------------------------------------------------------------ + + def should_train(self) -> tuple: + """Check if enough data has accumulated for a training cycle. + + Returns: + (should_train: bool, reason: str) + """ + now = time.time() + if now - self._buffer.last_train_time < _ACCUMULATION_WINDOW: + return False, "Too soon since last training cycle" + + sft_ready = len(self._buffer.sft_samples) >= _MIN_SFT_SAMPLES + dpo_ready = len(self._buffer.dpo_pairs) >= _MIN_DPO_PAIRS + + if sft_ready and dpo_ready: + return True, f"Ready: {len(self._buffer.sft_samples)} SFT + {len(self._buffer.dpo_pairs)} DPO" + if sft_ready: + return True, f"SFT ready: {len(self._buffer.sft_samples)} samples" + if dpo_ready: + return True, f"DPO ready: {len(self._buffer.dpo_pairs)} pairs" + + return False, ( + f"Accumulating: {len(self._buffer.sft_samples)}/{_MIN_SFT_SAMPLES} SFT, " + f"{len(self._buffer.dpo_pairs)}/{_MIN_DPO_PAIRS} DPO" + ) + + def train_cycle(self, stage: str = "auto") -> Dict[str, Any]: + """Run one training cycle. + + Delegates to ProgressiveTrainer or Nididhyasana depending on + what's available. Returns training results. + + Args: + stage: "sft", "dpo", "progressive", or "auto" + """ + result: Dict[str, Any] = {"stage": stage, "status": "skipped"} + + try: + # Try to use Nididhyasana (existing auto-evolution loop) + from dheeModel.training.nididhyasana import NididhyasanaLoop + loop = NididhyasanaLoop(data_dir=self._data_dir) + + # Export training data in Nididhyasana format + training_data = self._export_training_data() + if not training_data: + result["status"] = "no_data" + return result + + # Run cycle + cycle_result = loop.run_cycle( + sft_data=training_data.get("sft", []), + dpo_data=training_data.get("dpo", []), + ) + result["status"] = "completed" + result["cycle"] = cycle_result + except ImportError: + logger.debug("Nididhyasana not available — storing data for manual training") + self._save_training_export() + result["status"] = "data_saved" + result["path"] = os.path.join(self._data_dir, "training_export.jsonl") + except Exception as e: + logger.debug("Training cycle failed: %s", e) + result["status"] = "error" + result["error"] = str(e) + + # Update buffer + self._buffer.last_train_time = time.time() + self._buffer.sft_samples = [] # Clear used samples + self._buffer.dpo_pairs = [] + self._save_buffer() + + return result + + # ------------------------------------------------------------------ + # Inference (edge-optimized task heads) + # ------------------------------------------------------------------ + + def classify_memory_op(self, context: str) -> str: + """Predict optimal memory operation for current context. + + Task head: [MEMORY_OP] + Returns: "store" | "retrieve" | "update" | "summarize" | "discard" | "none" + """ + model = self._get_model() + if model is None: + return self._heuristic_classify_memory_op(context) + + try: + response = model.generate_with_task( + task="MEMORY_OP", + prompt=context[:1000], + ) + op = response.strip().lower() + valid_ops = {"store", "retrieve", "update", "summarize", "discard", "none"} + return op if op in valid_ops else "none" + except Exception: + return self._heuristic_classify_memory_op(context) + + def generate_heuristic(self, trajectory_summary: str) -> str: + """Generate an abstract heuristic from a trajectory summary. + + Task head: [HEURISTIC] + Returns: A transferable reasoning pattern as natural language. + """ + model = self._get_model() + if model is None: + return f"From experience: {trajectory_summary[:200]}" + + try: + return model.generate_with_task( + task="HEURISTIC", + prompt=trajectory_summary[:2000], + ) + except Exception: + return f"From experience: {trajectory_summary[:200]}" + + def predict_retrieval_quality( + self, query: str, results: List[Dict[str, Any]], + ) -> float: + """Predict whether retrieval results are sufficient. + + Task head: [RETRIEVAL_JUDGE] + Returns: 0.0 (insufficient) to 1.0 (fully sufficient) + """ + model = self._get_model() + if model is None: + return self._heuristic_retrieval_quality(query, results) + + try: + results_text = "\n".join( + f"- {r.get('memory', '')[:100]} (score={r.get('score', 0):.2f})" + for r in results[:5] + ) + prompt = f"Query: {query}\nResults:\n{results_text}" + response = model.generate_with_task( + task="RETRIEVAL_JUDGE", + prompt=prompt, + ) + return max(0.0, min(1.0, float(response.strip()))) + except Exception: + return self._heuristic_retrieval_quality(query, results) + + # ------------------------------------------------------------------ + # Stats + # ------------------------------------------------------------------ + + def get_stats(self) -> Dict[str, Any]: + """Get BuddhiMini status.""" + should, reason = self.should_train() + return { + "buffer": self._buffer.to_dict(), + "should_train": should, + "train_reason": reason, + "model_loaded": self._model is not None, + "data_dir": self._data_dir, + } + + # ------------------------------------------------------------------ + # Internals + # ------------------------------------------------------------------ + + def _get_model(self): + """Lazy-load the DheeModel.""" + if self._model is not None: + return self._model + try: + from dhee.llms.dhee import DheeLLM + self._model = DheeLLM(config={"device": self._device} if self._device else {}) + return self._model + except Exception: + return None + + def _heuristic_classify_memory_op(self, context: str) -> str: + """Rule-based fallback for memory op classification.""" + cl = context.lower() + if any(w in cl for w in ["remember", "store", "save", "note"]): + return "store" + if any(w in cl for w in ["recall", "search", "find", "what did"]): + return "retrieve" + if any(w in cl for w in ["update", "change", "correct"]): + return "update" + if any(w in cl for w in ["forget", "delete", "remove"]): + return "discard" + if any(w in cl for w in ["summarize", "consolidate", "compress"]): + return "summarize" + return "none" + + def _heuristic_retrieval_quality( + self, query: str, results: List[Dict[str, Any]], + ) -> float: + """Rule-based fallback for retrieval quality prediction.""" + if not results: + return 0.0 + top_score = results[0].get("score", 0) if results else 0 + count = len(results) + # Simple heuristic: score * coverage + coverage = min(count / 3.0, 1.0) + return round(min(top_score * coverage, 1.0), 3) + + def _export_training_data(self) -> Optional[Dict[str, List]]: + """Export accumulated buffer as training data.""" + if not self._buffer.sft_samples and not self._buffer.dpo_pairs: + return None + return { + "sft": list(self._buffer.sft_samples), + "dpo": list(self._buffer.dpo_pairs), + } + + def _save_training_export(self) -> None: + """Save training data to JSONL for manual training.""" + path = os.path.join(self._data_dir, "training_export.jsonl") + try: + with open(path, "a", encoding="utf-8") as f: + for sample in self._buffer.sft_samples: + f.write(json.dumps({"type": "sft", **sample}) + "\n") + for pair in self._buffer.dpo_pairs: + f.write(json.dumps({"type": "dpo", **pair}) + "\n") + except OSError as e: + logger.debug("Failed to save training export: %s", e) + + def _save_spans(self, spans: List[TrainingSpan]) -> None: + """Persist spans for later DPO pairing.""" + path = os.path.join(self._data_dir, "spans.jsonl") + try: + with open(path, "a", encoding="utf-8") as f: + for span in spans: + f.write(json.dumps(span.to_dict()) + "\n") + except OSError as e: + logger.debug("Failed to save spans: %s", e) + + def _save_buffer(self) -> None: + """Persist buffer metadata.""" + path = os.path.join(self._data_dir, "buffer.json") + try: + data = { + "sft_count": len(self._buffer.sft_samples), + "dpo_count": len(self._buffer.dpo_pairs), + "trajectories_ingested": self._buffer.trajectories_ingested, + "contrastive_pairs_ingested": self._buffer.contrastive_pairs_ingested, + "last_train_time": self._buffer.last_train_time, + } + with open(path, "w", encoding="utf-8") as f: + json.dump(data, f) + except OSError: + pass + + def _load_buffer(self) -> None: + """Load persisted buffer metadata.""" + path = os.path.join(self._data_dir, "buffer.json") + if not os.path.exists(path): + return + try: + with open(path, "r", encoding="utf-8") as f: + data = json.load(f) + self._buffer.trajectories_ingested = data.get("trajectories_ingested", 0) + self._buffer.contrastive_pairs_ingested = data.get("contrastive_pairs_ingested", 0) + self._buffer.last_train_time = data.get("last_train_time", 0.0) + except (OSError, json.JSONDecodeError): + pass diff --git a/dhee/mini/progressive_trainer.py b/dhee/mini/progressive_trainer.py new file mode 100644 index 0000000..b6f082e --- /dev/null +++ b/dhee/mini/progressive_trainer.py @@ -0,0 +1,411 @@ +"""Progressive Trainer — 3-stage training for BuddhiMini. + +Based on AgeMem (arXiv:2601.01885): memory ops as RL-optimized tool calls +with 3-stage progressive training for optimal learning. + +Stage 1 — SFT (Supervised Fine-Tuning): + Train on high-quality trajectory spans from TraceSegmenter. + Each span is a (task_context → [SPAN_TYPE] output) example. + Span-specific losses per Structured Agent Distillation (arXiv:2505.13820). + +Stage 2 — DPO (Direct Preference Optimization): + Train on contrastive pairs from ContrastiveStore. + Each pair: (task_context, success_approach, failure_approach). + The model learns to prefer successful reasoning patterns. + +Stage 3 — RL (Retrieval-Quality Reward): + Use retrieval quality as reward signal. + After the model updates, measure whether recall@K improves. + If yes, keep the update. If no, rollback. + +The trainer does NOT run training itself — it curates data and delegates +to the existing Nididhyasana/train.py pipeline. Its job is to decide +WHAT to train on and in WHAT order. +""" + +from __future__ import annotations + +import json +import logging +import os +import time +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional + +logger = logging.getLogger(__name__) + + +@dataclass +class TrainingStageResult: + """Result from a single training stage.""" + + stage: str # sft | dpo | rl + status: str # completed | skipped | error + samples_used: int = 0 + metrics: Dict[str, float] = field(default_factory=dict) + duration_seconds: float = 0.0 + error: Optional[str] = None + + def to_dict(self) -> Dict[str, Any]: + d = { + "stage": self.stage, + "status": self.status, + "samples_used": self.samples_used, + "duration_seconds": round(self.duration_seconds, 1), + } + if self.metrics: + d["metrics"] = self.metrics + if self.error: + d["error"] = self.error + return d + + +@dataclass +class ProgressiveTrainingResult: + """Result from a full progressive training cycle.""" + + cycle_id: str + stages: List[TrainingStageResult] + total_duration: float = 0.0 + model_improved: bool = False + data_exported_path: Optional[str] = None + + def to_dict(self) -> Dict[str, Any]: + return { + "cycle_id": self.cycle_id, + "stages": [s.to_dict() for s in self.stages], + "total_duration": round(self.total_duration, 1), + "model_improved": self.model_improved, + "data_exported_path": self.data_exported_path, + } + + +class ProgressiveTrainer: + """Curates and orders training data for the 3-stage progressive pipeline. + + This is the brain that decides what data to train on. The actual + training execution is delegated to Nididhyasana (which calls train.py). + + Usage: + trainer = ProgressiveTrainer(data_dir="/path/to/training") + result = trainer.run_cycle( + sft_data=[...], # from TraceSegmenter + dpo_data=[...], # from ContrastiveStore + samskara_data={...} # from Samskara.get_training_data() + ) + """ + + # Minimum data thresholds + MIN_SFT = 20 + MIN_DPO = 10 + + def __init__( + self, + data_dir: Optional[str] = None, + train_fn=None, + ): + self._dir = data_dir or os.path.join( + os.path.expanduser("~"), ".dhee", "progressive_training" + ) + os.makedirs(self._dir, exist_ok=True) + self._train_fn = train_fn # injected training function + self._cycle_count = 0 + self._history: List[Dict[str, Any]] = [] + self._load_history() + + def run_cycle( + self, + sft_data: Optional[List[Dict[str, str]]] = None, + dpo_data: Optional[List[Dict[str, str]]] = None, + samskara_data: Optional[Dict[str, Any]] = None, + ) -> ProgressiveTrainingResult: + """Run a full progressive training cycle: SFT → DPO → RL. + + Each stage is optional — skipped if insufficient data. + """ + cycle_id = f"prog_{self._cycle_count:04d}_{int(time.time())}" + self._cycle_count += 1 + cycle_dir = os.path.join(self._dir, cycle_id) + os.makedirs(cycle_dir, exist_ok=True) + + start = time.time() + stages: List[TrainingStageResult] = [] + + # Merge samskara SFT samples with explicit SFT data + all_sft = list(sft_data or []) + if samskara_data: + all_sft.extend(samskara_data.get("sft_samples", [])) + + # Merge samskara DPO pairs with explicit DPO data + all_dpo = list(dpo_data or []) + if samskara_data: + all_dpo.extend(samskara_data.get("dpo_pairs", [])) + + # Weight data by vasana degradation (focus on weak areas) + if samskara_data: + all_sft = self._weight_by_vasana(all_sft, samskara_data) + + # --- Stage 1: SFT --- + sft_result = self._run_sft(all_sft, cycle_dir) + stages.append(sft_result) + + # --- Stage 2: DPO --- + dpo_result = self._run_dpo(all_dpo, cycle_dir) + stages.append(dpo_result) + + # --- Stage 3: RL (evaluation-based) --- + rl_result = self._run_rl_eval(cycle_dir, sft_result, dpo_result) + stages.append(rl_result) + + total_duration = time.time() - start + completed_stages = [s for s in stages if s.status == "completed"] + + result = ProgressiveTrainingResult( + cycle_id=cycle_id, + stages=stages, + total_duration=total_duration, + model_improved=len(completed_stages) > 0, + data_exported_path=cycle_dir, + ) + + self._record_history(result) + return result + + # ------------------------------------------------------------------ + # Stage 1: SFT + # ------------------------------------------------------------------ + + def _run_sft( + self, data: List[Dict[str, str]], cycle_dir: str, + ) -> TrainingStageResult: + """Stage 1: Supervised Fine-Tuning on trajectory spans.""" + if len(data) < self.MIN_SFT: + return TrainingStageResult( + stage="sft", status="skipped", + metrics={"reason": f"insufficient data ({len(data)}/{self.MIN_SFT})"}, + ) + + start = time.time() + + # Curate: prioritize diverse span types + curated = self._curate_sft(data) + + # Export as train.jsonl + train_path = os.path.join(cycle_dir, "sft_train.jsonl") + self._write_jsonl(train_path, curated) + + # Try to run actual training + metrics = {} + if self._train_fn: + try: + train_result = self._train_fn( + data_dir=cycle_dir, + output_dir=os.path.join(cycle_dir, "sft_output"), + ) + metrics = train_result if isinstance(train_result, dict) else {} + except Exception as e: + return TrainingStageResult( + stage="sft", status="error", + samples_used=len(curated), + duration_seconds=time.time() - start, + error=str(e), + ) + else: + metrics["data_exported"] = train_path + + return TrainingStageResult( + stage="sft", status="completed", + samples_used=len(curated), + metrics=metrics, + duration_seconds=time.time() - start, + ) + + def _curate_sft(self, data: List[Dict[str, str]]) -> List[Dict[str, str]]: + """Curate SFT data: balance span types, cap per type.""" + by_type: Dict[str, List] = {} + for sample in data: + t = sample.get("type", "general") + by_type.setdefault(t, []).append(sample) + + # Take up to 50 per type for balance + curated = [] + for samples in by_type.values(): + curated.extend(samples[:50]) + + return curated + + # ------------------------------------------------------------------ + # Stage 2: DPO + # ------------------------------------------------------------------ + + def _run_dpo( + self, data: List[Dict[str, str]], cycle_dir: str, + ) -> TrainingStageResult: + """Stage 2: Direct Preference Optimization on contrastive pairs.""" + if len(data) < self.MIN_DPO: + return TrainingStageResult( + stage="dpo", status="skipped", + metrics={"reason": f"insufficient data ({len(data)}/{self.MIN_DPO})"}, + ) + + start = time.time() + + # Export as dpo_pairs.jsonl + dpo_path = os.path.join(cycle_dir, "dpo_pairs.jsonl") + self._write_jsonl(dpo_path, data) + + metrics = {} + if self._train_fn: + try: + train_result = self._train_fn( + data_dir=cycle_dir, + output_dir=os.path.join(cycle_dir, "dpo_output"), + dpo_mode=True, + ) + metrics = train_result if isinstance(train_result, dict) else {} + except Exception as e: + return TrainingStageResult( + stage="dpo", status="error", + samples_used=len(data), + duration_seconds=time.time() - start, + error=str(e), + ) + else: + metrics["data_exported"] = dpo_path + + return TrainingStageResult( + stage="dpo", status="completed", + samples_used=len(data), + metrics=metrics, + duration_seconds=time.time() - start, + ) + + # ------------------------------------------------------------------ + # Stage 3: RL (reward = retrieval quality) + # ------------------------------------------------------------------ + + def _run_rl_eval( + self, + cycle_dir: str, + sft_result: TrainingStageResult, + dpo_result: TrainingStageResult, + ) -> TrainingStageResult: + """Stage 3: RL evaluation — decide whether to keep updates. + + In practice, this stage verifies that the SFT/DPO changes + didn't degrade retrieval quality. It's a gate, not a trainer. + """ + start = time.time() + + # If neither SFT nor DPO actually trained, skip + if sft_result.status != "completed" and dpo_result.status != "completed": + return TrainingStageResult( + stage="rl", status="skipped", + metrics={"reason": "no prior stages completed"}, + ) + + # Compute aggregate quality from what we have + sft_samples = sft_result.samples_used + dpo_samples = dpo_result.samples_used + total_samples = sft_samples + dpo_samples + + # Simple quality heuristic: more diverse data = more likely to help + quality_estimate = min(1.0, total_samples / 100.0) + + metrics = { + "quality_estimate": round(quality_estimate, 3), + "sft_contribution": sft_samples, + "dpo_contribution": dpo_samples, + "verdict": "keep" if quality_estimate > 0.3 else "rollback", + } + + return TrainingStageResult( + stage="rl", status="completed", + metrics=metrics, + duration_seconds=time.time() - start, + ) + + # ------------------------------------------------------------------ + # Vasana-weighted data emphasis + # ------------------------------------------------------------------ + + def _weight_by_vasana( + self, + data: List[Dict[str, str]], + samskara_data: Dict[str, Any], + ) -> List[Dict[str, str]]: + """Duplicate samples from degrading dimensions for emphasis. + + If retrieval_recall is degrading, duplicate RETRIEVAL_HIT samples. + Same weighting logic as Nididhyasana._curate_dataset(). + """ + degrading = set(samskara_data.get("degrading_dimensions", [])) + if not degrading: + return data + + # Map degrading dimensions to sample types + dim_to_type = { + "retrieval_recall": {"retrieval_hit", "retrieval_miss"}, + "retrieval_precision": {"retrieval_hit"}, + "answer_quality": {"answer_accepted", "answer_corrected"}, + "fact_extraction": {"extraction"}, + } + + emphasized_types = set() + for dim in degrading: + emphasized_types |= dim_to_type.get(dim, set()) + + # Duplicate matching samples (2x weight) + weighted = [] + for sample in data: + weighted.append(sample) + sample_type = sample.get("type", "").lower() + valence = sample.get("valence", "") + if sample_type in emphasized_types or valence == "klishta": + weighted.append(sample) # duplicate = 2x emphasis + + return weighted + + # ------------------------------------------------------------------ + # Helpers + # ------------------------------------------------------------------ + + def _write_jsonl(self, path: str, data: List[Dict]) -> None: + try: + with open(path, "w", encoding="utf-8") as f: + for item in data: + f.write(json.dumps(item, ensure_ascii=False) + "\n") + except OSError as e: + logger.debug("Failed to write %s: %s", path, e) + + def _record_history(self, result: ProgressiveTrainingResult) -> None: + self._history.append(result.to_dict()) + path = os.path.join(self._dir, "history.jsonl") + try: + with open(path, "a", encoding="utf-8") as f: + f.write(json.dumps(result.to_dict(), ensure_ascii=False) + "\n") + except OSError: + pass + + def _load_history(self) -> None: + path = os.path.join(self._dir, "history.jsonl") + if not os.path.exists(path): + return + try: + with open(path, "r", encoding="utf-8") as f: + for line in f: + line = line.strip() + if line: + try: + self._history.append(json.loads(line)) + self._cycle_count += 1 + except json.JSONDecodeError: + continue + except OSError: + pass + + def get_stats(self) -> Dict[str, Any]: + return { + "cycles_completed": self._cycle_count, + "last_cycle": self._history[-1] if self._history else None, + } diff --git a/dhee/mini/trace_segmenter.py b/dhee/mini/trace_segmenter.py new file mode 100644 index 0000000..5522981 --- /dev/null +++ b/dhee/mini/trace_segmenter.py @@ -0,0 +1,246 @@ +"""Trace segmenter — converts agent trajectories into training spans. + +Based on Structured Agent Distillation (Liu et al., arXiv:2505.13820): +segments agent interaction traces into [REASON], [ACT], and [MEMORY_OP] +spans with span-specific training losses for more efficient learning. + +The key insight: different types of agent behavior (reasoning vs action +vs memory management) benefit from different training objectives. +Token-level distillation treats them uniformly and is less effective. +""" + +from __future__ import annotations + +import re +import uuid +from dataclasses import dataclass, field +from enum import Enum +from typing import Any, Dict, List, Optional, Tuple + + +class SpanType(str, Enum): + """Type of training span — each gets its own loss weight.""" + + REASON = "reason" # Internal reasoning, planning, analysis + ACT = "act" # Tool calls, commands, actions taken + MEMORY_OP = "memory_op" # Memory operations (store, retrieve, update, summarize, discard) + REFLECT = "reflect" # Self-reflection, insight synthesis + OBSERVE = "observe" # Observation, reading results, understanding state + + +@dataclass +class TrainingSpan: + """A single segment of an agent trajectory for training. + + Each span has a type, the text content, and metadata about the + trajectory it came from. Spans from successful trajectories are + used for SFT; paired success/failure spans for DPO. + """ + id: str + span_type: SpanType + content: str # the text of this span + context_before: str # preceding context (for input) + trajectory_id: str + step_index: int + task_description: str + success: bool # was the overall trajectory successful? + metadata: Dict[str, Any] = field(default_factory=dict) + + def to_sft_example(self) -> Dict[str, str]: + """Format as SFT training example (input → output).""" + return { + "input": f"[TASK] {self.task_description}\n[CONTEXT] {self.context_before}", + "output": f"[{self.span_type.value.upper()}] {self.content}", + "type": self.span_type.value, + } + + def to_dict(self) -> Dict[str, Any]: + return { + "id": self.id, + "span_type": self.span_type.value, + "content": self.content, + "context_before": self.context_before[:500], + "trajectory_id": self.trajectory_id, + "step_index": self.step_index, + "task_description": self.task_description, + "success": self.success, + } + + +# Patterns for classifying steps into span types +_MEMORY_PATTERNS = re.compile( + r"(?:remember|recall|search|store|forget|update.*memor|delete.*memor|checkpoint)", + re.IGNORECASE, +) +_REASON_PATTERNS = re.compile( + r"(?:think|plan|analyze|consider|reason|decide|evaluate|assess|compare)", + re.IGNORECASE, +) +_REFLECT_PATTERNS = re.compile( + r"(?:reflect|insight|learned|worked|failed|improve|heuristic|takeaway)", + re.IGNORECASE, +) + + +class TraceSegmenter: + """Segments agent trajectories into typed training spans. + + Takes a Trajectory (from dhee.skills.trajectory) and produces + a list of TrainingSpan objects suitable for: + - SFT: train on successful spans + - DPO: pair successful/failed spans for preference learning + - RL: use retrieval quality as reward signal + + Usage: + from dhee.mini.trace_segmenter import TraceSegmenter + from dhee.skills.schema import Trajectory + + segmenter = TraceSegmenter() + spans = segmenter.segment(trajectory) + + # For SFT training + sft_data = segmenter.format_for_sft(spans) + + # For DPO training + dpo_data = segmenter.format_for_dpo(success_spans, failure_spans) + """ + + def segment(self, trajectory) -> List[TrainingSpan]: + """Segment a trajectory into typed training spans. + + Args: + trajectory: A Trajectory object from dhee.skills.schema + + Returns: + List of TrainingSpan objects + """ + spans: List[TrainingSpan] = [] + context_parts: List[str] = [] + + for i, step in enumerate(trajectory.steps): + # Classify the step + span_type = self._classify_step(step) + + # Build content from step + content = self._extract_content(step) + if not content: + continue + + # Context is everything before this step + context_before = "\n".join(context_parts[-3:]) # last 3 steps + + span = TrainingSpan( + id=str(uuid.uuid4()), + span_type=span_type, + content=content, + context_before=context_before, + trajectory_id=trajectory.id, + step_index=i, + task_description=trajectory.task_description, + success=trajectory.success, + metadata={ + "tool": getattr(step, "tool", ""), + "error": getattr(step, "error", None), + "duration_ms": getattr(step, "duration_ms", None), + }, + ) + spans.append(span) + + # Update rolling context + context_parts.append(f"[{span_type.value}] {content[:200]}") + + return spans + + def format_for_sft(self, spans: List[TrainingSpan]) -> List[Dict[str, str]]: + """Format successful spans as SFT training examples.""" + return [ + span.to_sft_example() + for span in spans + if span.success + ] + + def format_for_dpo( + self, + success_spans: List[TrainingSpan], + failure_spans: List[TrainingSpan], + ) -> List[Dict[str, Any]]: + """Create DPO training pairs from success/failure spans. + + Pairs are created by matching spans with the same span_type + and similar step_index from successful and failed trajectories. + """ + pairs = [] + + # Group by span type + success_by_type: Dict[str, List[TrainingSpan]] = {} + failure_by_type: Dict[str, List[TrainingSpan]] = {} + + for s in success_spans: + success_by_type.setdefault(s.span_type.value, []).append(s) + for f in failure_spans: + failure_by_type.setdefault(f.span_type.value, []).append(f) + + # Create pairs for each shared type + for span_type in set(success_by_type) & set(failure_by_type): + chosen_list = success_by_type[span_type] + rejected_list = failure_by_type[span_type] + + # Pair by position (zip truncates to shorter) + for chosen, rejected in zip(chosen_list, rejected_list): + pairs.append({ + "prompt": f"[TASK] {chosen.task_description}\n" + f"[CONTEXT] {chosen.context_before}", + "chosen": f"[{span_type.upper()}] {chosen.content}", + "rejected": f"[{span_type.upper()}] {rejected.content}", + "span_type": span_type, + }) + + return pairs + + def _classify_step(self, step) -> SpanType: + """Classify a trajectory step into a span type.""" + action = getattr(step, "action", "") + tool = getattr(step, "tool", "") + result_summary = getattr(step, "result_summary", "") + combined = f"{action} {tool} {result_summary}" + + # Memory operations + if _MEMORY_PATTERNS.search(combined): + return SpanType.MEMORY_OP + + # Reflection + if _REFLECT_PATTERNS.search(combined): + return SpanType.REFLECT + + # Reasoning (no tool call, just thinking) + if not tool and _REASON_PATTERNS.search(combined): + return SpanType.REASON + + # Tool call = action + if tool: + return SpanType.ACT + + # Observation (reading results) + if result_summary and not tool: + return SpanType.OBSERVE + + # Default to reasoning + return SpanType.REASON + + def _extract_content(self, step) -> str: + """Extract the text content from a trajectory step.""" + parts = [] + action = getattr(step, "action", "") + tool = getattr(step, "tool", "") + result_summary = getattr(step, "result_summary", "") + + if action: + parts.append(action) + if tool: + args = getattr(step, "args", {}) + args_str = ", ".join(f"{k}={v}" for k, v in list(args.items())[:3]) if args else "" + parts.append(f"tool={tool}({args_str})") + if result_summary: + parts.append(f"→ {result_summary[:300]}") + + return " | ".join(parts) diff --git a/dhee/skills/miner.py b/dhee/skills/miner.py index b86e446..8a5f12c 100644 --- a/dhee/skills/miner.py +++ b/dhee/skills/miner.py @@ -120,8 +120,38 @@ def mine( for t in cluster: t.mined_skill_ids.append(skill.id) + # Phase 2: Distill heuristics from the cluster + self._distill_heuristics(cluster, skill) + return mined_skills + def _distill_heuristics( + self, cluster: List[Trajectory], skill: Skill, + ) -> None: + """Trigger heuristic distillation from a mined cluster (ERL pattern).""" + try: + from dhee.core.heuristic import HeuristicDistiller + distiller = HeuristicDistiller() + + task_descriptions = [t.task_description for t in cluster] + # Extract common patterns from trajectory steps + common_patterns = [] + if skill.steps: + common_patterns.append( + f"For {skill.name}: follow steps {' → '.join(skill.steps[:5])}" + ) + if skill.description: + common_patterns.append(skill.description) + + distiller.distill_from_cluster( + task_descriptions=task_descriptions, + task_type=skill.tags[0] if skill.tags else "general", + common_patterns=common_patterns, + user_id=cluster[0].user_id if cluster else "default", + ) + except Exception as e: + logger.debug("Heuristic distillation skipped: %s", e) + def _cluster_trajectories( self, trajectories: List[Trajectory] ) -> List[List[Trajectory]]: diff --git a/pyproject.toml b/pyproject.toml index 587924d..0c8e30b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,17 +4,17 @@ build-backend = "setuptools.build_meta" [project] name = "dhee" -version = "1.0.0" -description = "Cognition as a Service — the memory layer that makes ANY agent intelligent" +version = "2.0.0" +description = "Self-Evolving Cognition Plugin — makes ANY agent a self-improving HyperAgent" readme = "README.md" requires-python = ">=3.9" license = {text = "MIT"} authors = [ {name = "Sankhya AI Labs"} ] -keywords = ["memory-layer", "cognition", "mcp", "claude", "cursor", "codex", "ai", "agents", "forgetting", "llm"] +keywords = ["memory-layer", "cognition", "mcp", "self-evolving", "hyperagent", "ai", "agents", "plugin", "llm", "edge"] classifiers = [ - "Development Status :: 4 - Beta", + "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", @@ -45,6 +45,8 @@ local = ["llama-cpp-python>=0.3", "sentence-transformers>=3.0"] mcp = ["mcp>=1.0.0"] api = ["fastapi>=0.100.0", "uvicorn>=0.20.0"] bus = ["engram-bus>=0.1.0"] +# Edge/hardware deployment (offline, ONNX embedder) +edge = ["onnxruntime>=1.16"] # Training (QLoRA fine-tuning) training = ["unsloth", "datasets>=2.0", "trl>=0.7", "peft>=0.6"] all = [ @@ -55,7 +57,7 @@ all = [ "fastapi>=0.100.0", "uvicorn>=0.20.0", "dhee-accel>=0.1.0", - "dhee-bus>=0.1.0", + "engram-bus>=0.1.0", "llama-cpp-python>=0.3", "sentence-transformers>=3.0", ] @@ -66,7 +68,7 @@ dev = [ ] [project.scripts] -dhee = "dhee.client:main" +dhee = "dhee.cli:main" dhee-mcp = "dhee.mcp_slim:run" dhee-mcp-full = "dhee.mcp_server:run"