Skip to content

Latest commit

 

History

History
220 lines (163 loc) · 15.8 KB

README.md

File metadata and controls

220 lines (163 loc) · 15.8 KB

clickhouse-loom

Python SDK for ClickHouse Loom, a self-hosted server that gives AI agents shared long-term memory, document search and a code index.

This package contains the client SDK only. The server runs as the Docker image ghcr.io/clickhouse/loom against ClickHouse: on your own machine as a single-user install, or as a team deployment an operator runs.

Install

pip install clickhouse-loom

Python 3.11+. Runtime deps: httpx[http2] (HTTP/2 via h2).

The SDK's version is the server release's; which servers it works with is in supported versions. A release not yet on PyPI installs from its assets (installing from a GitHub Release). From the root of a clone, pip install ./sdk/python.

Use

Against a single-user install on this machine (http://localhost:7777, the dev/local namespace, no token):

import asyncio

from loom import Agent


async def main() -> None:
    async with Agent(org="dev", namespace="local", agent="my-agent") as agent:
        await agent.memory.set("deploy_window", "Deploy Tue/Thu, never Friday")
        got = await agent.memory.get("deploy_window")  # by key
        hits = await agent.memory.search("when do we deploy")  # by meaning
        print(got["value"] if got else "not found")
        print([hit.memory_key for hit in hits])


asyncio.run(main())

For a team server, pass base_url= (or set LOOM_BASE_URL) and a credential, with your account's org and namespace. Either sign in once with python -m loom login --server <url> (or loom.login(url) in a notebook), approving on any browser with your password or SSO, and every Agent pointed at that server uses that session and refreshes it; or pass a token from Settings → Tokens, or one your operator mints, as api_key= or LOOM_API_KEY. A token is the right choice for CI and services.

The Agent exposes 28 client modules: client, account, deployment, assistant, memory, working_memory, knowledge, conversations, profile, context, facts, learnings, conflicts, blacklist, grants, erasure, cost, audit, namespaces, procedure, run, tasks, policy, decision, calibration, briefs, code, tracker. Each of the named ones is the hand-written layer over a wire-op group on the Loom server (account, deployment and assistant cover the routes outside the op registry: service accounts, MCP setup, model settings, worker health, the assistant's streamed chat); client holds every served op as a generated method (below).

Framework adapters live under loom.integrations: LangChain retrievers (pip install 'clickhouse-loom[langchain]'), a LlamaIndex retriever and memory block ([llamaindex]), a Haystack document store and retriever ([haystack]), OpenAI Agents SDK tools and sessions ([openai-agents]), CrewAI crew memory ([crewai]), an AutoGen memory ([autogen]), an AG2 recall policy and tools ([ag2]), a Microsoft Agent Framework context provider ([agent-framework]), a Claude API memory tool backend ([claude-memory]), a Google ADK memory service ([google-adk]), an Agno user-memory database ([agno]), a Pydantic AI toolset ([pydantic-ai]), a Pipecat memory service ([pipecat]), DSPy retrievers ([dspy]), a LangGraph store and a Strands Agents memory store (frameworks).

For Pipecat installation, pipeline placement, the single-file text example and its checked scope, see the Pipecat integration README.

Full guide: docs/reference/python-sdk.md. Deployment paths (how an operator stands up the server): docs/operations/deployment.md.

Every op by name, any op by call, a probe before you rely

loom.ops is generated from the server's operation registry: OPS (name → idempotency, required role, maturity, latency budget), OpName, and one TypedDict per request / response (MemorySearchInput, …) with Required[...] on the fields the server requires — names for your editor and type checker while the SDK stays dict-in / dict-out. MemorySearchBody is the same shape with the identity keys (org, namespace, agent, user_id, session_id, run_id) optional: what you write when agent.call stamps the identity for you.

agent.client has one method per served op, generated from the same registry: the group is the op name's first segment and the method the rest with . → _, so tracker.source.add is agent.client.tracker.source_add({...}). It takes the op's <Op>Body, returns its <Op>Output, and passes user_id=, session_id=, timeout_s= and the other agent.call options through. The TypeScript SDK exposes the same ops at the same group, camel-cased: loom.tracker.sourceAdd({...}); a unit test fails when the two differ.

from loom.ops import OPS, MemorySearchBody

body: MemorySearchBody = {"query": "deploy window", "top_k": 5, "exclude_keys": ["seen_1"]}
out = await agent.call(
    "memory.search", body, user_id="u-42"
)  # identity stamped; Idempotency-Key when OPS says idempotent
p = await agent.probe(
    ops=["memory.ask", "context.pack"], fields={"memory.search": ["exclude_keys"]}
)
assert p["ok"], (p["missing_ops"], p["missing_fields"])
print(OPS["memory.search"].idempotent, OPS["memory.search"].role)

agent.memory.search(...) and agent.knowledge.search(...) also accept **extra, so a field the server shipped this week rides the body without an SDK release. Errors are typed by the server's detail.code, and by HTTP status when a body carries none: NotFound, ValidationError, AuthInvalid, AuthInsufficient, PreconditionFailed, RateLimited, BackendUnavailable — every one carries .request_id (the X-Request-Id the server echoed), .status, and the server's .category and .retryable, the SDK's own when the body names none (the error envelope).

await agent.capabilities() pairs ops.capabilities with what the bearer token itself says (roles / ns_roles, decoded, not verified) so an integration does not poll ops its token cannot call: caps.can_call("conflicts.list") is True / False when known and None when the SDK cannot tell — then make the call; await agent.can_call(op) answers for the Agent's namespace; caps.callable() lists every served op the token may call. Cached 60 s.

agent.call_with_meta(...) is the same call returning a CallResult: .data, .request_id (finds the call in the server's ledger), .schema_version, .server_timing_ms, .rate_limit and .attempts. Every request identifies itself (User-Agent / X-Loom-SDK: loom-py/<version>, X-Loom-Runtime, X-Loom-Retry-Count); Retry-After is honoured as seconds or an HTTP-date. The credential never appears in repr(agent.transport) (it shows api_key=…1234), and a blank LOOM_API_KEY is treated as unset. The package ships py.typed and passes mypy --strict.

Give your agent the guide

await agent.guide() returns how to use Loom, written for a model: the loop, where each kind of state goes and how to read what comes back (the experimental mentor.guide, the loom skill in skills/loom/). Put it in the system prompt of an agent that has no plugin to carry it; part="reference" returns every plane with its ops.

system_prompt = "You triage support tickets.\n\n" + await agent.guide()

Ask the mentor

The store reads what your agent actually calls and names the planes it never uses — the code index, the tracker source, documents, records, offload, the entity graph — with the counts behind each hint. Two channels need nothing from you: every successful response may carry one X-Loom-Hint, which the transport logs at INFO once per distinct text and exposes as CallResult.hint, and a context.pack at session start carries up to three hints. A bot that never renders a pack asks once at startup:

async with Agent(org="acme", namespace="support", agent="triage-bot") as bot:
    brief = await bot.mentor()  # logs up to three hints; returns the brief
    for h in brief["hints"]:
        print(h["code"], h["severity"], h["text"], h["evidence"])

mentor_brief=True (or LOOM_MENTOR_BRIEF=on) runs it in the background on the first call instead. Name your agent: "", "-" and "*" default to sdk-python:<script> with a warning, because an anonymous agent is every other anonymous caller to the ledger, the mentor and the cost views.

Ask the assistant

The Loom assistant answers a question about a namespace as your token: it reads only what you may read, and each answer spends from the namespace's daily model cost cap. stream yields the text as it is written, then one AssistantAnswer whose citations the server checked against Loom:

from loom import AssistantText

async for event in bot.assistant.stream("When are production deploys frozen?"):
    if isinstance(event, AssistantText):
        print(event.text, end="")
    elif not event.verified:
        print("\n(no verified citation: treat the answer as unsupported)")

ask returns the AssistantAnswer whole; history carries earlier turns and namespace asks about another namespace you can read. An answer that began is never resent, nor one refused with a 429: asking again is a new answer with its own spend.

Sessions and tools: what an agent framework holds

agent.session(id, user_id=...) binds one (user, session) scope to the verbs an agent loop needs between turns: resume() returns the context pack as system plus the transcript tail as messages; append(*turns) writes the transcript now and runs memory.set_from_messages every flush_every turns; offload() / fetch() park a large tool result and read it back by pattern; note() appends to working memory; close() flushes. It is an async context manager; SyncAgent.session() blocks.

async with agent.session("thread-42", user_id="u1") as s:
    resumed = await s.resume(query="resume the billing migration")
    system_prompt, history = resumed.system, resumed.messages
    await s.append({"role": "user", "content": text}, {"role": "assistant", "content": reply})
    blob = await s.offload(build_log, summary="pytest run 812")
    await s.note(blob["pointer"])

agent.tools(user_id=..., session_id=...) gives a model six loom verbs — loom_remember, loom_recall, loom_ask, loom_offload, loom_fetch, loom_note — as definitions("openai" | "anthropic" | "generic") and one await execute(name, arguments) that returns the string the model reads. The schemas are the same as the TypeScript SDK's.

An explicit code namespace offers a read-only code lookup before the memory tools. The model chooses symbol, search, callers or impact; tenant scope and request budgets stay in the application. Use only for a code-only tool loop:

tools = agent.tools(code_namespace="code-repo", only=["loom_code_lookup"], max_result_chars=12_000)
definitions = tools.definitions("anthropic")
result = await tools.execute("loom_code_lookup", {"kind": "symbol", "target": "pkg.Reader.read"})

Code results preserve valid JSON within the character cap and disclose whole-body or row omissions. Honor abstention, ambiguity, source citations and graph lower bounds. See the code tools guide and examples/code_lookup.py.

loom_remember returns the server's receipt: stored confirms a persisted row, while noop and merged_into_key distinguish reuse of an existing fact. Conflicts retain their peers and next action. loom_recall preserves abstention, code freshness, row status, provenance and verification fields when the server supplies them. retrieval_risk and retrieval_confidence score non-governed hits; governed identifies authoritative definitions and term_leg reports the definition lookup status. A missing attestation is not verification. Memory results remain valid JSON: whole memories that do not fit are counted in omitted; metadata that cannot fit returns insufficient and metadata_omitted. A budget too small for that marker is rejected before the call.

For a durable fact about code, the application can use agent.memory.set(..., files=["src/reader.py"], tags=[repo_tag]), where repo_tag is the matching repo:<id> tag established by the integration. It is not an arbitrary repository name. A configured code namespace also identifies the repository; code.repos reports its primary_namespace for shared file rules. files=None detects references from the value; files=[] clears prior references. The compact loom_remember schema does not expose these fields: use the direct SDK call when file or repository association matters. Linking is asynchronous, so a saved receipt does not confirm a resolved link. Read the link state before relying on the claim; see memories linked to code.

Retries and logging

The retry policy is one frozen dataclass, set on the Agent and overridable per call with a RetryPolicy or a dict of its fields; every field is validated and named when wrong. Defaults: 3 attempts; 0.1 s backoff doubling to 5 s with ±25 % jitter; 408 and 5xx retried for registry-idempotent ops (any other op only on a 408 or a 503 AUTH_UNAVAILABLE, which the server answers before running it), except an answer the server classes configuration; connection errors retried; a read timeout, a cut-off response or a write that broke off retried only for an op the registry marks idempotent and not read-only (the server dedups its Idempotency-Key); a Retry-After on a retried status honoured up to 60 s. A 429 is paced, not retried.

from loom import Agent, RetryPolicy

agent = Agent(
    "acme",
    "support",
    "bot",
    retry=RetryPolicy(max_attempts=5, total_backoff_budget_s=30),
    consumer="support-bot",
)  # the label every ledger row carries
await agent.call("memory.get", memory_key="k", retry={"max_attempts": 1})  # this call: no retries
await agent.call("memory.set_from_messages", messages=msgs, retry={"retry_timeouts": "always"})

The SDK logs through the stdlib logger loom (a NullHandler by default; your logging config routes it): one line per completed call at INFO (/v1/memory.get 200 12 ms attempts=2 rid=…), retries and rate-limit waits at WARNING, the final failure at ERROR, the request headers at DEBUG. LOOM_LOG_LEVEL (any stdlib level name — debug, info, warning, error, critical — or off) sets the logger's level when your application has not; a value it cannot read is a warning, not an import error. Credential headers are redacted to their scheme and last four characters, and a filter on the logger rewrites any bearer token that reaches a record anyway.

Examples

examples/agent_memory.py — probe → remember ×3 → recall → ask with citations → outcome → typed errors, with its actual run in examples/README.md. The TypeScript SDK ships the same example.

License

Apache-2.0. See LICENSE.