Skip to content

LLM calls are broken when the context contains generic Tools #505

@kiranandcode

Description

@kiranandcode

This is a bug I accidentally introduced in the latest polymorphic functions PR.

from dataclasses import dataclass

from effectful.handlers.llm import Template
from effectful.handlers.llm.completions import LiteLLMProvider
from effectful.handlers.llm.template import Tool
from effectful.ops.semantics import handler


@dataclass
class ExampleSet[T]:
    elts: tuple[T,...]
    
@Tool.define
def append_example[T](examples: ExampleSet[T], new_example: T) -> ExampleSet[T]:
    """Append a new example onto the example set. Raises an exception if the example is repeated."""
    return ExampleSet(elts=tuple((*examples.elts, new_example)))

@Template.define
def haiku(topic: str) -> str:
    """Generate a haiku about the topic {topic}."""
    raise NotImplementedError

with handler(LiteLLMProvider()):
    print(haiku('apple'))

Produces the error

    return self.func(self.dispatch, *args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kirancodes/Documents/code/effectful/effectful/handlers/llm/encoding.py", line 64, in type_to_encodable_type
    return __dispatch(origin_ty)(ty)
           ~~~~~~~~~~^^^^^^^^^^^
  File "/Users/kirancodes/.local/share/uv/python/cpython-3.13.9-macos-aarch64-none/lib/python3.13/functools.py", line 863, in dispatch
    impl = _find_impl(cls, registry)
  File "/Users/kirancodes/.local/share/uv/python/cpython-3.13.9-macos-aarch64-none/lib/python3.13/functools.py", line 810, in _find_impl
    mro = _compose_mro(cls, registry.keys())
  File "/Users/kirancodes/.local/share/uv/python/cpython-3.13.9-macos-aarch64-none/lib/python3.13/functools.py", line 765, in _compose_mro
    bases = set(cls.__mro__)
                ^^^^^^^^^^^
AttributeError: 'typing.TypeVar' object has no attribute '__mro__'. Did you mean: '__or__'?

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions