Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
89d8217
Add InfiniteParameterError exception for unenumerable parameters
fabianliebig Jul 20, 2026
8282f14
Add SequenceParameter class for handling sequence parameters
fabianliebig Jul 20, 2026
d5d07c2
feat: add sequence parameter and encode protocol
fabianliebig Jul 21, 2026
90367e6
test: add sequence parameter testes
fabianliebig Jul 21, 2026
6eea423
fix: Add ignore to validators for docs building
fabianliebig Jul 21, 2026
27c756d
fix: Update test description and method calling for _enumerate_values
fabianliebig Jul 21, 2026
5075575
fix: Simplify validation logic in validate_parameter_input function
fabianliebig Jul 21, 2026
2755385
Refactor: Improve error message by including invalid values
fabianliebig Jul 21, 2026
dbd1f6d
refactor: Simplify encoder handling in SequenceParameter class
fabianliebig Jul 21, 2026
925db4f
refactor: Enhance validation and converter logic in SequenceParameter…
fabianliebig Jul 21, 2026
192437d
refactor: Remove docstrings from overwritten methods and move cached_…
fabianliebig Jul 21, 2026
94f7341
refactor: sort alphabeth natively and allow many letter words
fabianliebig Jul 21, 2026
ed6e06d
refactor: Improve error message to use programatically defined class …
fabianliebig Jul 21, 2026
f033ced
refactor: Replace InfiniteParameterError with InfiniteSpaceError in e…
fabianliebig Jul 21, 2026
0254aed
refactor: Update SequenceParameter to use SequenceEncoderCallable and…
fabianliebig Jul 22, 2026
49d5e4a
refactor: Replace custom encoder classes with a dummy encoder functio…
fabianliebig Jul 22, 2026
3d5621a
refactor: Add type ignore comment for Converter in SequenceParameter
fabianliebig Jul 22, 2026
c34a9be
refactor: Use different typ hints and narwahls convertion
fabianliebig Jul 22, 2026
364b8d5
refactor: Remove unused encoder classes and simplify SequenceParamete…
fabianliebig Jul 22, 2026
8c2db01
Rename SequenceEncoderCallable to Encoder and add docstring
AdrianSosic Jul 23, 2026
62e19df
Fix encoder signature
AdrianSosic Jul 23, 2026
6ab4dd4
Rework attributes and corresponding docstrings
AdrianSosic Jul 23, 2026
03b7310
Inline enumeration logic into values property
AdrianSosic Jul 23, 2026
cb07cd6
Mark comp_rep_columns as pending using NotImplementError
AdrianSosic Jul 23, 2026
5971a3b
Enable generic backend support for encoders via auto-inference approach
AdrianSosic Jul 24, 2026
34c2444
Add __len__ to DiscreteParameter and override in SequenceParameter
AdrianSosic Jul 24, 2026
6bf02d3
Refactor tests and adjust validators
AdrianSosic Jul 30, 2026
fc24179
Update CHANGELOG.md
AdrianSosic Jul 30, 2026
d029850
Drop unneeded and_ import
AdrianSosic Jul 30, 2026
604845c
Add equality test for mismatching alphabet orders
AdrianSosic Jul 30, 2026
3c2c46f
Generalize input type of SequenceParameter.is_in_range to Sequence
AdrianSosic Jul 30, 2026
d2023ae
Add missing attribute docstrings
AdrianSosic Jul 30, 2026
d5c805d
Drop unnecessary type ignore
AdrianSosic Jul 30, 2026
71ce4a7
Fix Never import
AdrianSosic Jul 30, 2026
8e1925e
Add comment explaining usage of __slots__ in protocol
AdrianSosic Jul 31, 2026
ad6d27a
Fix typos
AdrianSosic Jul 31, 2026
c98fbd0
Add encoder equality test
AdrianSosic Jul 31, 2026
d8c6e11
Exclude _Encoder._implementation from equality comparison
AdrianSosic Jul 31, 2026
44180a7
Raise clear error when unserializable object is passed to base unstru…
AdrianSosic Aug 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.md
Comment thread
AdrianSosic marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Optional/secondary fields of discrete parameter classes are now keyword-only

### Added
- `SequenceParameter` class for modeling parameters whose values are configurable-length
token sequences from a predefined alphabet
Comment on lines +25 to +26

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add here: linear & grammar free?

- `EncoderProtocol` as a public interface for specifying parameter encoders
- `coefficients` attribute for `DiscreteSumConstraint`, enabling weighted sums. Follows
the same pattern as `ContinuousLinearConstraint.coefficients`
- `simplex_coefficients` keyword argument to `SubspaceDiscrete.from_simplex` for
Expand All @@ -30,7 +33,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `CandidatesProtocol` as an interface for candidates generation
- `EmptyCandidates`, `TableCandidates` and `ProductCandidates` classes implementing
`CandidatesProtocol`
- `DiscreteParameter.is_finite` property
- `DiscreteParameter.__len__` and `DiscreteParameter.is_finite` property
- `SubspaceDiscrete.batch_constraints` field for storing batch-level constraints
- `SubspaceDiscrete.from_dataframe` now accepts `batch_constraints`
- `validate_parameter_input` now accepts an `allow_empty` flag to permit zero-row input
Expand Down
2 changes: 1 addition & 1 deletion baybe/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ class UnsupportedEarlyFilteringError(Exception):


class InfiniteSpaceError(Exception):
"""An operation requires a finite search space but the space is infinite."""
"""An operation requires a finite space but the space is infinite."""


# Collect leftover original slotted classes processed by `attrs.define`
Expand Down
4 changes: 4 additions & 0 deletions baybe/parameters/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from baybe.parameters.categorical import CategoricalParameter, TaskParameter
from baybe.parameters.custom import CustomDiscreteParameter
from baybe.parameters.encoding import EncoderProtocol
from baybe.parameters.enum import (
CategoricalEncoding,
CustomEncoding,
Expand All @@ -11,6 +12,7 @@
NumericalContinuousParameter,
NumericalDiscreteParameter,
)
from baybe.parameters.sequence import SequenceParameter
from baybe.parameters.substance import SubstanceParameter
from baybe.utils.metadata import MeasurableMetadata

Expand All @@ -19,9 +21,11 @@
"CategoricalParameter",
"CustomDiscreteParameter",
"CustomEncoding",
"EncoderProtocol",
"MeasurableMetadata",
"NumericalContinuousParameter",
"NumericalDiscreteParameter",
"SequenceParameter",
"SubstanceEncoding",
"SubstanceParameter",
"TaskParameter",
Expand Down
22 changes: 15 additions & 7 deletions baybe/parameters/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from narwhals.stable.v2.dependencies import is_into_series
from typing_extensions import override

from baybe.exceptions import InfiniteSpaceError
from baybe.serialization import (
SerialMixin,
)
Expand Down Expand Up @@ -131,11 +132,18 @@ class DiscreteParameter(Parameter, ABC):
def values(self) -> tuple:
"""The values the parameter can take."""

def __len__(self) -> int:
"""Return the number of values the parameter can take."""
Comment thread
AVHopp marked this conversation as resolved.
return len(self.values)

@property
def is_finite(self) -> bool:
"""Indicates whether the parameter has a finite number of values."""
len(self.values) # <-- raises an error if the parameter is infinite
return True
try:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused by this. According to the typing of the values property, this is intended to return a tuple with the values the parameter can take. As far as I know, tuples are always finite, so in what case is this being triggered resp. how does such a case align with values being a tuple?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly, values is a tuple, but the return types only ever describe the non-error path. In the infinite case, values will throw an InfiniteSpaceError which we catch and then conclude that the space is infinite.

Raising an error in a property is certainly not 100% nice and we may want to adjust this in the future. However, without a refactoring, this is what we can do, so I simply comply with the existing protocols

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this was meant as a functionality test to avoid making mistakes during development. Since the sequence parameter overrides and uses is_finite to determine if the InfiniteSpaceError exception should be raised, it's now kind of an Ouroboros scenario, isn't it?

@AdrianSosic AdrianSosic Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fabianliebig: yeah, you are right, I did not properly think this through. So we have three options:

  • Keep the current code in the base class and delete the override
  • Keep the override and replace the code in base with simply return True
  • Keep both --> Ouroboros

What do you think?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would prefer having an explicit assert, returning true and keeping the override. By that we avoid three things:

  • No exception as flow control. Maybe that is personal preference, but I think that can mask errors and is not unavoidable in this case.
  • Too many redundant implementations (only the sequence parameter needs to determine if it's finite, so a abstractmethod would be an overkill).
  • We will still be noticed if something is not properly implemented in the future.

What do you think? Does that make sense to you?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So basically the version of base that we had before, right?

@fabianliebig fabianliebig Aug 7, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, would most appreciate a real assert actually but as we discuss that previously, the base version is also fine for me 🙃

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you maybe just suggest (using github) the version you'd ideally like to have? 🙃

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I did not phrase that really well. Ideally I would like to have an assert like assert isinstance(self.values, Sized) but it doesn't really matter because in the end, it still only produces a runtime error. So I probably should just have answered, 'Yeah, let's use the old version.' 🙈

len(self)
return True
except InfiniteSpaceError:
return False

@property
def active_values(self) -> tuple:
Expand Down Expand Up @@ -242,7 +250,7 @@ def summary(self) -> dict:
return dict(
Name=self.name,
Type=self.__class__.__name__,
nValues=len(self.values),
nValues=len(self),
)


Expand All @@ -261,7 +269,7 @@ class _EncodedDiscreteParameter(DiscreteParameter, ABC):
_active_values: tuple[str | bool, ...] | None = field(
default=None,
converter=optional_c(
Converter( # type: ignore[misc, call-overload]
Converter( # type: ignore[misc]
nonstring_to_tuple, takes_self=True, takes_field=True
)
),
Expand Down Expand Up @@ -301,10 +309,10 @@ def _validate_active_values( # noqa: DOC101, DOC103
)
if len(set(content)) != len(content):
raise ValueError("The active parameter values must be unique.")
if not all(v in self.values for v in content):
if invalid := [v for v in content if not self.is_in_range(v)]:
raise ValueError(
f"All active values must be valid parameter choices from: "
f"{self.values}, provided: {content}"
f"All active values must be valid parameter choices. "
f"Provided invalid values: {invalid}"
)

@override
Expand Down
4 changes: 2 additions & 2 deletions baybe/parameters/categorical.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ class CategoricalParameter(_EncodedDiscreteParameter):
encoding: CategoricalEncoding = field(
default=CategoricalEncoding.OHE, converter=CategoricalEncoding, kw_only=True
)
# See base class.
"""The encoding used the generate the parameters computational representation."""

@override
@property
Expand Down Expand Up @@ -108,7 +108,7 @@ class TaskParameter(CategoricalParameter):
"""Parameter class for task parameters."""

encoding: CategoricalEncoding = field(default=CategoricalEncoding.INT, init=False)
# See base class.
"""The encoding used the generate the parameters computational representation."""


# Collect leftover original slotted classes processed by `attrs.define`
Expand Down
133 changes: 133 additions & 0 deletions baybe/parameters/encoding.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
"""Sequence encoders."""

from __future__ import annotations

import gc
from typing import TYPE_CHECKING, Protocol, runtime_checkable

import narwhals.stable.v2 as nw
from attrs import define, field
from attrs.validators import instance_of
from exceptiongroup import ExceptionGroup

from baybe.settings import active_settings
from baybe.utils.dataframe import _df_with_backend

if TYPE_CHECKING:
from narwhals.stable.v2.typing import IntoDataFrame, IntoSeries


@runtime_checkable
class EncoderProtocol(Protocol):
Comment thread
AVHopp marked this conversation as resolved.
"""Type protocol specifying the interface encoders need to implement."""

# Use slots so that derived classes also remain slotted
# See also: https://www.attrs.org/en/stable/glossary.html#term-slotted-classes
__slots__ = ()
Comment thread
AVHopp marked this conversation as resolved.

def __call__(self, series: IntoSeries, /) -> IntoDataFrame:
"""Encode a given series of values.

Args:
series: A series in an arbitrary backend, containing the values to encode.

Returns:
A dataframe containing the encoded representations of the input values, in
the same backend as the input series and with the same row order.
Comment thread
AVHopp marked this conversation as resolved.
"""


@define
class _Encoder:
"""A narwhals wrapper for user-specified encoders to hide their native backend.

Wraps a user-provided instance of an :class:`EncoderProtocol` and automatically
infers which native dataframe backend it expects via trial and error. The inferred
backend is cached after the first successful call so that trial-and-error detection
runs only once.
"""

_encoder: EncoderProtocol = field(
alias="encoder", validator=instance_of(EncoderProtocol)
)
"""The user-provided encoder."""

_implementation: nw.Implementation | None = field(
default=None, init=False, eq=False
)
"""The inferred native backend, cached after the first successful call."""

def __call__(self, series: nw.Series, /) -> nw.DataFrame:
"""Encode a narwhals series, inferring the required backend if not yet known.

On the first call, tries available backends in order (starting with
:attr:`~baybe.settings.Settings.default_dataframe_backend`) until the wrapped
callable succeeds. The successful backend is cached for all subsequent calls.

Args:
series: A narwhals series containing the values to encode.

Returns:
A narwhals dataframe containing the encoded representations, collected into
the same backend used for the input series.
"""
if self._implementation is None:
self._implementation, result = self._infer_backend(series)
return result

return self._encode(series, self._implementation)

def _infer_backend(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on this? The user will need to wirte an Encoder with a call that uses some backend, can't we simply infer which the user wants to use from that? Just trying it out according to an arbitrary ordering feels weird to me.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have a better solution, let me know. The problem is that the user can write their callable using arbitrary logic and the hole point is that we don't want to impose any backend on them. So without "looking into" the blackbox, I see no way how we could possibly know what they used.

The situation is essentially the following:

def my_encoder(dataframe):  # no annotations
    # code hidden

The above is what we get. So with which backend would you now call it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we could also have proper typing of the encoder as a requirement. Like "We promise that we can handle all backends, but please tell us which you use", I think this is not too hard of a requirement

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would also prefer it this way. Automatic backend inference could lead to unforeseen issues, so I would rather make the backend explicit and ensure that users are aware of their responsibility.

I could only come up with a toy example, but in the following code the loop detects Polars, since both Polars and pandas share the interface used in the first encoder call. However, it later crashes because Polars Series objects do not expose values as a property:

class StatefulEncoder:
    def __init__(self) -> None:
        """Initialize the call counter."""
        self._n_calls = 0

    def __call__(self, series):
        self._n_calls += 1
        if self._n_calls == 1:
            return pd.DataFrame({"encoded": list(series)})
        return pd.DataFrame({"encoded_upper": series.values.tolist()})

def main() -> None:
    s = nw.from_native(pd.Series(["a", "b", "c"], name="x"), series_only=True)

    enc = _Encoder(encoder=StatefulEncoder())

    result1 = enc(s)
    print(f"Detected backend : {enc._implementation}")
    print(f"Result 1 columns : {result1.columns}")
    print()

    result2 = enc(s). #<-- Crash

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a valid point! I think the decision we have to make is here is the balance between precision and convenience:

  • Your example is perfectly valid in the sense that the current implementation has no way to fix it.
  • On the other hand, the stateful case is the only one where I can see problems like this (since if a stateless one passes once, it'll also pass the subsequent calls), stateful encoders are also more on the exotic end and rather irrelevant for 99% of the users.

So one potential middle ground I could see is to simply make both the _Encoder class and its _implementation attribute public, and only set None as the default for the latter. The implications would be:

  • The average user could still just pass an unlabeled callable and it'll be auto-wrapped into an Encoder with auto-inferred backend.
  • A user who wants to explicitly specify the backend would manually wrap their callable into an Encoder and manually specify the backend. That step is unavoidable because we need some place to store the backend specification next to the callable itself, and the Encoder is exactly the object responsible for that.

What do you think? Implementation-wise, the only difference to the case where we drop the auto-inference completely is having the None default. In the enforced explicit case, that default value would simply be dropped.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that sounds like it also shapes awareness. Seems sufficient for me. I actually don't think that it will be that uncommon for something analogous to my example to appear; the most straightforward counterexample might be stateful, which is why I provided it, but I could also imagine some more complex logic that involves different sources like file systems or databases that are used in combination, just because it offers that freedom. :D

self, series: nw.Series, /
) -> tuple[nw.Implementation, nw.DataFrame]:
"""Infer the encoder's backend by trial-and-error across available backends.

Args:
series: The series to use for probing.

Returns:
A tuple containing:

* the first backend for which the encoder succeeds
* the resulting encoded dataframe

Raises:
ExceptionGroup: If the encoder raises an exception for every tried backend.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Statement is unclear, sounds like it raises all exceptions every time, but it is just in the case that no suitable backend is found.
If no suitable backend is found, raises exception for every tried backend.

"""
preferred = active_settings.default_dataframe_backend
ordered = [preferred] + [b for b in nw.Implementation if b is not preferred]
backends = [b for b in ordered if _is_backend_imported(b)]

exceptions: list[Exception] = []
for backend in backends:
try:
result = self._encode(series, backend)
return backend, result
except Exception as ex: # noqa: BLE001
exceptions.append(ex)

raise ExceptionGroup("The encoder failed for all tried backends", exceptions)

def _encode(self, series: nw.Series, backend: nw.Implementation, /) -> nw.DataFrame:
"""Call the encoder with the given series using the specified backend.

Args:
series: The series to encode.
backend: The native backend to convert the series to before encoding.

Returns:
A narwhals dataframe in the original series backend.
"""
native_series = _df_with_backend(series, backend).to_native()
result = nw.from_native(self._encoder(native_series), eager_only=True)
return _df_with_backend(result, series.implementation)


def _is_backend_imported(backend: nw.Implementation) -> bool:
"""Check if the given native backend has already been imported."""
getter = getattr(nw.dependencies, f"get_{backend.value}", None)
return getter is not None and getter() is not None


# Collect leftover original slotted classes processed by `attrs.define`
gc.collect()
Loading