Added example notebook and changes for added user defined filter functions

[goirik-chakrabarty]

No description provided.

[gitnotebooks]

Found 2 changed notebooks. Review the changes at https://app.gitnotebooks.com/sensorium-competition/experanto/pull/88

[Copilot]

Pull Request Overview

This PR adds support for user-defined filter functions via a registry and provides example implementations and usage in a notebook.

• Introduce CALLABLE_REGISTRY and register_callable decorator for registering filter factories.
• Extend _get_callable_filter to instantiate filters configured with a __key__.
• Add an example filter implementation in common_filters.py and a demo notebook showing how to plug in custom filters.

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File	Description
experanto/datasets.py	Added a callable registry and extended _get_callable_filter logic to handle __key__ mappings.
examples/my_functions/common_filters.py	Example custom nan_filter registered via register_callable.
examples/create_dataset_by_id.ipynb	Notebook demonstrating definition and use of user-defined filters.

Comments suppressed due to low confidence (2)

examples/my_functions/common_filters.py:16

• [nitpick] The parameter name device_ is not very descriptive. Rename it to something like interpolator or device_interpolator to clarify its purpose.

    def implementation(device_: Interpolator, vicinity=vicinity):

experanto/datasets.py:360

• There are no unit tests covering the new registry-based instantiation path in _get_callable_filter. Add tests for registering a callable, invoking it with valid arguments, and handling missing keys to ensure this feature remains reliable.

        if isinstance(filter_config, (dict, DictConfig)) and "__key__" in filter_config:

[Copilot]

Avoid using a mutable default argument id_list=[], which can lead to unexpected shared state between calls. Use id_list=None and initialize inside the function (e.g., if id_list is None: id_list = []).

Suggested change

	"def id2interval(dataset=None, id_list=[], complement=False):\n",
	"def id2interval(dataset=None, id_list=None, complement=False):\n",

[Copilot]

The error message prints the entire filter_config dict, which may be verbose. Consider referencing only the missing key (filter_config['__key__']) to make the message clearer and more actionable.

Suggested change

	f"Filter config string '{filter_config}' not found in CALLABLE_REGISTRY {CALLABLE_REGISTRY}."
	f"Filter key '{filter_config['__key__']}' not found in CALLABLE_REGISTRY {CALLABLE_REGISTRY}."

[pollytur]

why are these parts deleted here?

[goirik-chakrabarty]

This is part of Rene's PR which I applied to my fork therefore it appeared in my PR #85 too

[schewskone]

This was simply moved to experiment from dataset since interpolation across devices has also been moved from dataset to experiment level via. get_data_from_chunks.

[schewskone]

Looks good

[schewskone]

I could not find any use for this function did I miss it? Or is it also just part of Rene's PR?

[pollytur]

we need to resolve #85 first and then adjust this one accordingly

[pollytur]

@goirik-chakrabarty am I missing sth or both of the example notebooks are actually unfinished? subset_dataset.ipynb finishes with the error and create_dataset_by_id.ipynb has a section ToDo: Use the time intervals to dataloader function to complete the demo
or am i missing sth?

[goirik-chakrabarty]

@goirik-chakrabarty am I missing sth or both of the example notebooks are actually unfinished? subset_dataset.ipynb finishes with the error and create_dataset_by_id.ipynb has a section ToDo: Use the time intervals to dataloader function to complete the demo or am i missing sth?

@pollytur Both the files look correct on my main branch apart from the usual user warnings. Am I missing something? Do I need to modify the PR to ensure these changes are reflected correctly?

[pollytur]

@goirik-chakrabarty - should these specific changes really be a part of the common repo?