#944 longitudinal normalization

[agerardy]

PR Checklist

• This comment contains a description of changes (with reason)
• Referenced issue is linked
• If you've fixed a bug or added code that should be tested, add tests!
• Documentation in docs is updated

Description of changes
#944
This PR implements normalization support for 3D EHRData objects. The implementation enables all existing normalization functions to work with longitudinal data with shape (n_obs, n_var, n_timestamps) but maintains backward compatibility with 2D data.

Technical details
Treats .R as a named layer with 3D structure. Uses helper functions (_get_target_layer, _set_target_layer, and normalize_3d_data, _normalize_2d_data) to avoid code duplication.
Each variable is processed independently by flattening the time dimension (n_obs x n_timestamps), applying the sklearn normalization function, then reshaping to 3D.

Added tests for the new functions, including group functionality and NaN cases

Examples:

edata = ed.dt.ehrdata_blobs(n_observations=100, base_timepoints=24, cluster_std=0.5, n_centers=3, seasonality=True, 
    time_shifts=True, variable_length=False)

# standard scaling
ep.pp.scale_norm(edata)

# log transformation
ep.pp.offset_negative_values(edata)
ep.pp.log_norm(edata)

[eroell]

If the edata object has R, then it will be used regardless of what the layer argument specified.

This line is a good start to investigate and set a stopping point with a debugger, to see what branch of the if/else statement is actually entered by the code, and whether it matches what you think it should :)

[Zethson]

@agerardy this hasn't been addressed right? Eljas first part of the comment.

[agerardy]

I've moved this logic to its own function _get_target_layer and changed it to check for layers first.

    if layer is None:
        if hasattr(edata, "R") and edata.R is not None:
            return edata.R, "R"
        else:
            return edata.X, "X"
    else:
        return edata.layers[layer], layer

I hope this is how it needs to work

[Zethson]

Thank you! Already looks pretty good.

1. Many of my comments are repetitive so I stopped repeating them after some time 😄
2. Many of your tests have tons of useless comments. Let the code speak for itself and clean up any LLM leftovers, please.
3. Please also follow the comments that I make in Öyku's PRs. One of them is to improve the PR description and add some usage examples.

Just a first quick pass. I'll let @eroell have a go and then I might have a look again.

Thanks!

[Zethson]

Let's keep it simple and not distinguish between 2D and 3D. We should rather finally have a proper 3D test dataset @eroell .

[eroell]

Gathering a few comments from other places here, to make it less dispersed
a) Showing some output is very helpful, see e.g. here
b) One test of 2D or 3D is enough; Can you use a 3D here, ideally the physionet2012 dataset?
c) edata_3d variable would never have been introduced

[agerardy]

I'll just reply to this one and mark the duplicates resolved :) I've written the examples with fake numbers for now, but will try to get it actually running with the physionet2012 dataset. it certainly looks better but doesnt work yet

[eroell]

Every changed function should be tested - are you preferring to make one classic example here, which you then expand to every other normalization function once we have iterated, or create them now already? Both options are OK

[agerardy]

I'm not sure what you mean by this. I have written a 3D test function for every normalization function?