Skip to content

feat(distance): add na_comp option aligned with baseR dist behavior for NA handling - #95

Merged
SpatLyu merged 19 commits into
stscl:mainfrom
SpatLyu:dev
Sep 17, 2026
Merged

SpatLyu merged 19 commits into
stscl:mainfrom
SpatLyu:dev

Conversation

@SpatLyu

@SpatLyu SpatLyu commented Sep 17, 2026

Copy link
Copy Markdown
Member

Previously, when na_rm = true, the distance calculation would simply skip NA/NaN dimensions
and sum over the remaining valid dimensions. This creates a systematic bias: pairs with more missing
values would artificially appear "closer" than they truly are, simply because fewer dimensions
contributed to the total distance.

This behavior is inconsistent with base R's dist() function, which applies proportional scaling when computing (Euclidean, Manhattan, Canberra or Minkowski) distances with missing values. This inconsistency can lead to:

  • Unexpected results for users migrating from base R
  • Biased downstream analyses (e.g., Information Imbalance, KNN-based methods)
  • Difficulty in reproducing results across different R packages

Changes

  • Added na_comp parameter to Euclidean and Manhattan distance functions in C++ backend
  • When na_comp = true and na_rm = true, the computed distance sum is scaled up proportionally:
    adjusted_distance = raw_distance * (total_dims / valid_dims)

@SpatLyu SpatLyu self-assigned this Sep 17, 2026
@SpatLyu
SpatLyu merged commit 11f9d9e into stscl:main Sep 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant