Skip to content

WIP: CPU fallbacks for CUDA kernels - #29

Draft
RisingRedstone wants to merge 6 commits into
JeffreyXiang:mainfrom
RisingRedstone:cpu-support
Draft

WIP: CPU fallbacks for CUDA kernels#29
RisingRedstone wants to merge 6 commits into
JeffreyXiang:mainfrom
RisingRedstone:cpu-support

Conversation

@RisingRedstone

@RisingRedstone RisingRedstone commented Jul 19, 2026

Copy link
Copy Markdown

This PR introduces CPU implementations for coordinate and map building kernels to support execution on systems with limited GPU VRAM.

Context & Discussion: See issue #30 for the discussion on architectural alignment and if this is something worth adding.

Implementation Status

  • hashmap_build_sparse_conv_out_coords (Implemented and tested)
  • expand_unique_build_sparse_conv_out_coords (Implemented and tested)
  • hashmap_build_sparse_conv_neighbour_map
  • hashmap_build_submanifold_conv_neighbour_map

Current Optimizations & Performance Notes

  • The initial CPU kernels prioritize correctness and roughly mirror the thread block semantics of the CUDA implementations.
  • Masked Select Bypass: After some profiling, I found out that the torch implementation of masked_select was slowing the code down significantly. I implemented my own masked_select function which sped up the whole function by 2-2.5x.
  • Future Roadmap: Pending approval of this integration approach, the next phase will involve deep profiling of the CPU path. I will refactor and flatten the loops in each cpu kernel implementation (to reduce integer divisions), but I am deferring those structural changes until the full pipeline can be profiled.

Notes for Reviewers

  • This PR is currently in a Draft state to validate the integration approach before the final two map-building functions are written.

…sert in the hashmap.

Profiled the CPU performance and found out that the masked_select(unexpected) and unique(expected) were the biggest performance costs.
So I will have to figure out a better algorithm that reduce the strain on these 2 functions.
…cifically focus on the CPU for performance testing.

Also added flags to the cpp compilation in setu.py to be able to inspect the binaries better.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant