perf: optimize core trajectory and clustering bottlenecks

[svdrecbd]

Summary

This PR introduces critical performance optimizations to monocle3's most computationally intensive steps, significantly improving scalability for large datasets (e.g., 20k+ cells/genes). It replaces slow R-based loops with vectorized C++ implementations for trajectory learning and marker detection, achieving speedups of 40x to 500x.

Additionally, it integrates BPCells support and Multi-Scale Embedding updates from the development branch, ensuring the package is ready for massive, out-of-core single-cell analysis.

Benchmark Results (M-Series Mac)
Benchmarks verified correctness (Diff < 1e-10) and performance across dataset scales.

Task	Baseline (Master)	Optimized (PR)	Speedup
Marker Detection (20k genes)	5.92s	0.012s	493x
Trajectory Learning (20k cells)	4.04s	0.100s	40x
Graph Construction (200k cells)	2.08s	0.718s	3x

File Change Atlas

The PR affects ~49 files, broken down as follows:

1. The Performance Engine (C++)

• src/clustering.cpp: (Critical) Added project_point_to_graph (fast geometric projection) and calc_specificity_cpp (vectorized entropy calculation). Optimized jaccard_coeff to reduce memory churn.
• src/RcppExports.cpp: Auto-generated bindings.

2. The Bottleneck Fixes (R)

• R/learn_graph.R: project2MST now delegates the O(N) loop to C++.
• R/find_markers.R: specificity_matrix now uses the C++ implementation for Jensen-Shannon divergence.
• R/cluster_cells.R: cluster_cells_make_graph now constructs graphs directly from integer vectors (avoiding data.frame overhead).
• R/graph_test.R: Applied similar make_graph optimizations for consistency.

3. BPCells & Infrastructure

• R/matrix.R: Updates for BPCells on-disk matrix handling.
• R/reduce_dimensions.R: Updates for Multi-Scale Embeddings (ResNet50 L3+L4).
• R/pca.R, R/projection.R: Support for larger, out-of-core datasets.

4. Testing & Glue

• R/RcppExports.R: R wrappers for the new C++ functions.
• tests/testthat/: Updated tests to ensure parity with the new optimized logic.
• Fixed deprecated igraph::graph.data.frame calls throughout the codebase.

Validation

• Passed R CMD check --as-cran (excluding vignettes due to local LaTeX missing).
• Numerical parity verified against baseline R implementations.
• No regressions in existing functionality.