WIP: Communication refactor - #5404

Open

RudolfWeeber wants to merge 70 commits into

espressomd:pythonfrom

RudolfWeeber:comm

Commits on Jul 26, 2026

docs: design spec for async, topology-agnostic ghost communication

RudolfWeeber
and
claude
committed
a817027
docs: Ph0+Ph1 implementation plan for async ghost communication

RudolfWeeber
and
claude
committed
82769fd

Commits on Jul 27, 2026

core/ghosts: add Caliper markers to CellStructure ghost entry points

RudolfWeeber
and
claude
committed
b723ea1
testsuite: pin ghost-communication Caliper labels

RudolfWeeber
and
claude
committed
d0b4657
benchmarks: ghost A/B runner + machine-idle gate
RudolfWeeber
committed
92cfe5e
core/ghosts: add topology-agnostic HaloPlan data model

RudolfWeeber
and
claude
committed
112e755
core/ghosts: extract reusable particle packing (no behavior change)

RudolfWeeber
and
claude
committed
da9c0d8
core/ghosts: async halo_exchange engine (p2p + local, split-phase)

RudolfWeeber
and
claude
committed
17552cc
core/ghosts: enforce one-NeighborComm-per-peer precondition in halo_exchange

RudolfWeeber
and
claude
committed
5d3c4ed
cell_system: RegularDecomposition emits a direct-neighbor HaloPlan

RudolfWeeber
and
claude
committed
0e08d3f
benchmarks: fix ghost_ab idle gate falsely refusing on idle machine (set -e/grep no-match)

RudolfWeeber
and
claude
committed
d1d8211
benchmarks: capture staggered ghost-comm baseline (LJ + P3M, 1/2/4/8 ranks)

RudolfWeeber
and
claude
committed
626180f
cell_system: route regular ghost exchange through async engine (parity)

RudolfWeeber
and
claude
committed
d388d60
core/ghosts: pack all NeighborComm cells in one archive (fix bond-buffer concat)

RudolfWeeber
and
claude
committed
314e7c3
core/ghosts: document + forward-note the pack_regions shift-equality invariant

RudolfWeeber
and
claude
committed
bfe119d
cell_system: n-square and hybrid emit HaloPlan collective section

RudolfWeeber
and
claude
committed
ed0bd33
core/ghosts: document collective PARTNUM-first invariant + plan pointer-lifetime TODO

RudolfWeeber
and
claude
committed
5e527d2
core/ghosts: remove legacy staggered engine; async HaloPlan is the only path

RudolfWeeber
and
claude
committed
f5474fc
test: drop unused variable in HaloExchange collective test (gcc-14 -Werror)

RudolfWeeber
and
claude
committed
9bfddfd
docs: Ph2 implementation plan (unified HaloPlan validator)

RudolfWeeber
and
claude
committed
68f5ee9
core/ghosts: add local HaloPlan validator (coverage, neighborship-match, peer-uniqueness)

RudolfWeeber
and
claude
committed
c174e65
test: cover recv-outside-ghost-set and isolated shape-mismatch in HaloPlan validator

RudolfWeeber
and
claude
committed
4f7858d
core/ghosts: add cross-rank HaloPlan symmetry check (all-to-all send/recv counts)

RudolfWeeber
and
claude
committed
bf5c6b9
cell_system: validate HaloPlan at build under ADDITIONAL_CHECKS; drop ad-hoc Ph1 asserts

RudolfWeeber
and
claude
committed
98d380d
core/ghosts: validator handles collective section; guard RATTLE; validate Atom on populated cells

RudolfWeeber
and
claude
committed
25055ba
core/ghosts: fix op-sanity assert -Werror portability

RudolfWeeber
and
claude
committed
92c5e98
core/ghosts: fix HaloPlan validator/make_halo_plan for 1-rank & small-grid configs

RudolfWeeber
and
claude
committed
957fbb9
docs: Ph3 plan (interior/boundary tags + Euclidean-vs-MI)

RudolfWeeber
and
claude
committed
043d3d8
cell_system: classify interior/boundary cells; validator checks consistency

RudolfWeeber
and
claude
committed
e4377fc

Commits on Jul 28, 2026

docs: Ph4 plan (latency hiding / 8-rank mitigation, profile-then-fix)

RudolfWeeber
and
claude
committed
490b838
core/ghosts: Caliper phase markers (pack/post/wait/unpack) in halo_exchange

RudolfWeeber
and
claude
committed
2430ffe
core/ghosts: reuse per-neighbor exchange buffers across steps (no per-step alloc)

RudolfWeeber
and
claude
committed
322f060
core/ghosts: RAII ExchangeBuffers ownership (fix GHOSTTRANS_NONE leak)

RudolfWeeber
and
claude
committed
3e13be3
docs: Ph4 Task 4.4 (progressive wait_any receive-unpack)

RudolfWeeber
and
claude
committed
46fa759
core/ghosts: progressive wait_any receive-unpack in halo_exchange_finish

RudolfWeeber
and
claude
committed
584571a
core/ghosts: hoist wait_any slot map into ExchangeBuffers scratch

RudolfWeeber
and
claude
committed
cbaaf94
core: guard inactive Caliper markers with getenv-once check

RudolfWeeber
and
claude
committed
d78ca2f
core/caliper: guard loop annotations; document CALI_CONFIG-only activation

RudolfWeeber
and
claude
committed
2b2daf9

Commits on Jul 29, 2026

Merge remote-tracking branch 'origin/python' into worktree-comm
RudolfWeeber
committed
08090fd
docs: Ph5 plan (local transfers in start; wrap-aware interior tags; reduce/step-2 overlap)

RudolfWeeber
and
claude
committed
9820911
docs: Ph5 Task 5.4 (non-zeroing CommBuf grow, perf-mandated)

RudolfWeeber
and
claude
committed
2f74f28
docs: Ph5 eligibility += comm.size()>1 (skip step-2 split at 1 rank)

RudolfWeeber
and
claude
committed
0636ce0
core/ghosts: run local transfers in halo_exchange_start (overlap with in-flight messages)

RudolfWeeber
and
claude
committed
3b48757
cell_system: boundary tag accounts for periodic-wrap neighborships; validator overlap invariant

RudolfWeeber
and
claude
committed
50c874c
cell_system: cell_grid==1 wrap axes force all-boundary (fix false interior)

RudolfWeeber
and
claude
committed
24fe5fd
docs: append fix-round entry to task-5.2-report.md

RudolfWeeber
and
claude
committed
5149cc3
core: overlap ghost force reduction with interior-cell velocity update

RudolfWeeber
and
claude
committed
438dd54
core: exception-safe pending ghost reduce; caliper region ordering

RudolfWeeber
and
claude
committed
51f629d
core/ghosts: grow exchange buffers without zero-fill (perf: -1% memset at 10k/8)

RudolfWeeber
and
claude
committed
d1c4c67
docs: task-5.4 report (DefaultInitAllocator CommBuf scheme, audit, test evidence)

RudolfWeeber
and
claude
committed
9f9cc38
docs: Ph5 Task 5.5 (runtime-conditional quat/torque ghost payload)

RudolfWeeber
and
claude
committed
99efa70
core/ghosts: send quat/torque only when orientation physics is active

RudolfWeeber
and
claude
committed
5736762
core: explicit Stoner-Wohlfarth arm in orientation ghost whitelist

RudolfWeeber
and
claude
committed
50b93bf
core: fix PR 5404 CI (AppleClang unused lambda capture; doxygen refs)

RudolfWeeber
and
claude
committed
5d39db5
core/ghosts: sweep remaining unresolvable doxygen refs in HaloExchange.hpp

RudolfWeeber
and
claude
committed
7ea9cf3
core/ghosts: reduce self-copy ghost forces back to owner (fix ICC 3-rank)

RudolfWeeber
and
claude
committed
3c62b13
test: pin LocalComm reduce role-swap in halo_exchange (regression for 3c62b135)

RudolfWeeber
and
claude
committed
2d0fb8a
cell_system: mark plan-exported cells boundary; diagnostic halo-plan asserts

RudolfWeeber
and
claude
committed
e839e36
core/ghosts: validate plan symmetry at first use, not construction

RudolfWeeber
and
claude
committed
430591b
docs: Ph5 Task CI-r3 (plan symmetry at first use, checkpoint fix)

RudolfWeeber
and
claude
committed
eba737a
docs: fix last unresolvable doxygen ref (for_each_interior_particle)

RudolfWeeber
and
claude
committed
ed1c582
core: appease clang-tidy configs (emplace_back, guarded whitelist helper, ctor-qualified calls, braced returns)

RudolfWeeber
and
claude
committed
aa8620f
core/ghosts: null-safe buffer pointers in collective exchange (UBSan)

RudolfWeeber
and
claude
committed
aae9223

Commits on Jul 30, 2026

rm specs/plans/ledger
RudolfWeeber
committed
e0a05b4
Merge branch 'python' of ssh://github.com/espressomd/espresso into worktree-comm
RudolfWeeber
committed
f5d2335

Commits on Aug 2, 2026

Reduce cross-L3 traffic of per-step buffer sweeps and parallelize resort

RudolfWeeber
and
claude
committed
cedc374
Merge branch 'l3_fixes' into comm
RudolfWeeber
committed
4c7945a

Commits on Aug 3, 2026

Merge branch 'comm_tmp' into worktree-comm
RudolfWeeber
committed
1a56a9e
core/ghosts: exception-safe ghost-reduce finish; plug buffer-pool leak

RudolfWeeber
and
claude
committed
433a96a
core: share Gay-Berne activity predicate; assert torque scatter implies TORQUE reduce

RudolfWeeber
and
claude
committed
60961fc