feat(cni): add dual-stack IPv6/IPv4 IPAM allocators#257
Open
privateip wants to merge 2 commits into
Open
Conversation
PR datum-cloud/enhancements#740 moves galactic-cni to a hierarchical, dual-stack tenant addressing model (VPC /48 -> region /64 -> endpoint /96 for IPv6; a shared IPv4 /9 -> macro-region /12 -> site /20 -> endpoint /32). This lands the allocator-level groundwork: - PoolAllocator's DefaultSubnetLen moves from 80 to 96, matching the new endpoint-subnet size drawn from a /64 region pool instead of a /48 VPC pool; the local-IPAM fallback constants in cni.go and docs/cni/configuration.md are updated to match. - New IPv4PoolAllocator allocates individual /32s from an IPv4 pool, reserving the network, gateway, and last two addresses using pool-relative offsets (works for any prefix length, not just /24). - New DualStackAllocator wraps both allocators behind a single Allocate() call; the IPv4 side is optional so IPv6-only VPCs are unaffected. Also fixes a pre-existing bug in incSubnet: it zeroed the same byte it had just incremented, so any PoolAllocator could never produce more than two distinct subnets from one instance before spinning forever on the third Allocate() call. This had no observable effect under normal CNI usage (each invocation is a fresh process/allocator) but blocks DualStackAllocator's exhaustion test, which allocates repeatedly from one long-lived instance. NAD schema changes, CNI config wiring, and BGP advertisement updates that consume these allocators are separate, later phases. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Each CNI ADD/DEL is a separate OS process, so IPv4PoolAllocator's in-memory sync.Map never guarded against concurrent invocations. PR #740's shared site-wide IPv4 /20 makes that reachable: two pods in different VPCs on the same node can race to allocate from the identical pool. Persist allocations as marker files under a pool-scoped state directory and guard reads/writes with a cross-process flock, so concurrent allocator instances never hand out the same address. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 of the
datum-cloud/enhancements#740tenant addressing model — the allocator-level groundwork for dual-stack (IPv6 + IPv4) endpoint IP assignment:PoolAllocator'sDefaultSubnetLenmoves from80to96, matching the new endpoint-subnet size drawn from a/64region pool instead of a/48VPC pool. Local-IPAM fallback constants (cni.go) anddocs/cni/configuration.mdupdated to match.IPv4PoolAllocator(internal/cni/ipam/ipv4.go) allocates individual/32s from an IPv4 pool, reserving the network, gateway, and last two addresses via pool-relative offsets (correct for any prefix length, not hardcoded to/24).DualStackAllocator(internal/cni/ipam/dualstack.go) wraps both allocators behind oneAllocate()call; the IPv4 side is optional, so IPv6-only VPCs are unaffected.incSubnet: it zeroed the same byte it had just incremented, so aPoolAllocatorcould never produce more than two distinct subnets from one instance before spinning forever on the thirdAllocate()call. No observable effect under normal CNI usage (each invocation is a fresh process/allocator), but it blocksDualStackAllocator's exhaustion test, which allocates repeatedly from one long-lived instance.NAD schema changes, CNI config wiring (
PluginConf,allocateIPAM), and BGP advertisement updates that consume these allocators are separate, later phases — not included here.Test plan
go build ./...go vet ./...go test ./internal/cni/ipam/... -race -v(new + existing tests)go test ./...(full repo)task lint(0 issues)Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com