High-Performance Real-Time Matchmaking Engine

A lock-free, SIMD-vectorized matchmaking server for real-time applications: competitive games, instant team formation, and high-concurrency pairing workloads. Players connect over WebSocket, submit a score and acceptable match range, and the engine pairs them against eligible opponents using batched vectorized comparisons on the CPU.

This is a systems-engineering project. Every component was chosen to solve a specific throughput or latency bottleneck under real load testing, not a tutorial or demo.

Benchmark

Validated with k6 WebSocket load tests. The second round, run against a single server node:

Metric	Result
Concurrent WebSocket connections	3,000
Total messages sent	761,394
Total messages processed server-side	19,768,488
Amplification (processing per request)	~26x
Send rate	~2,677 messages/sec
Avg connection latency	1.05s
P95 connection latency	1.92s
Bandwidth	~5.5 MB/s
Test duration	4m 44s

The server maintained all 3,000 connections and processed nearly 20 million match operations under sustained load with no crashes or OOM events.

The Engineering Problem

Real-time matchmaking is deceptively hard at scale:

1. Sheer comparison volume. With N players in the pool, a naive O(N²) comparison across all pairs becomes the bottleneck long before the network does. At 3,000 concurrent players each submitting every 100ms, the engine faces ~760k incoming events per test run that each fan out into dozens of comparisons.
2. Latency-sensitive state. Every player has a live WebSocket connection waiting for a match result. The engine must pair players and push notifications back within a tight window, or the match is already stale.
3. Backpressure under burst. Load is bursty. A naive queue unbounded queues and OOMs; a naive drop loses players silently. The engine needs to throttle gracefully without crashing.
4. False sharing in hot paths. Mutable state shared between threads sits on cache lines. Without padding, CAS operations on adjacent fields invalidate each other's cache lines and throughput collapses.

Architecture

                            ┌─────────────────────────────┐
   Player A ────ws──────────▶│       Netty WS Server        │
   Player B ────ws──────────▶│   (NIO, 1 boss + N workers)  │
   Player C ────ws──────────▶│   port 8889, path /ws        │
   ...                       └──────────────┬──────────────┘
                                           │ submitEvent()
                                           ▼
                    ┌───────────────────────────────────────┐
                    │      ShardedChannelRegistry            │
                    │  128-way sharded ConcurrentHashMap     │
                    │  (key=channelId → ctx lookup)         │
                    └───────────────────────────────────────┘
                                           │
                                           ▼
       ┌────────────────────────────────────────────────────────┐
       │              EnhancedMatchEngine                        │
       │                                                        │
       │  score → bucket   (numBuckets = maxScore/bucketSize)   │
       │                                                        │
       │  bucket[0]  ── Agrona ManyToOneConcurrentArrayQueue    │
       │  bucket[1]  ── Agrona ManyToOneConcurrentArrayQueue    │
       │  ...                                                   │
       │  bucket[n]  ── Agrona ManyToOneConcurrentArrayQueue    │
       │                                                        │
       │  VarHandle CAS state per bucket (lock-free acquire)    │
       │  Reactor Flux.parallel() processes buckets concurrently│
       │  scheduleWithFixedDelay runCycle (every 200ms)         │
       │  Backpressure: pendingEvents counter + heap threshold  │
       │  Low-concurrency fallback: doGlobalMatch() sweep       │
       └───────────────────────┬────────────────────────────────┘
                               │ processBatch(buf, count)
                               ▼
       ┌────────────────────────────────────────────────────────┐
       │            VectorizedMatchPipeline                       │
       │                                                        │
       │  1. Extract scores/ranges into int[]                   │
       │  2. Sort by score, keep original index                 │
       │  3. Binary search to find candidate window per player   │
       │  4. SIMD compare (IntVector SPECIES_256):              │
       │     |scoreJ - scoreI| <= rangeI  AND  <= rangeJ        │
       │  5. VectorMask → first matching lane → CAS markMatched │
       │  6. Emit MatchPair, return unmatched to bucket          │
       └───────────────────────┬────────────────────────────────┘
                               │ emitPairs()
                               ▼
       ┌────────────────────────────────────────────────────────┐
       │       DisruptorNotificationService                      │
       │                                                        │
       │  LMAX Disruptor RingBuffer (ProducerType.MULTI)        │
       │  BusySpinWaitStrategy, single-consumer                 │
       │  onEvent → tryNotify() CAS state machine               │
       │  → reactive WebSocket push (Mono.when(sendA, sendB))   │
       │  Failure: markFailure() → retry/drop with audit trail  │
       └────────────────────────────────────────────────────────┘

Key Design Decisions

1. Score-bucketed Agrona queues instead of a single global queue

Players are partitioned into buckets by score (score / bucketSize). Each bucket is a lock-free ManyToOneConcurrentArrayQueue. This gives two wins:

• Parallelism without locks. Reactor Flux.parallel() dispatches buckets to separate threads. Each bucket's CAS state (VarHandle on int[]) prevents two threads from draining the same bucket, with zero lock contention.
• Smaller search space. Matching only needs to compare players in nearby score buckets, not the entire pool. This turns the N² problem into a set of much smaller local problems.

2. JDK 17 Vector API (SIMD) for batch comparison

Instead of comparing candidate pairs one-by-one in a loop, VectorizedMatchPipeline loads 8 integers at a time (256-bit IntVector.SPECIES_256) and does the range check |scoreJ - scoreI| ≤ rangeI ∧ |scoreJ - scoreI| ≤ rangeJ as a single vector operation. A VectorMask then identifies the first matching lane in O(1).

Combined with a sort + binary search to narrow the candidate window first, this replaces the inner comparison loop with SIMD-batched execution that the JIT can lower to hardware vector instructions.

3. Disruptor for notification, not a thread pool

When two players are matched, the result is published to an LMAX Disruptor RingBuffer (ProducerType.MULTI, BusySpinWaitStrategy). A single consumer thread drains the ring and pushes results over WebSocket using reactive Mono.when(). This avoids thread-pool queueing overhead and context switches in the hottest path, and the ring buffer provides natural backpressure when notifications can't keep up.

4. False-sharing prevention via manual cache-line padding

MatchEvent and MatchPair both use manual long padding fields (pad1 through pad7) to ensure that the VarHandle-operated state field occupies its own 64-byte cache line. Without this, CAS operations on adjacent event objects in an array would invalidate each other's cache lines and collapse throughput under contention.

5. 128-way sharded channel registry

The ShardedChannelRegistry splits the channel map into 128 ConcurrentHashMap shards instead of one global map. This reduces lock contention on the connection lookup path (which fires for every match notification) from a single map's stripe contention to 128 independent partitions.

6. Backpressure via pending-event counter and heap threshold

submitEvent() increments an AtomicLong pending counter. If it exceeds 20,000, or if heap usage crosses 80% (polled via MemoryMXBean), events are rejected rather than queued. This is a deliberate crash-prevention trade: it's better to drop or defer a match request than to OOM the JVM under burst load.

7. Low-concurrency fallback path

A second scheduler monitors a per-second match counter. If throughput drops below 10 matches/sec (off-peak or cold start), it triggers a doGlobalMatch() sweep across all buckets. This ensures players aren't stranded waiting for the next 200ms cycle when the system is idle.

Tech Stack

Component	Technology	Version
Runtime	JDK	17 (--enable-preview for Vector API)
Framework	Spring Boot	3.0.2
Async transport	Netty	4.1.115.Final
Lock-free queues	Agrona	1.23.0
SIMD	JDK Incubator Vector API	IntVector SPECIES_256
Event bus	LMAX Disruptor	3.4.2
Reactive	Project Reactor	3.7.2
Load testing	k6	WebSocket experimental

Quick Start

Prerequisites

• JDK 17+
• Maven 3.6+

Build

git clone https://github.com/xuzhaorui/matching.git
cd matching
mvn clean package -DskipTests

Run

The Vector API requires preview features and the incubator module:

java \
  --add-exports=java.base/jdk.internal.vm.annotation=ALL-UNNAMED \
  --add-modules=jdk.incubator.vector \
  --enable-preview \
  -XX:+UseParallelGC \
  -XX:+AlwaysPreTouch \
  -XX:MaxRAMPercentage=80 \
  -jar target/match-1.jar

Server starts on port 8889, WebSocket endpoint at ws://localhost:8889/ws.

Protocol

Send a JSON message over WebSocket:

{
  "mode": "match",
  "username": "player_1",
  "score": 750,
  "matchRange": 100
}

When a match is found, both players receive the result on their WebSocket connection.

Load Testing with k6

Install k6, then run the included load test profile:

k6 run loadtest.js

Sample test script and the full results from the second benchmark round are documented in loadtest/.

Project Structure

matching/
├── src/main/java/com/match/
│   ├── MatchApplication.java              # Spring Boot entry point
│   ├── EnhancedMatchEngine.java           # Core engine: bucketing, scheduling, backpressure
│   ├── VectorizedMatchPipeline.java       # SIMD batch matching pipeline
│   ├── DisruptorNotificationService.java  # LMAX Disruptor notification bus
│   ├── ShardedChannelRegistry.java        # 128-way sharded connection registry
│   ├── WebSocketServer.java               # Netty NIO WebSocket server
│   ├── WebSocketHandler.java              # Message routing handler
│   ├── MatchEvent.java                    # Padded, CAS-state event object
│   ├── MatchPair.java                     # Padded, CAS-state match result
│   ├── Player.java                        # Player data carrier
│   ├── MatchProperties.java               # Configurable engine parameters
│   └── MatchSystemAutoConfiguration.java  # Auto-config
├── src/test/java/com/match/
│   ├── DisruptorNotificationServiceTest.java
│   └── MatchApplicationTests.java
├── src/main/resources/
│   ├── application.yaml
│   └── static/index.html
└── pom.xml

What This Project Demonstrates

This is not a CRUD app or a framework tutorial. It demonstrates:

• Low-level JVM performance engineering: Vector API, VarHandle CAS, cache-line padding, GC tuning, and off-heap-aware memory management under load.
• Lock-free and wait-free concurrency design: Agrona queues, Disruptor ring buffers, and CAS-based state machines, chosen deliberately over locks where contention is hot.
• Real-time systems architecture: end-to-end pipeline from WebSocket ingest through match computation to push notification, with backpressure and graceful degradation.
• Evidence-driven optimization: every component was validated under k6 load tests with concrete throughput and latency numbers, not theoretical reasoning.

License

MIT