redis-lite

A production-inspired, concurrent in-memory key-value store written in Go — built from scratch using only the standard library.

Demonstrates: TCP networking, goroutine-per-connection concurrency, sharded RWMutex locking, RESP protocol parsing, Append-Only File persistence with crash recovery, atomic counter operations, background TTL expiry, and lock-free performance metrics.

---

Features

Feature	Implementation
RESP v2 + inline protocol	internal/parser
Goroutine-per-connection TCP server	internal/server
256-shard RWMutex store	internal/store
Key TTL with lazy + active expiry	internal/store, internal/expiry
Append-Only File persistence	internal/persistence
Crash recovery via AOF replay	startup sequence in cmd/server
Atomic INCR	store.Incr() with shard lock
List support (LPUSH / LRANGE)	internal/store
Lock-free metrics collection	internal/metrics
Concurrent load benchmarking	bench/

---

Supported Commands

PING [message]
SET key value [EX seconds]
GET key
DEL key [key ...]
EXISTS key [key ...]
EXPIRE key seconds
TTL key
INCR key
LPUSH key value [value ...]
LRANGE key start stop

---

Quick Start

Prerequisites

• Go 1.22+

Build and run

git clone https://github.com/ashna/redis-lite
cd redis-lite
go build -o redis-lite ./cmd/server
./redis-lite -addr :6380 -aof redis-lite.aof

Connect with redis-cli

redis-cli -p 6380
127.0.0.1:6380> PING
PONG
127.0.0.1:6380> SET name "alice" EX 60
OK
127.0.0.1:6380> GET name
"alice"
127.0.0.1:6380> TTL name
(integer) 58
127.0.0.1:6380> INCR counter
(integer) 1
127.0.0.1:6380> LPUSH mylist a b c
(integer) 3
127.0.0.1:6380> LRANGE mylist 0 -1
1) "c"
2) "b"
3) "a"

---

Server Flags

Flag	Default	Description
-addr	:6380	TCP listen address
-aof	redis-lite.aof	AOF file path
-no-aof	false	Disable persistence
-sweep	100ms	Expiry sweep interval

---

Architecture

Client
  │ TCP
  ▼
server.acceptLoop()  — one goroutine per connection
  │
  ▼
parser.ReadCommand()  — RESP array or inline
  │
  ▼
dispatcher.Dispatch()
  ├─▶ store (sharded RWMutex, 256 shards)
  ├─▶ aof.Write()  (non-blocking channel → background fsync)
  └─▶ metrics.Record()  (atomic counters)

See docs/ARCHITECTURE.md for full component diagrams, locking model, data flow, and failure recovery model.

---

Running Tests

# All unit and integration tests
go test ./...

# With race detector (required before any PR)
go test -race ./...

# Coverage report
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

---

Benchmarks

Standard library benchmarks

go test -bench=. -benchmem -count=3 ./bench/

Example output (Apple M1, Go 1.22):

BenchmarkSetParallel-8    30000000    39 ns/op    48 B/op    1 allocs/op
BenchmarkGetParallel-8    80000000    14 ns/op     0 B/op    0 allocs/op
BenchmarkMixedReadWrite-8 60000000    18 ns/op     8 B/op    0 allocs/op

Concurrent load generator

# Start the server first
./redis-lite -no-aof

# In another terminal:
go run ./bench -addr :6380 -clients 50 -ops 100000 -cmd SET
go run ./bench -addr :6380 -clients 50 -ops 100000 -cmd GET
go run ./bench -addr :6380 -clients 50 -ops 100000 -cmd INCR

Example output:

─────────────────────────────────────────
Command:      SET
Total ops:    100000
Duration:     312ms
Throughput:   320,512 ops/sec
Avg latency:  155.43 µs
P50 latency:  142.00 µs
P95 latency:  280.00 µs
P99 latency:  510.00 µs
─────────────────────────────────────────

---

Project Structure

redis-lite/
├── cmd/
│   └── server/
│       └── main.go          # Entrypoint — wires all components
├── internal/
│   ├── store/
│   │   ├── store.go         # 256-shard RWMutex key-value store
│   │   └── errors.go        # Sentinel errors (WRONGTYPE, NOTINTEGER)
│   ├── parser/
│   │   ├── parser.go        # RESP v2 + inline command decoder
│   │   └── writer.go        # RESP response encoder
│   ├── dispatcher/
│   │   └── dispatcher.go    # Command router + AOF persistence trigger
│   ├── persistence/
│   │   └── aof.go           # Append-only file writer + replay
│   ├── expiry/
│   │   └── worker.go        # Background TTL sweep goroutine
│   ├── server/
│   │   ├── server.go        # TCP accept loop + graceful shutdown
│   │   ├── handler.go       # Per-connection goroutine
│   │   └── addr.go          # Addr() helper for testing
│   └── metrics/
│       └── metrics.go       # Lock-free per-command latency tracking
├── tests/
│   ├── store_test.go        # Store unit + race tests
│   ├── parser_test.go       # Parser unit tests
│   ├── persistence_test.go  # AOF write + replay integration tests
│   └── integration_test.go  # Full TCP round-trip tests
├── bench/
│   ├── bench.go             # Concurrent load generator CLI
│   └── store_bench_test.go  # go test -bench benchmarks
├── docs/
│   └── ARCHITECTURE.md      # Diagrams, concurrency model, data flow
├── go.mod
└── README.md

---

Design Decisions

Why 256 shards?
The birthday bound means two keys collide (hit the same shard) with ~1% probability when there are 3,200 keys. Increasing to 512 gives diminishing returns vs. memory cost (each shard holds a map header + RWMutex = ~56 bytes).

Why RESP + inline?
Inline support lets telnet and nc work as debug clients. RESP array support is required for redis-cli and all Redis client libraries. Both are decoded in the same parser pass — no branching overhead.

Why a buffered channel for AOF writes?
The alternative (mutex on the file handle) forces every write command to acquire a global lock and potentially stall on a syscall. The channel decouples the hot path entirely — under normal load the channel is empty and writes are free. Under extreme load, writes are dropped rather than stalling clients, which is the correct trade-off for a cache-tier store.

Why no external dependencies?
Standard library only means zero supply-chain risk, reproducible builds with no module proxy, and a clean demonstration that fundamental systems concepts don't require frameworks.

---

Limitations vs Production Redis

Aspect	redis-lite	Redis
Protocol	RESP v2 (subset)	RESP v2 + v3
Persistence	AOF only	RDB + AOF
Replication	None	Leader-replica
Cluster	None	Redis Cluster
Pub/Sub	None	Full support
Lua scripting	None	Full support
Max keyspace	RAM-bound	RAM-bound

---

Scalability Roadmap

1. RDB snapshots — periodic binary serialisation for faster startup
2. Replication — leader broadcasts AOF stream to replicas
3. CONFIG command — runtime reconfiguration without restart
4. MULTI/EXEC transactions — per-client command queuing
5. Cluster mode — consistent hashing across nodes with gossip protocol
6. Prometheus metrics endpoint — expose ops/sec, p99 latency, key count

---

License

MIT