Wer Ordnung hält, ist nur zu faul zum Suchen.
(If you keep things tidily ordered, you’re just too lazy to go searching.)
— German proverb, Designing Data-Intensive Applications
Distributed Key-Value Store in Go
- HTTP Server API (with
fasthttp)-
Put(key, value) -
Get(key) -> value -
Delete(key)
-
- Storage Engines
- Hash Map (with
sync.RWMutex) - LevelDB
- LSM-Tree
- SSTable
- Bloom Filter
- B Tree
- Hash Map (with
- Single Leader Replication
- Service Discovery
- Leader Election with Raft Consensus Protocol
- Leaderless Replication (peer-to-peer coordination)
- Chaos Testing (like Netflix's Chaos Monkey)
- Range Queries
- Sharding
Enable the RBAC addon on microk8s and create a role binding which will grant the default service account view permissions.
microk8s kubectl create clusterrolebinding default-view-impulse --clusterrole=view --serviceaccount=impulse:default > ./scripts/concurrent.sh go build -o impulse ./... HASH_MAP ======== PUT Running 10s test @ http://127.0.0.1:3000/foo 100 goroutine(s) running concurrently 224432 requests in 9.93928912s, 12.20MB read Requests/sec: 22580.29 Transfer/sec: 1.23MB Avg Req Time: 4.428641ms Fastest Request: 120.027µs Slowest Request: 33.188626ms Number of Errors: 0 GET Running 10s test @ http://127.0.0.1:3000/foo 100 goroutine(s) running concurrently 249859 requests in 9.924716107s, 28.59MB read Requests/sec: 25175.43 Transfer/sec: 2.88MB Avg Req Time: 3.972126ms Fastest Request: 99.222µs Slowest Request: 51.840697ms Number of Errors: 0 DELETE Running 10s test @ http://127.0.0.1:3000/foo 100 goroutine(s) running concurrently 234442 requests in 9.951180146s, 12.74MB read Requests/sec: 23559.22 Transfer/sec: 1.28MB Avg Req Time: 4.244623ms Fastest Request: 112.367µs Slowest Request: 33.148716ms Number of Errors: 0 LEVELDB ======= PUT Running 10s test @ http://127.0.0.1:3000/foo 100 goroutine(s) running concurrently 229573 requests in 9.940867386s, 12.48MB read Requests/sec: 23093.86 Transfer/sec: 1.26MB Avg Req Time: 4.330155ms Fastest Request: 112.35µs Slowest Request: 37.433849ms Number of Errors: 0 GET Running 10s test @ http://127.0.0.1:3000/foo 100 goroutine(s) running concurrently 248876 requests in 9.9313757s, 28.48MB read Requests/sec: 25059.57 Transfer/sec: 2.87MB Avg Req Time: 3.990491ms Fastest Request: 96.86µs Slowest Request: 43.956081ms Number of Errors: 0 DELETE Running 10s test @ http://127.0.0.1:3000/foo 100 goroutine(s) running concurrently 234623 requests in 9.952669339s, 12.75MB read Requests/sec: 23573.88 Transfer/sec: 1.28MB Avg Req Time: 4.241983ms Fastest Request: 110.965µs Slowest Request: 31.113818ms Number of Errors: 0> N=2048 ./scripts/sequential_test.sh go build -o impulse ./... HASH_MAP ======== PUT: 104.59 requests per second GET (present): 96.58 requests per second DELETE: 104.70 requests per second GET (missing): 96.21 requests per second LEVELDB ======= PUT: 104.58 requests per second GET (present): 95.94 requests per second DELETE: 104.30 requests per second GET (missing): 95.17 requests per second