A disk based KV store (based on Bitcask implementation)
MIT License
A disk based key-value store based on Bitcask.
BarrelDB is a Golang implementation of Bitcask by Riak paper and aims to closely follow the spec.
Bitcask is based on a log-structured hash table to store key-value data on disk. It opens a "datafile" (term used for a Bitcask DB file) in an append-only mode and all the writes are sequentially written to this file. Additionally, it also updates an in-memory hash table which maps the key with the offset of the record in the file. This clever yet simple design decision makes it possible to retrieve records from the disk using a single disk seek.
You can refer to Writing a disk-based key-value store in Golang blog post to read about the internals of Bitcask which also explains how BarrelDB works.
import (
"github.com/mr-karan/barreldb"
)
barrel, _ := barrel.Init(barrel.WithDir("data/"))
// Set a key.
barrel.Put("hello", []byte("world"))
// Fetch the key.
v, _ := barrel.Get("hello")
// Delete a key.
barrel.Delete("hello")
// Set with expiry.
barrel.PutEx("hello", []byte("world"), time.Second * 3)
For a complete example, visit examples.
barreldb
implements the API over a simple Redis-compatible server (barreldb
):
127.0.0.1:6379> set hello world
OK
127.0.0.1:6379> get hello
"world"
127.0.0.1:6379> set goodbye world 10s
OK
127.0.0.1:6379> get goodbye
"world"
127.0.0.1:6379> get goodbye
ERR: invalid key: key is already expired
Method | Description |
---|---|
Init(...cfg) *Barrel |
Returns a new instance of Barrel . Additional options can be passed like WithDir , WithReadOnly etc. |
Put(string, []byte) error |
Store a key and value in the datastore. |
PutEx(string, []byte, time.Duration) error |
Store a key and value with expiry in the datastore. |
Get(string) []byte,error |
Retrieve a value by key from the datastore. |
Delete(string) error |
Delete a key from the datastore. |
Keys() []string |
List all keys in the datastore. |
Len() int |
Return the total count of keys in the datastore. |
Fold(func(string) error) error |
Fold over all K/V pairs in a Bitcask datastore. Calls a function on each key inside the datastore. |
Sync() error |
Force any writes to sync to disk. |
Shutdown() error |
Close a data store and flush all pending writes. Removes any lock on the data directory as well. |
Using make bench
:
go test -bench=. -benchmem ./...
HELLO
goos: linux
goarch: amd64
pkg: github.com/mr-karan/barreldb
cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
BenchmarkPut/DisableSync-8 385432 3712 ns/op 1103.48 MB/s 88 B/op 4 allocs/op
BenchmarkPut/AlwaysSync-8 222 5510563 ns/op 0.74 MB/s 115 B/op 4 allocs/op
BenchmarkGet-8 840627 1304 ns/op 3142.20 MB/s 4976 B/op 5 allocs/op
PASS
ok github.com/mr-karan/barreldb 10.751s
Using redis-benchmark
:
$ redis-benchmark -p 6379 -t set -n 10000 -r 100000000
Summary:
throughput summary: 140845.06 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.196 0.016 0.175 0.255 1.031 2.455
$ redis-benchmark -p 6379 -t set -n 200000 -r 100000000
Summary:
throughput summary: 143678.17 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.184 0.016 0.183 0.223 0.455 2.183
$ redis-benchmark -p 6379 -t get -n 100000 -r 100000000
Summary:
throughput summary: 170068.03 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.153 0.040 0.143 0.199 0.367 1.447