You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
##### KeyDB is now a part of Snap Inc! Check out the announcement [here](https://docs.keydb.dev/news/2022/05/12/keydb-joins-snap)
6
+
7
+
##### [Release v6.3.0](https://github.com/EQ-Alpha/KeyDB/releases/tag/v6.3.0) is here with major improvements as we consolodate our Open Source and Enterprise offerings into a single BSD-3 licensed project. See our [roadmap](https://docs.keydb.dev/docs/coming-soon) for details.
8
+
5
9
##### Want to extend KeyDB with Javascript? Try [ModJS](https://github.com/JohnSully/ModJS)
6
10
7
11
##### Need Help? Check out our extensive [documentation](https://docs.keydb.dev).
8
12
9
-
##### NEW!!! KeyDB now has a Slack Community Workspace. Click [here](https://docs.keydb.dev/slack/) to learn more and join the KeyDB Community Slack workspace.
13
+
##### KeyDB is on Slack. Click [here](https://docs.keydb.dev/slack/) to learn more and join the KeyDB Community Slack workspace.
10
14
11
15
What is KeyDB?
12
16
--------------
13
17
14
-
KeyDB is a high performance fork of Redis with a focus on multithreading, memory efficiency, and high throughput. In addition to multithreading, KeyDB also has features only available in Redis Enterprise such as [Active Replication](https://github.com/JohnSully/KeyDB/wiki/Active-Replication), [FLASH storage](https://github.com/JohnSully/KeyDB/wiki/FLASH-Storage) support, and some not available at all such as direct backup to AWS S3.
18
+
KeyDB is a high performance fork of Redis with a focus on multithreading, memory efficiency, and high throughput. In addition to performance improvements, KeyDB offers features such as Active Replication, FLASH Storage and Subkey Expires. KeyDB has a MVCC architecture that allows you to execute queries such as KEYS and SCAN without blocking the database and degrading performance.
15
19
16
20
KeyDB maintains full compatibility with the Redis protocol, modules, and scripts. This includes the atomicity guarantees for scripts and transactions. Because KeyDB keeps in sync with Redis development KeyDB is a superset of Redis functionality, making KeyDB a drop in replacement for existing Redis deployments.
17
21
@@ -30,48 +34,75 @@ KeyDB has a different philosophy on how the codebase should evolve. We feel tha
30
34
31
35
Because of this difference of opinion features which are right for KeyDB may not be appropriate for Redis. A fork allows us to explore this new development path and implement features which may never be a part of Redis. KeyDB keeps in sync with upstream Redis changes, and where applicable we upstream bug fixes and changes. It is our hope that the two projects can continue to grow and learn from each other.
32
36
37
+
Project Support
38
+
-------------------
39
+
40
+
The KeyDB team maintains this project as part of Snap Inc. KeyDB is used by Snap as part of its caching infrastructure and is fully open sourced. There is no separate commercial product and no paid support options available. We really value collaborating with the open source community and welcome PRs, bug reports, and open discussion. For community support or to get involved further with the project check out our community support options [here](https://docs.keydb.dev/docs/support) (slack, forum, meetup, github issues). Our team monitors these channlels regularly.
41
+
42
+
33
43
Additional Resources
34
44
--------------------
35
45
36
-
Check out KeyDB's[Docker Image](https://hub.docker.com/r/eqalpha/keydb)
46
+
Try the KeyDB [Docker Image](https://hub.docker.com/r/eqalpha/keydb)
37
47
38
48
Join us on [Slack](https://docs.keydb.dev/slack/)
39
49
40
-
Post to the [Community Forum](https://community.keydb.dev)
50
+
Learn more using KeyDB's extensive [documentation](https://docs.keydb.dev)
41
51
42
-
Learn more through KeyDB's [Documentation & Learning Center](https://docs.keydb.dev)
52
+
Post to our [Community Forum](https://community.keydb.dev)
53
+
54
+
See the [KeyDB Roadmap](https://docs.keydb.dev/docs/coming-soon) to see what's in store
43
55
44
56
45
57
Benchmarking KeyDB
46
58
------------------
47
59
48
-
Please note keydb-benchmark and redis-benchmark are currently single threaded and too slow to properly benchmark KeyDB. We recommend using a redis cluster benchmark tool such as [memtier](https://github.com/RedisLabs/memtier_benchmark). Please ensure your machine has enough cores for both KeyDB and memteir if testing locally. KeyDB expects exclusive use of any cores assigned to it.
60
+
Please note keydb-benchmark and redis-benchmark are currently single threaded and too slow to properly benchmark KeyDB. We recommend using a redis cluster benchmark tool such as [memtier](https://github.com/RedisLabs/memtier_benchmark). Please ensure your machine has enough cores for both KeyDB and memtier if testing locally. KeyDB expects exclusive use of any cores assigned to it.
49
61
50
-
For more details on how we benchmarked KeyDB along with performance numbers check out our blog post: [Redis Should Be Multithreaded](https://medium.com/@john_63123/redis-should-be-multi-threaded-e28319cab744?source=friends_link&sk=7ce8e9fe3ec8224a4d27ef075d085457)
51
62
52
63
New Configuration Options
53
64
-------------------------
54
65
55
-
With new features comes new options:
66
+
With new features comes new options. All other configuration options behave as you'd expect. Your existing configuration files should continue to work unchanged.
56
67
68
+
```
57
69
server-threads N
58
70
server-thread-affinity [true/false]
71
+
```
72
+
The number of threads used to serve requests. This should be related to the number of queues available in your network hardware, *not* the number of cores on your
73
+
machine. Because KeyDB uses spinlocks to reduce latency; making this too high will reduce performance. We recommend using 4 here. By default this is set to two.
59
74
60
-
The number of threads used to serve requests. This should be related to the number of queues available in your network hardware, *not* the number of cores on your machine. Because KeyDB uses spinlocks to reduce latency; making this too high will reduce performance. We recommend using 4 here. By default this is set to one.
61
-
62
-
scratch-file-path /path
75
+
```
76
+
min-clients-per-thread 50
77
+
```
78
+
The minimum number of clients on a thread before KeyDB assigns new connections to a different thread. Tuning this parameter is a tradeoff between locking overhead and distributing the workload over multiple cores
63
79
64
-
If you would like to use the [FLASH backed](https://github.com/JohnSully/KeyDB/wiki/FLASH-Storage) storage this option configures the directory for KeyDB's temporary files. This feature relies on snapshotting to work so must be used on a BTRFS filesystem. ZFS may also work but is untested. With this feature KeyDB will use RAM as a cache and page to disk as necessary. NOTE: This requires special compilation options, see Building KeyDB below.
65
-
66
-
db-s3-object /path/to/bucket
80
+
```
81
+
replica-weighting-factor 2
82
+
```
83
+
KeyDB will attempt to balance clients across threads evenly; However, replica clients are usually much more expensive than a normal client, and so KeyDB will try to assign fewer clients to threads with a replica. The weighting factor below is intented to help tune this behavior. A replica weighting factor of 2 means we treat a replica as the equivalent of two normal clients. Adjusting this value may improve performance when replication is used. The best weighting is workload specific - e.g. read heavy workloads should set this to 1. Very write heavy workloads may benefit from higher numbers.
67
84
68
-
If you would like KeyDB to dump and load directly to AWS S3 this option specifies the bucket. Using this option with the traditional RDB options will result in KeyDB backing up twice to both locations. If both are specified KeyDB will first attempt to load from the local dump file and if that fails load from S3. This requires the AWS CLI tools to be installed and configured which are used under the hood to transfer the data.
85
+
```
86
+
active-client-balancing yes
87
+
```
88
+
Should KeyDB make active attempts at balancing clients across threads? This can impact performance accepting new clients. By default this is enabled. If disabled there is still a best effort from the kernel to distribute across threads with SO_REUSEPORT but it will not be as fair. By default this is enabled
69
89
90
+
```
70
91
active-replica yes
71
-
92
+
```
72
93
If you are using active-active replication set `active-replica` option to “yes”. This will enable both instances to accept reads and writes while remaining synced. [Click here](https://docs.keydb.dev/docs/active-rep/) to see more on active-rep in our docs section. There are also [docker examples](https://docs.keydb.dev/docs/docker-active-rep/) on docs.
73
94
74
-
All other configuration options behave as you'd expect. Your existing configuration files should continue to work unchanged.
95
+
```
96
+
multi-master-no-forward no
97
+
```
98
+
Avoid forwarding RREPLAY messages to other masters? WARNING: This setting is dangerous! You must be certain all masters are connected to eachother in a true mesh topology or data loss will occur! This command can be used to reduce multimaster bus traffic
99
+
100
+
101
+
```
102
+
scratch-file-path /path
103
+
```
104
+
If you would like KeyDB to dump and load directly to AWS S3 this option specifies the bucket. Using this option with the traditional RDB options will result in KeyDB backing up twice to both locations. If both are specified KeyDB will first attempt to load from the local dump file and if that fails load from S3. This requires the AWS CLI tools to be installed and configured which are used under the hood to transfer the data.
105
+
75
106
76
107
Building KeyDB
77
108
--------------
@@ -104,6 +135,10 @@ To append a suffix to KeyDB program names, use:
104
135
***Note that the following dependencies may be needed:
Running the tests with TLS enabled (you will need `tcl-tls`
108
143
installed):
109
144
@@ -270,24 +305,6 @@ KeyDB works by running the normal Redis event loop on multiple threads. Network
270
305
271
306
Unlike most databases the core data structure is the fastest part of the system. Most of the query time comes from parsing the REPL protocol and copying data to/from the network.
272
307
273
-
Future work:
274
-
- Allow rebalancing of connections to different threads after the connection
275
-
- Allow multiple readers access to the hashtable concurrently
276
-
277
-
Docker Build
278
-
------------
279
-
Build the latest binaries from the github unstable branch within a docker container. Note this is built for Ubuntu 18.04.
280
-
Simply make a directory you would like to have the latest binaries dumped in, then run the following commmand with your updated path:
281
-
```
282
-
$ docker run -it --rm -v /path-to-dump-binaries:/keydb_bin eqalpha/keydb-build-bin
283
-
```
284
-
You should receive the following files: keydb-benchmark, keydb-check-aof, keydb-check-rdb, keydb-cli, keydb-sentinel, keydb-server
285
-
286
-
If you are looking to enable flash support with the build (make MALLOC=memkind) then use the following command:
287
-
```
288
-
$ docker run -it --rm -v /path-to-dump-binaries:/keydb_bin eqalpha/keydb-build-bin:flash
289
-
```
290
-
Please note that you will need libcurl4-openssl-dev in order to run keydb. With flash version you may need libnuma-dev and libtool installed in order to run the binaries. Keep this in mind especially when running in a container. For a copy of all our Dockerfiles, please see them on [docs](https://docs.keydb.dev/docs/dockerfiles/).
0 commit comments