Describe the bug
We are seeing a problem with KeyDB 6.3.4 where memory eviction does not work correctly under a large dataset.
When the dataset grows to around 1.7 million keys and disk usage reaches about 300 GB, the server continuously reports:
Failed to evict keys, falling back to flushing entire cache. Consider increasing maxmemory-samples.
We already increased maxmemory-samples from 16 to 256, but the error still continues.
At the same time, the instance memory usage goes beyond the configured maxmemory limit (44 GB), and memory usage keeps increasing until the whole server memory is exhausted, eventually causing OOM / service failure.
This makes it look like eviction is not functioning as expected when flash storage is enabled and the dataset becomes large.
Observed behavior
Dataset grows to about 1.7 million keys
Disk usage grows to about 300 GB
KeyDB repeatedly logs:
Failed to evict keys, falling back to flushing entire cache. Consider increasing maxmemory-samples.
Increasing maxmemory-samples from 16 to 256 does not solve the issue
Actual memory usage exceeds maxmemory 44gb
Memory continues to grow instead of being controlled by eviction
Eventually the server runs out of memory
Expected behavior
When memory usage reaches maxmemory, KeyDB should evict keys according to allkeys-lru and keep memory usage under control.
It should not continue growing beyond the configured memory limit until the server OOMs.
Additional information
Environment
KeyDB version: 6.3.4
OS: Ubuntu 22.04
Server spec: 8 CPU, 64 GB RAM
Disk: SSD 500 GB
Configuration
bind 0.0.0.0
port 6379
protected-mode yes
tcp-backlog 8192
timeout 0
tcp-keepalive 300
daemonize yes
pidfile /var/run/keydb/keydb-server.pid
supervised no
loglevel notice
logfile /var/log/keydb/keydb-server.log
databases 16
maxclients 10000
server-threads 4
maxmemory 44gb
maxmemory-policy allkeys-lru
maxmemory-samples 256
Flash
storage-provider flash /opt/data1/keydb/flash
dir /opt/data1/keydb
save ""
stop-writes-on-bgsave-error no
rdbcompression yes
rdbchecksum yes
lazyfree-lazy-eviction yes
lazyfree-lazy-expire yes
lazyfree-lazy-server-del yes
slowlog-log-slower-than 10000
slowlog-max-len 1024
latency-monitor-threshold 100
Describe the bug
We are seeing a problem with KeyDB 6.3.4 where memory eviction does not work correctly under a large dataset.
When the dataset grows to around 1.7 million keys and disk usage reaches about 300 GB, the server continuously reports:
Failed to evict keys, falling back to flushing entire cache. Consider increasing maxmemory-samples.
We already increased maxmemory-samples from 16 to 256, but the error still continues.
At the same time, the instance memory usage goes beyond the configured maxmemory limit (44 GB), and memory usage keeps increasing until the whole server memory is exhausted, eventually causing OOM / service failure.
This makes it look like eviction is not functioning as expected when flash storage is enabled and the dataset becomes large.
Observed behavior
Dataset grows to about 1.7 million keys
Disk usage grows to about 300 GB
KeyDB repeatedly logs:
Failed to evict keys, falling back to flushing entire cache. Consider increasing maxmemory-samples.
Increasing maxmemory-samples from 16 to 256 does not solve the issue
Actual memory usage exceeds maxmemory 44gb
Memory continues to grow instead of being controlled by eviction
Eventually the server runs out of memory
Expected behavior
When memory usage reaches maxmemory, KeyDB should evict keys according to allkeys-lru and keep memory usage under control.
It should not continue growing beyond the configured memory limit until the server OOMs.
Additional information
Environment
KeyDB version: 6.3.4
OS: Ubuntu 22.04
Server spec: 8 CPU, 64 GB RAM
Disk: SSD 500 GB
Configuration
bind 0.0.0.0
port 6379
protected-mode yes
tcp-backlog 8192
timeout 0
tcp-keepalive 300
daemonize yes
pidfile /var/run/keydb/keydb-server.pid
supervised no
loglevel notice
logfile /var/log/keydb/keydb-server.log
databases 16
maxclients 10000
server-threads 4
maxmemory 44gb
maxmemory-policy allkeys-lru
maxmemory-samples 256
Flash
storage-provider flash /opt/data1/keydb/flash
dir /opt/data1/keydb
save ""
stop-writes-on-bgsave-error no
rdbcompression yes
rdbchecksum yes
lazyfree-lazy-eviction yes
lazyfree-lazy-expire yes
lazyfree-lazy-server-del yes
slowlog-log-slower-than 10000
slowlog-max-len 1024
latency-monitor-threshold 100