TiKV PHP Client

PHP client for TiKV RawKV API using gRPC extension.

Requirements

PHP >= 8.2
gRPC extension
TiKV cluster with RawKV enabled

Quick Start

# Start TiKV cluster and run example
make up
make example

Installation

composer require crazy-goat/tikv-client

Usage

Basic CRUD Operations

use CrazyGoat\TiKV\Client\RawKv\RawKvClient;

$client = RawKvClient::create(['127.0.0.1:2379']);

// Basic CRUD
$client->put('key', 'value');
$value = $client->get('key');       // 'value'
$client->delete('key');

// Batch operations
$client->batchPut(['k1' => 'v1', 'k2' => 'v2', 'k3' => 'v3']);
$values = $client->batchGet(['k1', 'k2', 'k3']);
$client->batchDelete(['k1', 'k2', 'k3']);

$client->close();

Scanning

// Range scan [startKey, endKey). limit: 0 (the default) returns the whole
// range: the client pages internally and buffers every row, guarded by
// options['maxScanRows'] (default 100000).
$results = $client->scan('start', 'end', limit: 100);
// Returns: [['key' => 'k1', 'value' => 'v1'], ['key' => 'k2', 'value' => 'v2'], ...]

// Prefix scanning (limit: 0 returns all matching keys, buffered)
$results = $client->scanPrefix('user:');

// Lazy scan iterators — constant memory, auto-paginating (page of
// $batchSize rows at a time; batchSize must be 1..10240, default 1024).
// Use these instead of scanPrefix()/scan() when a range may be large.
foreach ($client->scanPrefixIterator('user:', batchSize: 500) as $key => $value) {
    process($key, $value);
}
// $value is null when keyOnly is true
foreach ($client->scanIterator('a', 'b', batchSize: 256, keyOnly: true) as $key => $_) {
    // ...
}

// For an unbounded scan that matches more than options['maxScanRows'] rows
// (default 100000) the client throws ScanLimitExceededException rather than
// silently truncating; the iterators above have no such limit.

// Reverse scan (descending order)
// Note: startKey = upper bound (exclusive), endKey = lower bound (inclusive)
$results = $client->reverseScan('end', 'start', limit: 100);

// Scan multiple non-contiguous ranges
$results = $client->batchScan([['a:', 'a;'], ['b:', 'b;']], eachLimit: 50);

Range Operations

// Delete all keys in range [startKey, endKey)
$client->deleteRange('temp:', 'temp;');

// Delete all keys with a given prefix
$client->deletePrefix('cache:');

TTL (Time-To-Live)

Requires enable-ttl=true in tikv.toml configuration.

Cluster mode is exclusive. enable-ttl = true puts TiKV into V1TTL storage mode, which serves RawKV with TTL but not transactional (TxnKV) requests. A single cluster cannot serve both RawKV-with-TTL and TxnKV.

RawKV with TTL → [storage] enable-ttl = true (see tikv.toml)

RawKV without TTL, or TxnKV → leave enable-ttl unset (see tikv-v1.toml)

This project's own E2E suites reflect the split: RawKV tests run against docker-compose.yml, TxnKV tests against docker-compose.yml + docker-compose.txnkv.yml.

Changing this setting on a live cluster is an operational migration — plan it before adopting either feature.

// Store with expiration (TTL in seconds)
$client->put('session', 'data', ttl: 3600);        // expires in 1 hour

// Get remaining TTL
$remaining = $client->getKeyTTL('session');          // seconds remaining, or null if not found/no TTL

Atomic Operations

Note: Compare-And-Swap (CAS) and Put-If-Absent require atomic mode to be enabled via setAtomicForCAS(true). In atomic mode, the underlying RawPut RPC sends the for_cas flag, which tells TiKV to use the atomic code path. Atomic mode is disabled by default for performance — the non-atomic code path is faster for regular writes. Enable it only when you need CAS semantics.

use CrazyGoat\TiKV\Client\RawKv\CasResult;

// Enable atomic mode for CAS operations
$client->setAtomicForCAS(true);

// Compare-And-Swap (CAS)
$result = $client->compareAndSwap('counter', '1', '2');
if ($result->swapped) {
    echo "Value was swapped from '1' to '2'";
    echo "Previous value: " . ($result->previousValue ?? 'null');
}

// Put if key does not exist (distributed lock pattern)
$existing = $client->putIfAbsent('lock', 'owner-1');
if ($existing === null) {
    echo "Lock acquired!";
} else {
    echo "Lock already held by: $existing";
}

Data Integrity

use CrazyGoat\TiKV\Client\RawKv\ChecksumResult;

// Compute CRC64-XOR checksum over key range
$checksum = $client->checksum('data:', 'data;');
echo "Checksum: {$checksum->checksum}";
echo "Keys: {$checksum->totalKvs}, Bytes: {$checksum->totalBytes}";

Bulk Import (SST Ingest)

// High-throughput bulk load: bypasses the Raft write path, writes
// pre-sorted SST data directly into regions. Optional TTL in seconds.
$client->ingest(['k1' => 'v1', 'k2' => 'v2'], ttl: 3600);

Cluster-wide hazard. For the duration of the call every TiKV store is switched into import mode and switched back on completion. The switch-back runs in a finally block, so exceptions are safe — but a killed process (OOM killer, deploy restart, max_execution_time) leaves the cluster in import mode, degrading it for all clients. Run ingest() only from a dedicated, supervised CLI process, and see Bulk Import (SST Ingest) in the operations guide for the full semantics, the fixed 60 s ingest deadline and the recovery procedure for a cluster stuck in import mode.

Transactions (TxnKV)

Optimistic transactions take write locks during prewrite. The prewrite lock TTL scales with the write-set size — 3000 ms + 10 ms per mutation, capped at 120000 ms — so a large multi-region prewrite cannot outlive its own locks (an expired lock is rolled back by concurrent readers and the commit fails). While the prewrite loop runs, the client automatically heartbeats the primary lock once half of the computed TTL has elapsed (single-region 1PC commits are exempt, and a lock can only be extended between region prewrites).

A transaction that stays open between operations keeps its locks only for the granted TTL, so it must extend them itself with Transaction::heartbeat() before the last granted TTL elapses (10 s is a safe default):

$txn = $txnClient->begin();

$txn->set('account:1', '100');

// ... a long computation or external call ...

$txn->heartbeat(10000); // extend the primary lock TTL by ~10 s

$txn->set('account:2', '0');
$txn->commit();

TLS/SSL Configuration

use CrazyGoat\TiKV\Client\RawKv\RawKvClient;
use CrazyGoat\TiKV\Client\Tls\TlsConfigBuilder;

// Configure TLS with CA certificate only (server verification)
$options = [
    'tls' => [
        'caCertFile' => '/path/to/ca.crt',
        'caCertBaseDir' => '/path/to', // optional: restrict to base directory
    ],
];

// Or with mutual TLS (mTLS) - client certificate authentication
$options = [
    'tls' => [
        'caCertFile' => '/path/to/ca.crt',
        // Optional base-directory restrictions (*BaseDir apply only to their
        // matching file reads; caCertBaseDir → caCertFile, clientCertBaseDir →
        // clientCertFile + clientKeyFile)
        'caCertBaseDir' => '/path/to',
        'clientCertFile' => '/path/to/client.crt',
        'clientKeyFile' => '/path/to/client.key',
        'clientCertBaseDir' => '/path/to',
    ],
];

// For inline PEM content, use the *Pem variants:
$options = [
    'tls' => [
        'caCertPem' => $caPemString,
        'clientCertPem' => $clientCertPemString,
        'clientKeyPem' => $clientKeyPemString,
    ],
];

$client = RawKvClient::create(['tikv.example.com:2379'], options: $options);

PSR-3 Logging

use CrazyGoat\TiKV\Client\RawKv\RawKvClient;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;

// Create a PSR-3 compatible logger
$logger = new Logger('tikv');
$logger->pushHandler(new StreamHandler('php://stdout', Logger::DEBUG));

// Pass logger to client
$client = RawKvClient::create(['127.0.0.1:2379'], logger: $logger);

// The client will log:
// - Connection attempts and failures
// - Retry attempts with backoff information
// - Region cache hits/misses
// - Error conditions

Complete Example

<?php
require 'vendor/autoload.php';

use CrazyGoat\TiKV\Client\RawKv\RawKvClient;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;

// Setup logging
$logger = new Logger('tikv');
$logger->pushHandler(new StreamHandler('php://stderr', Logger::INFO));

// Connect to TiKV
$pdEndpoints = ['127.0.0.1:2379'];
$client = RawKvClient::create($pdEndpoints, logger: $logger);

try {
    // Store user data with TTL
    $client->put('user:123', json_encode(['name' => 'Alice', 'age' => 30]), ttl: 3600);
    $client->put('user:456', json_encode(['name' => 'Bob', 'age' => 25]), ttl: 3600);
    
    // Batch retrieve
    $users = $client->batchGet(['user:123', 'user:456']);
    foreach ($users as $key => $value) {
        if ($value !== null) {
            $data = json_decode($value, true);
            echo "$key: {$data['name']}\n";
        }
    }
    
    // Scan all users with the lazy iterator (constant memory)
    $userCount = 0;
    foreach ($client->scanPrefixIterator('user:') as $_) {
        $userCount++;
    }
    echo "Total users: $userCount\n";
    
    // Check TTL
    $ttl = $client->getKeyTTL('user:123');
    echo "TTL remaining: $ttl seconds\n";
    
    // Enable atomic mode for CAS operations
    $client->setAtomicForCAS(true);
    
    // Atomic counter update
    $client->put('counter', '0');
    $result = $client->compareAndSwap('counter', '0', '1');
    if ($result->swapped) {
        echo "Counter incremented!\n";
    }
    
} finally {
    $client->close();
}

Implemented Operations

Core CRUD

✅ Get / Put / Delete — Single key operations
✅ BatchGet / BatchPut / BatchDelete — Batch operations with parallel execution

Scanning

✅ Scan — Range scan [startKey, endKey) with limit and keyOnly options
✅ ReverseScan — Reverse range scan (native reverse=true)
✅ ScanPrefix — Prefix-based scanning
✅ ScanIterator / ScanPrefixIterator — Lazy auto-paginating scan iterators (ScanIterator), constant memory for large ranges
✅ BatchScan — Multiple non-contiguous range scanning

Range Operations

✅ DeleteRange — Delete all keys in [startKey, endKey)
✅ DeletePrefix — Delete all keys with a given prefix

TTL

✅ PutWithTTL — Store with expiration (seconds)
✅ GetKeyTTL — Get remaining TTL of a key

Atomic Operations

✅ CompareAndSwap — Atomic CAS with CasResult (swapped + previousValue)
✅ PutIfAbsent — Conditional insert (returns existing value or null)

Data Integrity

✅ Checksum — CRC64-XOR checksum over key range with ChecksumResult

Infrastructure

✅ PD Region Discovery — With RegionEpoch support
✅ Region Routing — Direct to correct TiKV node
✅ Region Cache — In-memory caching of region metadata
✅ Store Cache — In-memory caching of store addresses
✅ Retry Logic — Automatic retry with exponential backoff
✅ NotLeader Handling — Automatic leader redirection
✅ Batch Async Execution — Parallel execution across regions
✅ TLS/SSL Support — Server and mutual TLS authentication
✅ PSR-3 Logging — Structured logging with any PSR-3 logger

Project Structure

src/
├── Client/
│   ├── Batch/
│   │   ├── BatchAsyncExecutor.php   # Concurrent fan-out with deadline-bounded collection
│   │   ├── CheckedGrpcFuture.php    # Lazy region-error check around dispatch futures
│   │   └── GrpcFuture.php           # Async gRPC operations
│   ├── Cache/
│   │   ├── RegionCache.php          # Region metadata cache
│   │   └── StoreCache.php           # Store address cache
│   ├── Connection/
│   │   └── PdClient.php             # PD discovery & region routing
│   ├── Grpc/
│   │   └── GrpcClient.php           # Low-level gRPC wrapper
│   ├── RawKv/
│   │   ├── RawKvClient.php          # Main client (20+ operations)
│   │   ├── CasResult.php            # CompareAndSwap result
│   │   ├── ChecksumResult.php       # Checksum result
│   │   ├── ScanIterator.php         # Lazy auto-paginating scan iterator
│   │   └── Dto/                     # Shared DTOs (KeyValue)
│   ├── Region/
│   │   ├── RegionResolver.php       # Region resolution & caching
│   │   ├── RegionContextFactory.php  # Region context factory
│   │   ├── RegionRangeClipper.php   # Range clipping
│   │   ├── RegionGrouper.php        # Group keys by region
│   │   ├── RegionErrorHandler.php   # Region error checking
│   │   └── Dto/                     # Region DTOs (RegionInfo, PeerInfo)
│   ├── Retry/
│   │   └── BackoffType.php          # Retry backoff strategies
│   └── Tls/
│       ├── TlsConfig.php            # TLS configuration
│       └── TlsConfigBuilder.php     # TLS builder
└── Proto/                           # Generated protobuf classes
    ├── Kvrpcpb/                     # TiKV request/response
    ├── Pdpb/                        # PD request/response
    └── Tikvpb/                      # gRPC service stubs

tests/
├── Unit/                            # Unit tests
└── E2E/                             # End-to-end tests

examples/
├── basic.php                        # Basic CRUD example
├── batch.php                        # Batch operations example
├── scan.php                         # Scanning examples
├── ttl.php                          # TTL operations example
├── atomic.php                       # Atomic operations example
├── tls.php                          # TLS configuration example
└── logging.php                      # PSR-3 logging example

Available Commands (Makefile)

make install          # Install PHP dependencies
make test             # Run all tests (unit + e2e)
make test-unit        # Run unit tests only
make test-e2e         # Run E2E tests with TiKV cluster
make proto-generate   # Generate PHP classes from proto files
make proto-clean      # Remove generated proto classes
make build            # Build Docker images
make up               # Start TiKV cluster
make down             # Stop TiKV cluster
make logs             # Show TiKV cluster logs
make clean            # Clean everything (containers + volumes)
make example          # Run basic example
make shell            # Open development shell

Examples

See the examples/ directory for complete working examples:

basic.php — Basic CRUD operations
batch.php — Batch operations with parallel execution
scan.php — Range scanning and prefix scanning
ttl.php — Time-to-live operations
atomic.php — Compare-and-swap and put-if-absent
tls.php — TLS/SSL configuration
logging.php — PSR-3 logging integration

Run any example:

make up  # Start TiKV cluster first
php examples/basic.php

Documentation

Getting Started — Installation, setup, and your first TiKV operations
Configuration — Client options, TLS, timeouts, retry budgets, logging
Operations — Complete guide to all RawKV operations
Advanced Features — Production-ready patterns and optimization
Error Handling — Exception hierarchy, per-operation exceptions and retryability
Troubleshooting — Common issues and solutions

Configuration

Client Options

$options = [
    'tls' => [
        'caCertFile' => '/path/to/ca.crt',      // Server CA certificate (file path)
        'clientCertFile' => '/path/to/client.crt', // Client certificate (for mTLS)
        'clientKeyFile' => '/path/to/client.key',  // Client private key (for mTLS)
        // Or inline PEM: caCertPem, clientCertPem, clientKeyPem
    ],
];

$client = RawKvClient::create(
    pdEndpoints: ['127.0.0.1:2379'],
    logger: $logger,           // PSR-3 logger (optional)
    options: $options          // Additional options (optional)
);

TiKV Configuration

For TTL support, enable it in tikv.toml:

[storage]
enable-ttl = true

Note: TTL mode is exclusive with TxnKV — see TTL (Time-To-Live).

Roadmap

Recently Completed

✅ TLS/SSL Support
✅ PSR-3 Logging
✅ Region & Store Caching
✅ Connection Pooling
✅ Batch Async Execution
✅ Retry with Exponential Backoff
✅ Per-key TTL in BatchPut
✅ Scan limit enforcement (MAX 10240 per RPC; limit: 0 auto-paginates)
✅ Batch auto-splitting by size/count

Contributing

Fork the repository
Create a feature branch
Write tests for your changes
Ensure all tests pass: make test
Submit a pull request

License

MIT

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
bin		bin
docs		docs
examples		examples
proto		proto
scripts		scripts
src		src
tests		tests
.dockerignore		.dockerignore
.gitignore		.gitignore
.gitmodules		.gitmodules
CHANGELOG.md		CHANGELOG.md
Dockerfile		Dockerfile
Makefile		Makefile
README.md		README.md
composer.json		composer.json
docker-compose.txnkv.yml		docker-compose.txnkv.yml
docker-compose.yml		docker-compose.yml
phpcs.xml.dist		phpcs.xml.dist
phpstan.neon		phpstan.neon
phpunit.xml		phpunit.xml
rector.php		rector.php
tikv-v1.toml		tikv-v1.toml
tikv.toml		tikv.toml
workflow.md		workflow.md

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

TiKV PHP Client

Requirements

Quick Start

Installation

Usage

Basic CRUD Operations

Scanning

Range Operations

TTL (Time-To-Live)

Atomic Operations

Data Integrity

Bulk Import (SST Ingest)

Transactions (TxnKV)

TLS/SSL Configuration

PSR-3 Logging

Complete Example

Implemented Operations

Core CRUD

Scanning

Range Operations

TTL

Atomic Operations

Data Integrity

Infrastructure

Project Structure

Available Commands (Makefile)

Examples

Documentation

Configuration

Client Options

TiKV Configuration

Roadmap

Recently Completed

Contributing

License

About

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages