Skip to content

Commit 60707ca

Browse files
committed
feat: stats-based pruning, workbench UI, and instance-level S3 config (Phase 3)
Replace predicate partitioning with flat files and per-file min/max stats for pruning. Add S3 Store to RDF4J Workbench with creation form and TTL config template. S3 connection settings (bucket, endpoint, credentials) resolve from environment variables (RDF4J_S3_*) or system properties so multiple repositories share a single bucket, each isolated by s3Prefix.
1 parent 306c533 commit 60707ca

20 files changed

Lines changed: 1301 additions & 862 deletions

File tree

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#
2+
# Configuration template for an S3Store
3+
#
4+
# S3 connection settings (bucket, endpoint, region, credentials) are configured
5+
# at the RDF4J instance level via environment variables or system properties:
6+
#
7+
# RDF4J_S3_BUCKET, RDF4J_S3_ENDPOINT, RDF4J_S3_REGION,
8+
# RDF4J_S3_ACCESS_KEY, RDF4J_S3_SECRET_KEY, RDF4J_S3_FORCE_PATH_STYLE
9+
#
10+
# Each repository uses s3Prefix to partition its data within the shared bucket.
11+
#
12+
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
13+
@prefix config: <tag:rdf4j.org,2023:config/>.
14+
@prefix s3: <http://rdf4j.org/config/sail/s3#>.
15+
16+
[] a config:Repository ;
17+
config:rep.id "{%Repository ID|s3%}" ;
18+
rdfs:label "{%Repository title|S3 Store%}" ;
19+
config:rep.impl [
20+
config:rep.type "openrdf:SailRepository" ;
21+
config:sail.impl [
22+
config:sail.type "rdf4j:S3Store" ;
23+
s3:s3Prefix "{%S3 Prefix|%}"
24+
]
25+
].

core/sail/s3/docker-compose.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
services:
2+
minio:
3+
image: minio/minio:latest
4+
ports:
5+
- "9000:9000"
6+
- "9001:9001"
7+
environment:
8+
MINIO_ROOT_USER: minioadmin
9+
MINIO_ROOT_PASSWORD: minioadmin
10+
command: server /data --console-address ":9001"
11+
volumes:
12+
- minio-data:/data
13+
healthcheck:
14+
test: ["CMD", "mc", "ready", "local"]
15+
interval: 5s
16+
timeout: 5s
17+
retries: 5
18+
19+
createbucket:
20+
image: minio/mc:latest
21+
depends_on:
22+
minio:
23+
condition: service_healthy
24+
entrypoint: >
25+
/bin/sh -c "
26+
mc alias set local http://minio:9000 minioadmin minioadmin;
27+
mc mb --ignore-existing local/rdf4j-data;
28+
echo 'Bucket ready';
29+
"
30+
31+
volumes:
32+
minio-data:

0 commit comments

Comments
 (0)