Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .agents/skills/devservices-troubleshooting/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
---
name: devservices-troubleshooting
description: Fix connection-refused/RPC Unavailable errors in objectstore-service tests by starting devservices (GCS/Bigtable emulators)
description: Fix connection-refused/RPC Unavailable errors in objectstore-service tests by starting devservices (GCS/Bigtable emulators, Cassandra)
---

Some tests require external services (GCS emulator, Bigtable emulator) managed by `devservices`.
Some tests require external services (GCS emulator, Bigtable emulator, Cassandra) managed by `devservices`.

**Symptoms of missing services:**
- Connection refused errors
- TCP connect error messages
- `RPC error: status: Unavailable`
- Tests in `objectstore-service` for GCS/Bigtable backends fail
- `Failed to find a node with working connection pool` (CQL backend)
- Tests in `objectstore-service` for GCS/Bigtable/CQL backends fail

**How to fix:**

Expand All @@ -24,3 +25,5 @@ Some tests require external services (GCS emulator, Bigtable emulator) managed b
```

3. Devservices run in the background - you only need to start them once per session

4. Cassandra takes up to a minute to start; wait until `devservices status` reports it healthy
68 changes: 68 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,19 @@ jobs:
google/cloud-sdk \
/run-bigtable.sh

- name: Start Cassandra
run: |
docker run -d \
--name cassandra \
-p 8088:9042 \
-e MAX_HEAP_SIZE=512M \
-e HEAP_NEWSIZE=128M \
-e CASSANDRA_BROADCAST_RPC_ADDRESS=127.0.0.1 \
-v $PWD/devservices/run-cassandra.sh:/run-cassandra.sh \
-v $PWD/devservices/cql-schema.cql:/cql-schema.cql \
cassandra:5.0.9 \
/run-cassandra.sh

- name: Build GCS Emulator
run: |
docker build -t gcs-emulator-local https://github.com/getsentry/google-cloud-storage-testbench.git#main
Expand Down Expand Up @@ -137,6 +150,11 @@ jobs:

- uses: taiki-e/install-action@cargo-llvm-cov

# Cassandra takes a while to boot; the schema exists once the start script is done.
- name: Wait for Cassandra
run: |
timeout 300 bash -c 'until docker exec cassandra cqlsh -e "describe table objectstore.objectstore" >/dev/null 2>&1; do sleep 3; done'

- name: Run Rust doctests
run: cargo test --workspace --all-features --doc --locked

Expand All @@ -151,6 +169,56 @@ jobs:
flags: rust
use_oidc: true

test-cql-scylla:
name: Test CQL backend (ScyllaDB)
runs-on: ubuntu-latest

steps:
# librdkafka is built from source and links against libcurl, whose headers are not
# on the runner by default. Needed by anything that builds the `kafka` feature.
- name: Install libcurl-dev
run: |
sudo apt-get update
sudo apt-get install -y libcurl4-openssl-dev

- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0

- name: Start ScyllaDB
run: |
docker run -d \
--name scylla \
-p 8090:9042 \
-v $PWD/devservices/cql-schema.cql:/cql-schema.cql \
scylladb/scylla:2026.1.14 \
--smp 1 --memory 750M --overprovisioned 1 --developer-mode 1 \
--broadcast-rpc-address 127.0.0.1

- name: Install Rust Toolchain
run: |
rustup toolchain install stable --profile minimal --no-self-update

- name: Install protoc
uses: arduino/setup-protoc@c65c819552d16ad3c9b72d9dfd5ba5237b9c906b # v3.0.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

- uses: swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
with:
key: ${{ github.job }}

- name: Build tests
run: cargo test -p objectstore-service --all-features --lib --locked --no-run

- name: Create ScyllaDB schema
run: |
timeout 300 bash -c 'until docker exec scylla cqlsh -e "describe keyspaces" >/dev/null 2>&1; do sleep 3; done'
docker exec scylla cqlsh -f /cql-schema.cql

- name: Run tests
run: cargo test -p objectstore-service --all-features --lib --locked backend::cql
env:
OS_TEST_CQL_NODES: localhost:8090

test-python:
name: Test Python
runs-on: ubuntu-latest
Expand Down
Loading
Loading