Skip to content

test: turn off CrewAI telemetry for the suite (#1202) - #1230

Merged
tommasobolis merged 1 commit into
developfrom
fix/1202-crewai-telemetry
Oct 25, 2026
Merged

tommasobolis merged 1 commit into
developfrom
fix/1202-crewai-telemetry

Conversation

@tommasobolis

@tommasobolis tommasobolis commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1202. It fixes the CrewAI half here. The ADK after_model_callback failure, the other half, is already fixed by #1214 (merged after the last red nightly ran). ADK 2.13.0 passes on current develop.

Cause

CrewAI exports anonymous telemetry from a background thread. test_transport_proxy_mounts.py sets a dead ALL_PROXY (127.0.0.1:9), and its no_sockets fixture patches socket.connect for the whole process. When CrewAI's export landed inside a test's window, test_sync_simulate_stays_offline_under_proxy[ALL_PROXY] failed on CrewAI's traffic, not ours. The failure depends on timing, which is why the lowest-direct (crewai) leg went red at random.

Fix (test-only)

  • tests/conftest.py sets CREWAI_DISABLE_TELEMETRY=true when the suite loads, next to the existing LITELLM_LOCAL_MODEL_COST_MAP switch. crewai 1.15.3 (the floor) and 1.15.23 both honour it. OTEL_SDK_DISABLED would also work, but the autouse env fixture clears every OTEL_* var per test.
  • A guard test, test_crewai_telemetry_is_off_for_the_suite, fails if CrewAI stops honouring the switch, so the nightly shows the real cause.

Verification

  • The full suite passes locally: 3718 passed, 118 skipped. mypy and ruff are clean.
  • The guard test fails with CREWAI_DISABLE_TELEMETRY=false.
  • The original flake depends on timing and couldn't be reproduced on demand.

No ADR needed (test-only change).

🤖 Generated with Claude Code

CrewAI exports anonymous telemetry from a background thread. Under the
dead ALL_PROXY in test_transport_proxy_mounts.py, that export opened a
socket while no_sockets patches socket.connect process-wide. The
lowest-direct (crewai) nightly leg then failed on CrewAI's traffic, not
ours, whenever the export landed inside a test's window.

conftest now sets CREWAI_DISABLE_TELEMETRY=true at load, next to the
LiteLLM cost-map switch. The switch is honoured by crewai 1.15.3 (the
floor) through 1.15.23. OTEL_SDK_DISABLED would also work but is
cleared per test with the other OTEL_* vars. A guard test fails if
CrewAI stops honouring the switch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@tommasobolis
tommasobolis merged commit 93633e6 into develop Oct 25, 2026
48 checks passed
@tommasobolis
tommasobolis deleted the fix/1202-crewai-telemetry branch October 25, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nightly framework matrix failing

1 participant