Skip to content

ci: Run CodSpeed benchmarks in simulation on GitHub-hosted runners - #75

Merged
emyller merged 1 commit into
mainfrom
ci/codspeed-simulation
Oct 30, 2026
Merged

emyller merged 1 commit into
mainfrom
ci/codspeed-simulation

Conversation

@emyller

@emyller emyller commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

CodSpeed benchmarks don't run: macro runner usage is exceeded, so PRs get no comparison.

Follows #71, as Flagsmith/flagsmith-python-client#274.

Changes

  • Benchmarks run on the runner and in the mode set by the CODSPEED_RUNS_ON and CODSPEED_MODE organisation variables, currently ubuntu-latest and simulation.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@emyller
emyller requested a review from a team as a code owner October 30, 2026 20:39
@emyller
emyller requested review from khvn26 and removed request for a team October 30, 2026 20:39
@coderabbitai

coderabbitai Bot commented Oct 30, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 621b7305-30c7-4f28-b881-bc63575b7d13
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@emyller
emyller merged commit f438520 into main Oct 30, 2026
7 checks passed
@emyller
emyller deleted the ci/codspeed-simulation branch October 30, 2026 20:53
@codspeed

codspeed Bot commented Oct 30, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ Some benchmarks were skipped: Macro Runners budget exceeded

Your organization has exceeded its Macro Runners budget, so some benchmarks have been skipped.

Update your plan or budget

Learn more about Macro Runners

🆕 15 new benchmarks
⏩ 15 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
🆕 get_environment_flags[large] N/A 7.9 ms N/A
🆕 get_environment_flags[medium] N/A 2.1 ms N/A
🆕 get_environment_flags[small] N/A 99.5 µs N/A
🆕 get_identity_flags_no_traits[large] N/A 45 ms N/A
🆕 get_identity_flags_no_traits[medium] N/A 7.5 ms N/A
🆕 get_identity_flags_no_traits[small] N/A 1.4 ms N/A
🆕 get_identity_flags_with_identity_overrides[large] N/A 45.1 ms N/A
🆕 get_identity_flags_with_identity_overrides[medium] N/A 7.5 ms N/A
🆕 get_identity_flags_with_identity_overrides[small] N/A 1.4 ms N/A
🆕 get_identity_flags_with_traits[large] N/A 54 ms N/A
🆕 get_identity_flags_with_traits[medium] N/A 9.1 ms N/A
🆕 get_identity_flags_with_traits[small] N/A 2.2 ms N/A
🆕 get_identity_segments_with_traits[large] N/A 52 ms N/A
🆕 get_identity_segments_with_traits[medium] N/A 8.6 ms N/A
🆕 get_identity_segments_with_traits[small] N/A 2.2 ms N/A

Comparing ci/codspeed-simulation (6ade217) with main (4bf93a4)

Open in CodSpeed

Footnotes

  1. 15 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants