[emyller]

Runs the Flagsmith SDK benchmarks against the SDK in CodSpeed. This covers 5 local evaluation scenarios × 3 environment document sizes (small, medium, large).

• make benchmark runs the suite through the CodSpeed CLI in walltime mode. The benchmarks are listed in codspeed.yml.
• benchmarks/ drives a locally evaluating client. The environment document is served from memory through a stub Faraday adapter, so the measurements don't include network time.
• spec/sdk-benchmarks is added as a submodule pinned to 0.0.2, and benchmarks/ is left out of the gem.
• The CodSpeed workflow runs on codspeed-macro runners and uploads results via OIDC.

Closes SDK-16.

🤖 Generated with Claude Code

[linear-code]

SDK-16

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: 8309cb8b-5c9d-45d0-bcda-94ae8c57375f

Commits

Reviewing files that changed from the base of the PR and between 6ff9115 and 867c9bb.

Files selected for processing: 8

• .github/workflows/codspeed.yml
• .gitmodules
• Makefile
• benchmarks/codspeed/codspeed.c
• benchmarks/codspeed/extconf.rb
• benchmarks/codspeed/instrument-hooks
• benchmarks/run.rb
• codspeed.yml

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Walkthrough

The change adds benchmark data and a Ruby harness with five benchmark operations. The harness measures each operation with CodSpeed when instrumentation is enabled, or uses local timing otherwise. A native extension provides the CodSpeed instrumentation interface. Makefile targets install the pinned CodSpeed installer, build the extension, and run benchmarks. A GitHub Actions workflow runs the benchmark target on pushes to main, pull requests, and manual dispatches. The gemspec excludes files under benchmarks/.

Priority: Pending latest changes

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: Moderate · up to cbb2a

Merge Risk: 🟡 Moderate · up to cbb2a

On systems where the installer’s executable directory is absent from PATH, benchmarks can fail or run a different CodSpeed version. Make the pinned executable available to the recipe before merging.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

---

Review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: 0d8ea988-006a-4bbc-8b71-08427cc26ce3

Commits

Reviewing files that changed from the base of the PR and between 51544ab and aa70be9.

Files selected for processing: 8

• .github/workflows/codspeed.yml
• .gitmodules
• Makefile
• benchmarks/run.rb
• benchmarks/sdk_benchmark.rb
• codspeed.yml
• flagsmith.gemspec
• spec/sdk-benchmarks

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

[coderabbitai]

Actionable comments posted: 2

---

Review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: 4bcc2769-bbad-4882-a56c-340b973caa87

Commits

Reviewing files that changed from the base of the PR and between eca3a1f and 6ff9115.

Files selected for processing: 2

• .github/workflows/codspeed.yml
• Makefile

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

[codspeed]

Congrats! CodSpeed is installed 🎉

🆕 15 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

1 warning

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

Detected benchmarks

• getEnvironmentFlags[large]: 70.8 ms
• getEnvironmentFlags[medium]: 11.6 ms
• getEnvironmentFlags[small]: 1.3 ms
• getIdentityFlagsNoTraits[large]: 106.5 ms
• getIdentityFlagsNoTraits[medium]: 15.8 ms
• getIdentityFlagsNoTraits[small]: 2.3 ms
• getIdentityFlagsWithIdentityOverrides[large]: 108.1 ms
• getIdentityFlagsWithIdentityOverrides[medium]: 15.8 ms
• getIdentityFlagsWithIdentityOverrides[small]: 2.3 ms
• getIdentityFlagsWithTraits[large]: 114.4 ms
• getIdentityFlagsWithTraits[medium]: 16.6 ms
• getIdentityFlagsWithTraits[small]: 3 ms
• getIdentitySegmentsWithTraits[large]: 108.7 ms
• getIdentitySegmentsWithTraits[medium]: 14.1 ms
• getIdentitySegmentsWithTraits[small]: 2.9 ms

---

[khvn26]

bom dia