You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: benchmark stacked PRs against main and retain failure logs - #1002
Pin the tip of main when planning each PR run. For a stack such as #958 on #954, measure the child's exact head against that pinned commit. Use the diff from the merge base with main for the core file filter so changes in the parent count. Keep the perf-check override.
Recover from build and reference failures
Retry Ubuntu package downloads when the hosted build sees a mirror sync error. The MCP run stopped before measuring. Failed build
Explain when the saved daily reference needs recalibration and use a separate issue title for measurement failures. The first reference run after Run the FFF reference directly on load #1000 stopped on a script checksum mismatch. Failed reference
Keep the benchmark console log with collected timings in fff-measurements for 30 days. Report build and measurement failures in the PR status. Use the report artifact ID for same-repository publication so a publisher-only retry can find the report.
No actionable comments were generated in the recent review.
Walkthrough
The performance workflows now accept PRs targeting any base branch, compare eligible changes with a revision from main, and capture benchmark logs and results. Report artifacts include PR and failure-stage metadata. The reference workflow validates its checksum and reports measurement failures.
Changes: Performance benchmark workflow
Layer / File(s)
Summary
Benchmark selection and revision .github/workflows/performance.yml, perf/build.sh, perf/README.md
The workflow checks changes against a revision from main and accepts PRs with any base branch. The build script selects seven named benchmarks and retries package installation. The README describes the comparison behavior.
Measurement capture and report artifact .github/workflows/performance.yml, perf/README.md
The measurement job captures benchmark logs and uploads them with timing files. The report job records the PR number and failure stage, and exposes the uploaded artifact ID. The README documents artifact retention.
Report artifact verification and publishing .github/workflows/performance-publish.yml
The publisher obtains the report artifact ID from its caller or locates it for a workflow run. It validates the PR head against the workflow-run source and distinguishes build failures from measurement failures.
The reference workflow verifies the saved checksum before measurement. It reports failures through issues and uploads logs and benchmark files after success or failure.
sequenceDiagram
participant PerformanceWorkflow
participant BuildJob
participant MeasureJob
participant ReportArtifact
participant PublishWorkflow
PerformanceWorkflow->>BuildJob: Build benchmark binaries
PerformanceWorkflow->>MeasureJob: Run benchmarks and capture logs
MeasureJob->>ReportArtifact: Upload timings and benchmark log
PerformanceWorkflow->>ReportArtifact: Upload PR number and failure stage
PerformanceWorkflow->>PublishWorkflow: Pass report_artifact_id
Loading
Possibly related issues
dmtrKovalenko/fff#1001 — The PR changes reference-run failure reporting, which the issue also concerns.
Possibly related PRs
dmtrKovalenko/fff#1000 — This PR updates the performance workflow and report artifact flow introduced there.
dmtrKovalenko/fff#914 — This PR explicitly builds the fuzzy_search_bench benchmark added by that PR.
Suggested reviewers:dmtrkovalenko
Merge Risk: Minimal · up to ebf9a
The workflow changes show no identified blocker in benchmark selection or failure reporting, so the PR appears mergeable after normal CI checks.
Pre-merge checks: 5
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title clearly summarizes the main changes: benchmarking stacked PRs against main and retaining failure logs.
Docstring Coverage
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 1 files. (2 skipped: 2 unsupported.)
Finishing Touches
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: a4f3858d-4658-4393-98ee-a70bf4802635
Commits
Reviewing files that changed from the base of the PR and between b5fd75c and ebf9a75.
Files selected for processing: 2
.github/workflows/performance.yml
perf/README.md
Files skipped from review as they are similar to previous changes: 1
perf/README.md
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Benchmark stacked PRs against main
Pin the tip of
mainwhen planning each PR run. For a stack such as #958 on #954, measure the child's exact head against that pinned commit. Use the diff from the merge base withmainfor the core file filter so changes in the parent count. Keep theperf-checkoverride.Recover from build and reference failures
fff-measurementsfor 30 days. Report build and measurement failures in the PR status. Use the report artifact ID for same-repository publication so a publisher-only retry can find the report.Verification
mainSHA.Summary by CodeRabbit
main.