[alamb]

Which issue does this PR close?

• Part of Merge queue times out PRs whose checks are still running, so merging takes many manual attempts and CI is re-run from scratch #26292

Rationale for this change

The merge queue evicts PRs whose checks have not reported within 60 minutes (the GitHub default), even when nothing failed and the remaining jobs are only waiting for a runner. Each eviction discards the CI that already passed, forces someone to re-queue the PR by hand, and reruns everything from scratch. See #26292 for examples and numbers.

Once the queue's CI is actually running it finishes in 10 to 25 minutes,

What changes are included in this PR?

Adds a Merge Queue ruleset to .asf.yaml that sets check_response_timeout_minutes: 360. Every other merge queue parameter is set to the value the queue has today (read back via the GraphQL API). Here are the current settings

What is the testing strategy for this PR?

My agent says:

• python3 ci/scripts/check_asf_yaml_status_checks.py passes.
• ASF's own schema validator, asfyaml-validate from apache/infrastructure-asfyaml, reports the file as valid.

After merge we should check it took effect via

(venv) andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion$ gh api graphql -f query='{ repository(owner:"apache", name:"datafusion") {
  mergeQueue(branch:"main") { configuration {
    checkResponseTimeout maximumEntriesToBuild maximumEntriesToMerge
    minimumEntriesToMerge minimumEntriesToMergeWaitTime mergeMethod mergingStrategy } } } }'
{
  "data": {
    "repository": {
      "mergeQueue": {
        "configuration": {
          "checkResponseTimeout": 3600,
          "maximumEntriesToBuild": 5,
          "maximumEntriesToMerge": 1,
          "minimumEntriesToMerge": 1,
          "minimumEntriesToMergeWaitTime": 300,
          "mergeMethod": "SQUASH",
          "mergingStrategy": "ALLGREEN"
        }
      }
    }
  }
}

checkResponseTimeout should read 21600 (seconds) instead of today's 3600, and gh api repos/apache/datafusion/rulesets should list a Merge Queue ruleset.

Are there any user-facing changes?

No. This only changes how long an entry may wait in the merge queue before being evicted.

[mbutrovich]

Let's give it a shot, thanks @alamb!

[Omega359]

fingers crossed

[alamb]

Thanks @mbutrovich and @Omega359 -- I'll merge this one in and hopefully we can start to see our PRs make it through the queue

[alamb]

Here are the current settings

(venv) andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion$ gh api graphql -f query='{ repository(owner:"apache", name:"datafusion") {
  mergeQueue(branch:"main") { configuration {
    checkResponseTimeout maximumEntriesToBuild maximumEntriesToMerge
    minimumEntriesToMerge minimumEntriesToMergeWaitTime mergeMethod mergingStrategy } } } }'
{
  "data": {
    "repository": {
      "mergeQueue": {
        "configuration": {
          "checkResponseTimeout": 3600,
          "maximumEntriesToBuild": 5,
          "maximumEntriesToMerge": 1,
          "minimumEntriesToMerge": 1,
          "minimumEntriesToMergeWaitTime": 300,
          "mergeMethod": "SQUASH",
          "mergingStrategy": "ALLGREEN"
        }
      }
    }
  }
}

[alamb]

Here are Comet's settings https://github.com/apache/datafusion-comet/blob/7d8d9b1544b734c85a93cca2cc9d5380ad70accd/.asf.yaml#L128-L176

 rulesets:
    - name: Merge Queue
      target: branch
      enforcement: active
      bypass_actors:
        # apache/root - ASF Infra, so a wedged queue can always be recovered.
        - actor_id: 118420
          actor_type: Team
          bypass_mode: always
      conditions:
        ref_name:
          include:
            - "~DEFAULT_BRANCH"
          exclude: []
      rules:
        - type: merge_queue
          parameters:
            merge_method: SQUASH
            grouping_strategy: ALLGREEN
            max_entries_to_build: 4
            min_entries_to_merge: 1
            min_entries_to_merge_wait_minutes: 0
            max_entries_to_merge: 5
            check_response_timeout_minutes: 300

[alamb]

Sorry -- forgot to post my comments

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.79%. Comparing base (0d79252) to head (210ec8d).

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #26293      +/-   ##
==========================================
- Coverage   82.80%   82.79%   -0.01%     
==========================================
  Files        1149     1149              
  Lines      454146   454146              
  Branches   454146   454146              
==========================================
- Hits       376035   376027       -8     
- Misses      55260    55265       +5     
- Partials    22851    22854       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[blaginin]

i can force merge if no one objects?

[alamb]

i can force merge if no one objects?

Yes please 🙏

[alamb]

The irony is not lost on me

[Omega359]

"github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks 1 hour ago"

Now that is funny

[alamb]

Thank you for dropping the 🔨 @blaginin