Remove `brew which-update`

[Rylan12]

Verification

• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

Provide a detailed description of the proposed feature

We should remove brew which-update and, instead, update the executables.txt file as part of homebrew/core CI with each merge.

Additionally, we should include this information in the bottle manifest files when possible.

What is the motivation for the feature?

Right now, the DB update mechanism is not ideal: we download all updated formulae once a day and try to install them. We can make this more efficient by including this update as part of homebrew/core CI and removing the need for a dedicated job and remove the need for the which-update developer command.

How will the feature be relevant to at least 90% of Homebrew users?

Not relevant to users directly

What alternatives to the feature have been considered?

Leave as-is

[MikeMcQuaid]

We should remove brew which-update and, instead, update the executables.txt file as part of homebrew/core CI with each merge.

@Rylan12 What does this update process look like i.e. what tools should be run, at what point, how long does it take, etc.

Additionally, we should include this information in the bottle manifest files when possible.

I think this is already present under sh.brew.path_exec_files e.g. brew fetch zizmor && cat ~/Library/Caches/Homebrew/zizmor_bottle_manifest--1.16.0 | jq '.manifests[0].annotations."sh.brew.path_exec_files"'?

[Rylan12]

We should remove brew which-update and, instead, update the executables.txt file as part of homebrew/core CI with each merge.

@Rylan12 What does this update process look like i.e. what tools should be run, at what point, how long does it take, etc.

@MikeMcQuaid it should be essentially the following for each formula post-merge:

• Fetch the bottle manifest (or potentially use GitHub Actions artifcats or something similar)
• Read the sh.brew.path_exec_files (see comment below) and/or other relevant attributes (e.g. disabled/deprecated)
  • Ideally, end early at this point if the executable files haven't changed, if we can easily check
  • It's also possible that a formula will have been deleted which needs to be handled
• Use oras to download the executables.txt package
• Update the formula's entry
  • Either adjust the executables list, adding a new line, or removing the line entirely if deprecated, or disabled, or removed
• Use oras to push the executables.txt package

Some more notes:

• We should make sure concurrency is set up so only one of these executables.txt updates runs at a time, but additional jobs are just queued up, rather than cancelled
• We won't need to store formula version information in this file anymore, since that was added for keeping track of which formulae need to be re-checked. Instead, if we only ever check formulae when they're modified, we won't need that indicator.

Honestly, none of this should be particularly time-intensive or require anything special except oras. I think we can do this with the current tooling we have; it should be a matter of sitting down and writing the workflow.

Additionally, we should include this information in the bottle manifest files when possible.

I think this is already present under sh.brew.path_exec_files e.g. brew fetch zizmor && cat ~/Library/Caches/Homebrew/zizmor_bottle_manifest--1.16.0 | jq '.manifests[0].annotations."sh.brew.path_exec_files"'?

It might be! At one point, I thought this was removed due to some issues it was causing in homebrew/core CI, but it seems like it's working again (or I misremembered)

[MikeMcQuaid]

I think we can do this with the current tooling we have; it should be a matter of sitting down and writing the workflow.

Ok, thanks! Which command(s) cover any of this already?

At one point, I thought this was removed due to some issues it was causing in homebrew/core CI, but it seems like it's working again (or I misremembered)

That was listing all the files in the manifests rather than the executable ones.

[Rylan12]

I think we can do this with the current tooling we have; it should be a matter of sitting down and writing the workflow.

Ok, thanks! Which command(s) cover any of this already?

I may have misunderstood what you meant. What we have currently is the command-not-found-db-update.yml workflow that does some of this (things like downloading, updating, checking, and deleting the GitHub Packages stuff):

brew/.github/workflows/command-not-found-db-update.yml

Lines 28 to 69 in f69d9a5

	- name: Install oras for interacting with GitHub Packages
	run: brew install oras
	- name: Pull executables.txt from GitHub Packages
	run: oras pull ghcr.io/homebrew/command-not-found/executables:latest
	- name: Update database
	env:
	MAX_DOWNLOADS: ${{ github.event.inputs.max-downloads }}
	run: |
	set -eo pipefail
	if [[ -n "$MAX_DOWNLOADS" ]]
	then
	MAX_DOWNLOADS_ARGS="--max-downloads $MAX_DOWNLOADS"
	fi
	# Need to intentionally leave MAX_DOWNLOADS_ARGS unquoted.
	# shellcheck disable=SC2086
	brew which-update --update-existing --install-missing $MAX_DOWNLOADS_ARGS executables.txt --summary-file=$GITHUB_STEP_SUMMARY
	- name: Output database stats
	run: brew which-update --stats executables.txt
	- name: Log in to GitHub Packages
	if: github.ref == 'refs/heads/main'
	run: echo "${{ secrets.GITHUB_TOKEN }}" | oras login ghcr.io --username brewtestbot --password-stdin
	- name: Push to GitHub Packages
	if: github.ref == 'refs/heads/main'
	run: |
	oras push --artifact-type application/vnd.homebrew.command-not-found.executables \
	ghcr.io/homebrew/command-not-found/executables:latest \
	executables.txt:text/plain
	- name: Check upload
	if: github.ref == 'refs/heads/main'
	run: |
	shasum --algorithm=256 executables.txt > executables.txt.sha256
	rm -f executables.txt
	oras pull ghcr.io/homebrew/command-not-found/executables:latest
	shasum --algorithm=256 --check executables.txt.sha256

The actual database update (and logic that goes into that like pulling the manifest, etc) is the part that doesn't exist yet.

There is some existing database update code in executables_db.rb, but tbh I don't think this will be super useful because most of that is intended for tracking updates to lots of formulae at once rather than one at a time

[MikeMcQuaid]

The actual database update (and logic that goes into that like pulling the manifest, etc) is the part that doesn't exist yet.

@Rylan12 so the replacement of brew which-update basically that lives works incrementally on a single formula basis, right?

[Rylan12]

Yep, that's what I've been picturing!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.