Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
a3d0115
Fix Warning 1300 for varbinary columns with bytes invalid as utf8mb4 …
ggilder Apr 22, 2026
a6a5f49
Prevent heartbeat race condition after cutover completion (#1664)
jakubpliszka Apr 29, 2026
4deeadf
Prevent permanent worker deadlock when cutover times out waiting for …
VarunChandola Apr 29, 2026
b13e116
Fix OOM when allEventsUpToLockProcessed buffer equals MaxRetries() (#…
dnovitski Apr 30, 2026
29c0c33
Retry attempt InstantDDL up to `--default-retries` (#1667)
meiji163 Apr 30, 2026
4d37b8a
Update golangci-lint to v2.11 (#1657)
meiji163 May 1, 2026
4d38923
Upgrade to go1.25.9 (#1668)
meiji163 May 1, 2026
494570d
Prevent throttle() from blocking forever when context is cancelled (#…
ggilder May 11, 2026
e62debe
Add MigrationContext.Hooks for in-process hook implementations (#1675)
olsonjp May 14, 2026
b9ff6a6
Bump golang.org/x/crypto from 0.37.0 to 0.45.0 in the go_modules grou…
dependabot[bot] May 15, 2026
fe459a9
Do not report extremely high HeartbeatLag values until the first hear…
grodowski May 19, 2026
154d214
Handle context cancellation in consumeRowCopyComplete to prevent dead…
jakubpliszka May 19, 2026
72cf229
Fix resume data loss: route heartbeat coords through applyEventsQueue…
grodowski May 26, 2026
e59af3a
Add Datadog/statsd with simple client emitting startup (#1689)
forge33 May 27, 2026
59f400f
Add Go runtime metrics to statsd reporting (#1690)
forge33 May 27, 2026
c636347
Add GET_LOCK to prevent concurrent migrations of the same table (#1693)
jakubpliszka May 28, 2026
251b08d
Use os.TempDir() for test socket path (#1694)
ericyan May 29, 2026
835f537
Improve performance for tables with composite primary keys (#1686)
coding-chimp Jun 1, 2026
83aaa70
Disable CGO for release builds (#1697)
meiji163 Jun 4, 2026
aecf39f
StatsD instrumentation: row-copy, backlog, lag, throttle, cut-over, q…
forge33 Jun 19, 2026
f50264b
Upgrade go-mysql to v1.15.0
ericyan Jun 19, 2026
1409f8b
Don’t parse binlog events of unrelated tables (#1699)
coding-chimp Jun 19, 2026
dce7733
Merge branch 'master' into upgrade-go-mysql
meiji163 Jun 19, 2026
dcda427
Document missing command-line flags (#1706)
grodowski Jun 22, 2026
06b3a7e
test(localtests): multi-flavor docker replica-test harness (#1722)
conf Jun 22, 2026
653b99e
feat: 🦭 MariaDB support 🦭 (#1724)
conf Jun 23, 2026
7db8f74
Merge branch 'master' into upgrade-go-mysql
ericyan Jun 24, 2026
64a6e3d
Merge pull request #1715 from github/upgrade-go-mysql
ericyan Jun 24, 2026
c231272
build: standardize release asset naming (#1729)
conf Jun 25, 2026
167d444
feat: MariaDB GTID support
conf Jun 25, 2026
4da9dac
Bump go.opentelemetry.io/otel/sdk
dependabot[bot] Jul 29, 2026
39c6242
Bump actions/checkout from 4 to 7
dependabot[bot] Jul 29, 2026
a9ee34b
Bump github/codeql-action from 3 to 4
dependabot[bot] Jul 29, 2026
d091966
Merge pull request #1576 from github/dependabot/github_actions/action…
ericyan Jul 29, 2026
93586bd
Merge branch 'master' into dependabot/github_actions/github/codeql-ac…
ericyan Jul 29, 2026
7cb81b4
Merge pull request #1589 from github/dependabot/github_actions/github…
ericyan Jul 29, 2026
65ff5f0
Merge branch 'master' into dependabot/go_modules/go_modules-5f729b4834
ericyan Jul 29, 2026
20e2105
Merge pull request #1740 from github/dependabot/go_modules/go_modules…
ericyan Jul 29, 2026
1c9f696
Upgrade testcontainers to v0.43.0 and x/crypto to v0.54.0
ericyan Jul 29, 2026
ed6b0cc
Upgrade Go to 1.25.12
ericyan Jul 29, 2026
8e498d1
Merge pull request #1741 from github/dep-update
ericyan Jul 29, 2026
d412341
Merge branch 'master' into go-1.25.12
ericyan Jul 29, 2026
83d4824
Fix Go 1.25.12 checksum constants in ensure-go-installed
Copilot Jul 29, 2026
08d2a5f
Merge pull request #1742 from github/go-1.25.12
ericyan Jul 29, 2026
014cdc8
refactor(logic): scope testcontainers helpers to the test build
davidham Jul 28, 2026
c32063f
Merge pull request #1739 from davidham/bump-deps-and-go-to-address-hi…
ericyan Jul 29, 2026
838e59a
`docs: clarify replica selection requirements for --throttle-control-…
soepic1 Jul 30, 2026
361aeb6
docs: condense throttle-control-replicas note per review feedback
soepic1 Jul 31, 2026
a82eeef
Merge branch 'master' into mariadb-gtid-support
ericyan Jul 31, 2026
84079b4
Merge pull request #1731 from conf/mariadb-gtid-support
ericyan Jul 31, 2026
c1b9cd1
Apply suggestions from code review
soepic1 Jul 31, 2026
ae84d3c
Merge branch 'master' into patch-1
ericyan Aug 3, 2026
0dda3ab
Merge pull request #1745 from soepic1/patch-1
ericyan Aug 3, 2026
2425722
Add opt-in ANALYZE TABLE on the ghost table before cut-over
VandhanaSelvaprakash-at Aug 5, 2026
7e43186
Address review: consistent receiver, migrator seam + test, localtest
VandhanaSelvaprakash-at Aug 6, 2026
5ed7f36
Merge pull request #1747 from VandhanaSelvaprakash-at/vaselvap-upstre…
ericyan Aug 7, 2026
060e090
Bump actions/upload-artifact from 4 to 7
dependabot[bot] Aug 7, 2026
145799e
Merge pull request #1598 from github/dependabot/github_actions/action…
ericyan Aug 7, 2026
778120b
docs: correct spelling
vladdoster Aug 11, 2026
e403fed
Merge pull request #1749 from vladdoster/docs/fix-spelling
ericyan Aug 12, 2026
22042c5
test: enable race detection in unit tests
ericyan Aug 13, 2026
b8d0832
fix: clone GTID coordinates for reader and syncer ownership
ericyan Aug 14, 2026
66059b7
fix: avoid global stream race in retry hook test
ericyan Aug 14, 2026
1cb2071
Merge pull request #1750 from github/race-tests
ericyan Aug 14, 2026
7296e4a
Add release workflow
ericyan Aug 17, 2026
ae531ce
Potential fix for pull request finding
ericyan Aug 17, 2026
2ed192c
Merge pull request #1753 from github/release-workflow
ericyan Aug 17, 2026
9171d42
feat: ghost tables in a separate database, with a ~ name prefix
d-bytebase Nov 2, 2026
f38c4c2
chore: use the migration uuid as the TLS config key
d-bytebase Nov 2, 2026
a1c64dd
fix: do not accept on a nil unixListener
d-bytebase Nov 2, 2026
69b01f2
fix: close the binlog reader when tearing down the streamer
d-bytebase Nov 2, 2026
912d8ff
chore: return after reporting a streaming failure to panicAbort
d-bytebase Nov 2, 2026
8d749e6
fix: release the original table lock when two-step cut-over fails
d-bytebase Nov 2, 2026
feba86b
chore: circuit breaker for repeated authentication failures
d-bytebase Nov 2, 2026
38eec5e
feat: support a custom MySQL dial network
d-bytebase Nov 2, 2026
6ff100b
fix: do not deadlock when instant DDL completes the migration
d-bytebase Nov 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7

- name: Set up Go
uses: actions/setup-go@v6
Expand All @@ -21,7 +21,7 @@ jobs:
run: script/cibuild

- name: Upload gh-ost binary artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: gh-ost
path: bin/gh-ost
6 changes: 3 additions & 3 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7

- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@v4
6 changes: 3 additions & 3 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ jobs:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
uses: golangci/golangci-lint-action@v9
with:
version: v1.61.0
version: v2.11
84 changes: 84 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: release

on:
push:
tags:
- "v*"

permissions:
contents: write
id-token: write
attestations: write

concurrency:
group: release-${{ github.ref }}

jobs:
release:
name: Build release artifacts
runs-on: ubuntu-latest

steps:
- name: Checkout release tag
uses: actions/checkout@v7
with:
fetch-depth: 0

- name: Build release artifacts
run: |
DOCKER_BUILDKIT=1 docker build . \
--file Dockerfile.packaging \
--target artifacts \
--output type=local,dest=release-artifacts

- name: Verify checksums
working-directory: release-artifacts
run: sha256sum --check SHA256SUMS

- name: Verify embedded version
working-directory: release-artifacts
env:
RELEASE_TAG: ${{ github.ref_name }}
run: |
# Match build.sh, which strips every "v" from the tag to form the version.
expected_version=$(echo "${RELEASE_TAG}" | tr -d 'v')

# linux-amd64 runs on the runner, so assert the reported version exactly.
tar -xzf gh-ost-linux-amd64.tar.gz
actual_version=$(./gh-ost --version)
test "${actual_version%% *}" = "${expected_version}"
rm -f gh-ost

# The other platforms can't execute here, so assert the version string is
# embedded in each binary instead.
for tarball in gh-ost-linux-arm64.tar.gz gh-ost-darwin-amd64.tar.gz gh-ost-darwin-arm64.tar.gz; do
tar -xzf "${tarball}"
grep -a -q -- "${expected_version}" gh-ost
rm -f gh-ost
done

- name: Attest release artifacts
uses: actions/attest-build-provenance@v4
with:
subject-path: release-artifacts/*

- name: Create draft release
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.ref_name }}
run: |
if ! gh release view "${RELEASE_TAG}" >/dev/null 2>&1; then
gh release create "${RELEASE_TAG}" \
--verify-tag \
--draft \
--generate-notes \
--title "${RELEASE_TAG}"
fi

- name: Upload release artifacts
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.ref_name }}
run: |
test "$(gh release view "${RELEASE_TAG}" --json isDraft --jq '.isDraft')" = "true"
gh release upload "${RELEASE_TAG}" release-artifacts/* --clobber
6 changes: 3 additions & 3 deletions .github/workflows/replica-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ jobs:
strategy:
fail-fast: false
matrix:
image: ['mysql/mysql-server:5.7.41','mysql:8.0.41','mysql:8.4.3','percona/percona-server:8.0.41-32']
image: ['mysql:5.7.41','mysql:8.0.41','mysql:8.4.3','percona/percona-server:8.0.41-32','mariadb:10.5.29','mariadb:10.6.27','mariadb:10.11.18','mariadb:11.4.12','mariadb:11.8.8']
env:
TEST_MYSQL_IMAGE: ${{ matrix.image }}

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7

- name: Install sysbench
run: |
Expand All @@ -36,7 +36,7 @@ jobs:

- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: ${{ env.ARTIFACT_NAME }}
path: /tmp/gh-ost-test.*
Expand Down
33 changes: 25 additions & 8 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
run:
timeout: 5m
version: "2"
linters:
disable:
- errcheck
enable:
- bodyclose
- containedctx
Expand All @@ -11,18 +8,38 @@ linters:
- durationcheck
- errname
- errorlint
- gofmt
- misspell
- nilerr
- nilnil
- noctx
- nolintlint
- nosprintfhostport
- prealloc
- rowserrcheck
- sqlclosecheck
- unconvert
- unparam
- unused
- wastedassign
- whitespace
disable:
- errcheck
- noctx
- prealloc
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
paths:
- third_party$
- builtin$
- examples$
formatters:
enable:
- gofmt
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
31 changes: 15 additions & 16 deletions Dockerfile.packaging
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
FROM golang:1.23-bullseye
FROM golang:1.25.12-bookworm AS build

RUN apt-get update
RUN apt-get install -y ruby ruby-dev rubygems build-essential
RUN gem install fpm
ENV GOPATH=/tmp/go
# build.sh shells out to fpm to produce the .deb/.rpm packages. fpm is a Ruby
# gem (needs ruby + ruby-dev + a C toolchain), and the rpm target needs
# rpmbuild from the `rpm` package. Everything else build.sh uses (git, tar,
# gcc, bash, ...) already ships in the golang:bookworm base image.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ruby ruby-dev build-essential rpm \
&& rm -rf /var/lib/apt/lists/* \
&& gem install --no-document fpm

RUN apt-get install -y curl
RUN apt-get install -y rsync
RUN apt-get install -y gcc
RUN apt-get install -y g++
RUN apt-get install -y bash
RUN apt-get install -y git
RUN apt-get install -y tar
RUN apt-get install -y rpm

RUN mkdir -p $GOPATH/src/github.com/github/gh-ost
WORKDIR $GOPATH/src/github.com/github/gh-ost
WORKDIR /gh-ost
COPY . .
RUN bash build.sh

# Export-only stage: `docker build --target artifacts --output type=local,dest=...`
# writes just the release assets to the host, no intermediate container needed.
FROM scratch AS artifacts
COPY --from=build /tmp/gh-ost-release/ /
4 changes: 2 additions & 2 deletions Dockerfile.test
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.23-bullseye
FROM golang:1.25.12-bookworm
LABEL maintainer="github@github.com"

RUN apt-get update
Expand All @@ -8,4 +8,4 @@ RUN rm -rf /var/lib/apt/lists/*
COPY . /go/src/github.com/github/gh-ost
WORKDIR /go/src/github.com/github/gh-ost

CMD ["script/test"]
CMD ["script/test", "-short=1"]
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ Please see [Coding gh-ost](doc/coding-ghost.md) for a guide to getting started d

`gh-ost` is a Go project; it is built with Go `1.15` and above. To build on your own, use either:
- [script/build](https://github.com/github/gh-ost/blob/master/script/build) - this is the same build script used by CI hence the authoritative; artifact is `./bin/gh-ost` binary.
- [build.sh](https://github.com/github/gh-ost/blob/master/build.sh) for building `tar.gz` artifacts in `/tmp/gh-ost-release`
- [build.sh](https://github.com/github/gh-ost/blob/master/build.sh) for building the release artifacts (`tar.gz` archives plus `.rpm`/`.deb` packages and a `SHA256SUMS` manifest) in `/tmp/gh-ost-release`

Generally speaking, `master` branch is stable, but only [releases](https://github.com/github/gh-ost/releases) are to be used in production.

Expand Down
64 changes: 40 additions & 24 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,46 +3,61 @@
RELEASE_VERSION=
buildpath=

# Create a scratch tree to stage the package filesystem for fpm. It lives in the
# system temp dir, not $buildpath, so it never lands among the release artifacts.
function setuptree() {
b=$( mktemp -d $buildpath/gh-ostXXXXXX ) || return 1
b=$( mktemp -d ) || return 1
mkdir -p $b/gh-ost
mkdir -p $b/gh-ost/usr/bin
echo $b
}

function build {
osname=$1
osshort=$2
GOOS=$3
GOARCH=$4
GOOS=$1
GOARCH=$2

if ! go version | egrep -q 'go1\.(1[5-9]|[2-9][0-9]{1})' ; then
echo "go version must be 1.15 or above"
exit 1
fi

echo "Building ${osname}-${GOARCH} binary"
echo "Building ${GOOS}-${GOARCH} binary"
export GOOS
export GOARCH
go build -ldflags "$ldflags" -o $buildpath/$target go/cmd/gh-ost/main.go
CGO_ENABLED="${CGO_ENABLED:-0}" go build -ldflags "$ldflags" -o "$buildpath/$target" go/cmd/gh-ost/main.go

if [ $? -ne 0 ]; then
echo "Build failed for ${osname} ${GOARCH}."
echo "Build failed for ${GOOS} ${GOARCH}."
exit 1
fi

(cd $buildpath && tar cfz ./gh-ost-binary-${osshort}-${GOARCH}-${timestamp}.tar.gz $target)
(cd $buildpath && tar cfz ./gh-ost-${GOOS}-${GOARCH}.tar.gz $target)

# build RPM and deb for Linux, x86-64 only
if [ "$GOOS" == "linux" ] && [ "$GOARCH" == "amd64" ] ; then
# build RPM and deb packages for Linux
if [ "$GOOS" == "linux" ] ; then
echo "Creating Distro full packages"
case "$GOARCH" in
amd64) rpm_arch=x86_64 ; deb_arch=amd64 ;;
arm64) rpm_arch=aarch64 ; deb_arch=arm64 ;;
*) rpm_arch=$GOARCH ; deb_arch=$GOARCH ;;
esac
builddir=$(setuptree)
cp $buildpath/$target $builddir/gh-ost/usr/bin
cd $buildpath
fpm -v "${RELEASE_VERSION}" --epoch 1 -f -s dir -n gh-ost -m 'GitHub' --description "GitHub's Online Schema Migrations for MySQL " --url "https://github.com/github/gh-ost" --vendor "GitHub" --license "Apache 2.0" -C $builddir/gh-ost --prefix=/ -t rpm --rpm-rpmbuild-define "_build_id_links none" --rpm-os linux .
fpm -v "${RELEASE_VERSION}" --epoch 1 -f -s dir -n gh-ost -m 'GitHub' --description "GitHub's Online Schema Migrations for MySQL " --url "https://github.com/github/gh-ost" --vendor "GitHub" --license "Apache 2.0" -C $builddir/gh-ost --prefix=/ -t deb --deb-no-default-config-files .

fpm_opts=(
-v "${RELEASE_VERSION}" --epoch 1 -f -s dir -n gh-ost
-m 'GitHub' --description "GitHub's Online Schema Migrations for MySQL "
--url "https://github.com/github/gh-ost" --vendor "GitHub" --license "Apache 2.0"
-C "$builddir/gh-ost" --prefix=/
)
fpm "${fpm_opts[@]}" -t rpm -a "${rpm_arch}" -p "gh-ost.${rpm_arch}.rpm" --rpm-rpmbuild-define "_build_id_links none" --rpm-os linux .
fpm "${fpm_opts[@]}" -t deb -a "${deb_arch}" -p "gh-ost.${deb_arch}.deb" --deb-no-default-config-files .
cd -
fi

# Drop the bare binary so only tarballs and packages are published as release assets
rm -f "$buildpath/$target"
}

main() {
Expand All @@ -60,22 +75,23 @@ main() {

buildpath=/tmp/gh-ost-release
target=gh-ost
timestamp=$(date "+%Y%m%d%H%M%S")
ldflags="-X main.AppVersion=${RELEASE_VERSION} -X main.GitCommit=${GIT_COMMIT}"

mkdir -p ${buildpath}
rm -rf ${buildpath:?}/*
build GNU/Linux linux linux amd64
build GNU/Linux linux linux arm64
build macOS osx darwin amd64
build macOS osx darwin arm64

bin_files=$(find $buildpath/gh-ost* -type f -maxdepth 1)
echo "Binaries found in:"
echo "$bin_files"

build linux amd64
build linux arm64
build darwin amd64
build darwin arm64

# Generate a SHA256SUMS manifest with basenames only, so a downloaded set can be
# verified in place with `sha256sum -c SHA256SUMS` (no absolute build paths leak in).
# The name has no gh-ost prefix, so it self-excludes from the gh-ost* glob below.
echo "Checksums:"
(shasum -a256 $bin_files 2>/dev/null)
( cd "$buildpath" && shasum -a256 gh-ost* | tee SHA256SUMS )

echo "Release assets:"
ls -1 "$buildpath"

echo "Build Success!"
}
Expand Down
2 changes: 1 addition & 1 deletion doc/coding-ghost.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ If additional steps are needed, please add them into this workflow so that the w

## `golang-ci` linter

To enfore best-practices, Pull Requests are automatically linted by [`golang-ci`](https://golangci-lint.run/). The linter config is located at [`.golangci.yml`](https://github.com/github/gh-ost/blob/master/.golangci.yml) and the `golangci-lint` GitHub Action is located at [`.github/workflows/golangci-lint.yml`](https://github.com/github/gh-ost/blob/master/.github/workflows/golangci-lint.yml).
To enforce best-practices, Pull Requests are automatically linted by [`golang-ci`](https://golangci-lint.run/). The linter config is located at [`.golangci.yml`](https://github.com/github/gh-ost/blob/master/.golangci.yml) and the `golangci-lint` GitHub Action is located at [`.github/workflows/golangci-lint.yml`](https://github.com/github/gh-ost/blob/master/.github/workflows/golangci-lint.yml).

To run the `golang-ci` linters locally _(recommended before push)_, use `script/lint`.

Expand Down
Loading