Repository navigation
PG19: use ShareUpdateExclusiveLock for REPACK CONCURRENTLY - #8805
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8805 +/- ##
==========================================
+ Coverage 88.81% 88.82% +0.01%
==========================================
Files 290 290
Lines 65494 65495 +1
Branches 8260 8261 +1
==========================================
+ Hits 58166 58175 +9
+ Misses 4942 4937 -5
+ Partials 2386 2383 -3 🚀 New features to boost your workflow:
|
This was referenced Sep 28, 2026
Contributor
Author
|
Beta 4 disposition: retained and still relevant. Please rebase onto current main and refresh the PG19 lock-contention regression; this PR should remain independent of the closed eager-aggregation stack. |
ibrahim halatci (ihalatci)
force-pushed
the
ihalatci-pg19-repack-lock-fix
branch
from
September 28, 2026 14:09
cc7c6a1 to
df2f245
Compare
This was referenced Oct 16, 2026
ibrahim halatci (ihalatci)
force-pushed
the
ihalatci-pg19-repack-lock-fix
branch
from
October 16, 2026 08:38
df2f245 to
805849c
Compare
ibrahim halatci (ihalatci)
changed the base branch from
main
to
ihalatci-postgresql-19-rc1
October 16, 2026 08:38
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Oct 16, 2026
DESCRIPTION: Update PostgreSQL 19 CI and regression coverage to RC1 Update build, nightly cassert, and pg_upgrade image pins to the published PostgreSQL 19 RC1 release images from merged citusdata/the-process#253 (image suffix `-va0a5582`). The stable image publication succeeded for merge commit `a0a5582b39e72cec476f7b6ba37a2cf97572b930` in [the-process build-test-images run](https://github.com/citusdata/the-process/actions/runs/38888435235). Preserve the existing exclusion of PG19 from Citus upgrade coverage. Add a distributed `ON CONFLICT` regression for PostgreSQL's fix to inference when an index expression on a NOT NULL column simplifies to a Var ([upstream commit](postgres/postgres@f9ccc5e)). No Citus source change was indicated: the regression fails on PostgreSQL 19 Beta 4 and passes on RC1. ## RC1 feature audit The audited upstream range is `REL_19_BETA4` (`b73d13c32c834a2c8e1c60cb92f79530376cedf1`) through `REL_19_RC1` (`0b8bff042a997fe50fcd1c4c83b9ef1cc9ed8a89`): 132 commits and 300 changed paths, with no feature-revert commit. The Citus umbrella features remain present in RC1: - REPACK / REPACK CONCURRENTLY: #8756 / #8805 remain relevant. - EXPLAIN IO: #8770 / #8807 remain relevant. - Constraint enforceability: #8758 / #8808 remain relevant. - JSON COPY: merged #8760 / #8765 remains compatible with RC1. - Distributed REPACK CONCURRENTLY support: #8772 remains a design-only exception without an implementation PR. The approved dependent topology is: - native stack `#8929 -> #8805 -> #8807`; - #8808 as an independent sibling based directly on this branch after removing unrelated DETACH changes. Version-neutral #8759 and #8762 are no longer subissues of #8782. PostgreSQL RC2 is scheduled for 2026-11-12 and GA for 2026-11-19. RC2 must be rechecked for the open upstream eager-aggregation and CHECK-enforcement items before final PG19 feature completion. ## Validation Local validation on PostgreSQL 19 RC1: `pg19` (7/7), `multi_upsert` (7/7), `multi_insert_select_conflict` (7/7), `multi_copy` (15/15), `multi_outer_join` (7/7), `multi_deparse_shard_query` (7/7), `multi_deparse_function` (7/7), and `multi_join_order_additional` (16/16, with base schedule and whole schedule line). `merge` also passed on Beta 4 after installing the missing `postgres_fdw` test dependency. The repartition planning/pruning/task-assignment debug task-ID output differences reproduce on both Beta 4 and RC1; no expected output was changed for them. The prior Citus head `7259ac9c94e9a6595eac092b18afc47b69197d4a` passed all automatic checks. This suffix-only update switches CI to the successfully published stable RC1 images; checks for the current head will run after this update. --------- Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Copilot-Session: f2aeb1f7-5872-4dc8-a288-2bdafcf80c04
ibrahim halatci (ihalatci)
force-pushed
the
ihalatci-pg19-repack-lock-fix
branch
from
October 16, 2026 11:15
805849c to
33f3457
Compare
ibrahim halatci (ihalatci)
marked this pull request as ready for review
October 16, 2026 11:18
PreprocessClusterStmt resolved the target relation with
AccessExclusiveLock unconditionally. That is correct for CLUSTER and
for plain REPACK, but PostgreSQL deliberately runs REPACK
(CONCURRENTLY) under ShareUpdateExclusiveLock (RepackLockLevel() in
repack.c). Taking the stronger lock has two consequences:
* On a distributed table, a session holding only AccessShareLock
makes Citus block on AccessExclusiveLock, so the intended
"CONCURRENTLY is unsupported" error is masked by a lock wait.
* The lock is taken before the !IsCitusTable() early return and is
held until end of transaction, so on a plain local table the
repack decoding worker cannot lock the relation to export its
logical snapshot and the command waits forever on
RepackWorkerExport.
Resolve the relation with ShareUpdateExclusiveLock when CONCURRENTLY
is requested and keep AccessExclusiveLock for every other case, so
ordinary CLUSTER and REPACK locking is unchanged.
The unsupported-feature guard for distributed tables is retained; this
commit only stops it from being masked.
Add PG19-only regression coverage for both paths: a distributed table
whose rejection must not be masked by a concurrent reader, and a plain
local table whose REPACK (CONCURRENTLY) must complete.
Fixes #8756
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 61c1aeb4-1b56-4313-a35e-d65b59a557de
ibrahim halatci (ihalatci)
force-pushed
the
ihalatci-pg19-repack-lock-fix
branch
from
October 21, 2026 09:39
33f3457 to
a1ba764
Compare
Onur Tirtir (onurctirtir)
approved these changes
Oct 21, 2026
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Oct 23, 2026
Propagate PostgreSQL 19 RC1 EXPLAIN IO to worker nodes. ## Stack Native stack layer 3: #8929 -> #8805 -> #8807 This PR is based on `ihalatci-pg19-repack-lock-fix` at `805849c8ed7b3fa5e77edf289c5e2b159b4bb529`. ## Changes - Propagate `ExplainState.io` to worker EXPLAIN commands on PostgreSQL 19+. - Preserve `INSTRUMENT_IO` through the saved-plan EXPLAIN ANALYZE path. - Cover both saved-plan and live remote-EXPLAIN paths in `pg19` regression tests. - Keep PostgreSQL 17/18 behavior unchanged through existing version guards. The diff relative to #8805 is limited to: - `src/backend/distributed/planner/multi_explain.c` - `src/test/regress/bin/normalize.sed` - `src/test/regress/sql/pg19.sql` - `src/test/regress/expected/pg19.out` ## Validation - PostgreSQL 19rc1 Citus configure, build, and install: passed. - `src/test/regress/citus_tests/run_test.py pg19`: passed (7/7). - `src/test/regress/citus_tests/run_test.py multi_explain`: passed (16/16). - `src/test/regress/citus_tests/run_test.py multi_mx_explain`: passed (10/10). - PostgreSQL 17.10 configure/build: passed. - PostgreSQL 18.4 configure/build: passed. - `citus_indent --check`: passed. - `ci/editorconfig.sh`: passed. - `git diff --check`: passed. - `make check-style`: repository-baseline flake8 failures only; reproduced unchanged on #8805. The changed C file and editorconfig checks pass. Fixes #8770 --------- Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6481a75c-8b50-4476-832e-fbe9e4fb1196 Copilot-Session: 3964bc45-9e47-4861-9c6c-a31d8e900ea0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PG19 RC1: match PostgreSQL's ShareUpdateExclusiveLock for REPACK
(CONCURRENTLY), avoiding a preprocessing lock escalation that can mask the
unsupported distributed-table error or hang repack on a local table.
Native stack layer 2: #8929 -> #8805 -> #8807.
Validation (Ubuntu 22.04 WSL under
/home/ihalatci/Development):PG_CONFIG=.../postgresql-install/bin/pg_config ./configure && make -j8 && make installwith PostgreSQL 19rc1: passed.PATH=.../postgresql-install/bin:$PATH src/test/regress/citus_tests/run_test.py pg19: all 7 tests passed, includingpg19.PG_CONFIG=~/.pgenv/pgsql-17.10/bin/pg_config ./configure && make -j8: passed.PG_CONFIG=~/.pgenv/pgsql-18.4/bin/pg_config ./configure && make -j8: passed.citus_indent --check --quiet: passed.git diff --check 7259ac9c94e9a6595eac092b18afc47b69197d4a..HEAD: passed.make check-style:blackandisortpassed;flake8reached pre-existing naming violations in unchanged Python files, unrelated to this C/SQL-only diff.Fixes #8756