Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions .github/workflows/run-tests-tiered.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,8 @@ jobs:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

- name: Build docs
uses: ammaraskar/sphinx-action@c61ac11d9ee097caf8983c10c8b5af5861b32b54 # master
with:
docs-folder: "docs/"
- name: Lint Markdown
run: make lint-docs

- name: Set Version String From .git
run: make version
Expand Down
20 changes: 20 additions & 0 deletions .markdownlint-cli2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
globs:
- "docs/**/*.md"
- "README.md"
- "CONTRIBUTING.md"

config:
# Help blocks and terminal transcripts set their own width.
MD013: false

# Help blocks are generated; their fences carry no language.
MD040: false

# Command examples are written with a "$" prompt and no output.
MD014: false

# Bold labels are used for option names and short definitions.
MD036: false

# Two admonitions in a row read as one quote with a blank line.
MD028: false
22 changes: 0 additions & 22 deletions .readthedocs.yaml

This file was deleted.

8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,15 +57,15 @@ CI enforces style — PRs with formatting issues will fail the style check.

## Documentation

Documentation lives in `docs/` and is built with Sphinx (reStructuredText).
Documentation lives in `docs/` as Markdown, which GitHub renders directly.

**Never manually edit files in `docs/include/*.rst`.** These are auto-generated from CLI help text and manual edits will be overwritten.
**Never manually edit the text between `<!-- BEGIN HELP: ... -->` and `<!-- END HELP -->` markers.** Those blocks are auto-generated from CLI help text and manual edits will be overwritten.

When adding or modifying CLI commands:

1. Update help text in the C source files
2. Build in container: `PGVERSION=18 make build`
3. Run `make update-docs` to regenerate `docs/include/*.rst`
3. Run `make update-docs` to refresh the help blocks
4. Commit both the code changes and the generated docs

If `make update-docs` fails locally, use the container binary:
Expand Down Expand Up @@ -182,6 +182,6 @@ Never create PRs against the upstream `dimitri/pgcopydb` repository without expl

**Never:**
- Build on the host system
- Manually edit `docs/include/*.rst`
- Manually edit the text inside `<!-- BEGIN HELP: ... -->` markers
- Force push to any branch
- Create PRs against upstream `dimitri/pgcopydb`
51 changes: 33 additions & 18 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,28 @@
# Contributing to pgcopydb

PlanetScale maintains this fork directly and does not merge changes back to the
upstream project. Send everything here, not to `dimitri/pgcopydb`. See
[About this fork](README.md#about-this-fork).

We're happy you want to contribute! You can help us in different ways:

* Open an [issue](https://github.com/dimitri/pgcopydb/issues) with suggestions
for improvements, potential bugs, etc.
* Open an [issue](https://github.com/planetscale/pgcopydb/issues) with
suggestions for improvements, potential bugs, etc.
* Fork this repository and submit a pull request

### Building from source code
## Building from source code

Follow the relevant docs at our
[documentation](https://pgcopydb.readthedocs.io/en/latest/install.html#build-from-sources).
Follow [Build from sources](docs/install.md#build-from-sources) in the
documentation.

### Following our coding conventions
## Following our coding conventions

CI pipeline will automatically reject any PRs which do not follow our coding
conventions. The easiest way to ensure your PR adheres to those conventions is
to use the
[citus_indent](https://github.com/citusdata/tools/tree/develop/uncrustify) tool.
This tool uses `uncrustify` under the hood.


```bash
# On debian, run the following to install uncrustify.
# On other distros, use your package manager for a similar command.
Expand Down Expand Up @@ -47,7 +50,7 @@ __EOF__
chmod +x .git/hooks/pre-commit
```

### Running tests
## Running tests

pgcopydb uses docker containers to create test environments. Each test lives
under a separate directory under `tests/` directory at the top directory of the
Expand All @@ -63,20 +66,32 @@ make tests
make tests/pagila
```

### Documentation
## Documentation

User-facing documentation is published on
[pgcopydb.readthedocs.io](https://pgcopydb.readthedocs.io/). When adding a new
feature, function, or setting, you are expected to add relevant documentation
change in your pull request.
User-facing documentation lives in [docs/](docs/) as Markdown, which GitHub
renders directly. When adding a new feature, function, or setting, you are
expected to add relevant documentation change in your pull request.

If you changed the help output of a pgcopydb command, you are expected to update
the relevant pieces of our documentation. This can be done automatically by
running the following command:
If you changed the help output of a pgcopydb command, run:

```bash
make update-docs
```

This command will update the relevant documentation templates according to your
latest code changes.
Each documentation page embeds command help between markers:

```
<!-- BEGIN HELP: pgcopydb clone -->
...
<!-- END HELP -->
```

`make update-docs` builds the binary and rewrites the block between each pair of
markers with the current help output. CI fails when a page is out of date, so
run it and commit the result. Do not edit the text inside those markers by hand.

Markdown is linted in CI, and the same check runs locally in a container:

```bash
make lint-docs # markdownlint, configured by .markdownlint-cli2.yaml
```
1 change: 0 additions & 1 deletion Dockerfile.debian
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ RUN apt-get update \
postgresql-server-dev-all \
postgresql-common \
postgresql \
python3-sphinx \
lintian \
curl \
&& rm -rf /var/lib/apt/lists/*
Expand Down
1 change: 0 additions & 1 deletion Dockerfile.debian-qa
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ RUN apt-get update \
postgresql-server-dev-all \
postgresql-common \
postgresql \
python3-sphinx \
lintian \
curl \
&& rm -rf /var/lib/apt/lists/*
Expand Down
10 changes: 6 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ PGCOPYDB ?= $(TOP)src/bin/pgcopydb/pgcopydb
PGVERSION ?= 18
DOCKER ?= docker

MARKDOWNLINT ?= davidanson/markdownlint-cli2@sha256:9ae6011b3d978315ad283bf2af997ef08687a797b9e55086c94a90bf620f0783

all: bin ;

GIT-VERSION-FILE:
Expand All @@ -26,9 +28,6 @@ maintainer-clean:
$(MAKE) -C src/bin/ maintainer-clean
rm -f version

docs:
$(MAKE) -C docs clean man html

update-docs: bin
bash ./docs/update-help-messages.sh

Expand All @@ -37,6 +36,9 @@ check-docs:
$(DOCKER) build --file=ci/Dockerfile.docs --tag test-docs .
$(DOCKER) run test-docs

lint-docs:
$(DOCKER) run --rm -v "$(CURDIR)":/workdir $(MARKDOWNLINT)

test: build
$(MAKE) -C tests all

Expand Down Expand Up @@ -79,7 +81,7 @@ debsh-qa: deb-qa
$(DOCKER) run --rm -it pgcopydb_debian_qa bash

.PHONY: all
.PHONY: bin clean install docs maintainer-clean update-docs
.PHONY: bin clean install maintainer-clean update-docs lint-docs
.PHONY: test tests tests/ci tests/*
.PHONY: deb debsh deb-qa debsh-qa
.PHONY: GIT-VERSION-FILE
30 changes: 24 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
# pgcopydb

[![Documentation Status](https://readthedocs.org/projects/pgcopydb/badge/?version=latest)](https://pgcopydb.readthedocs.io/en/latest/?badge=latest)
## About this fork

PlanetScale launched this fork of pgcopydb in 2026 and maintains it directly.

pgcopydb was created by [Dimitri Fontaine](https://github.com/dimitri), whose
design carries this project: parallel COPY without intermediate files,
concurrent index builds, and change data capture through logical decoding. The
fork builds on that work.

PlanetScale no longer targets full parity with the upstream project, and does
not merge this work back to it. Direction comes from what PlanetScale customers
need when they migrate production databases. A fix that upstream also carries
gets ported deliberately, one change at a time, rather than by tracking the
upstream branch.

Report issues and open pull requests against
[planetscale/pgcopydb](https://github.com/planetscale/pgcopydb).

## Introduction

Expand Down Expand Up @@ -39,9 +55,8 @@ implement the offline migration approach.

## Documentation

Full documentation is available online, including manual pages of all the
pgcopydb sub-commands. Check out
[https://pgcopydb.readthedocs.io/](https://pgcopydb.readthedocs.io/en/latest/).
The documentation lives in [docs/](docs/), including manual pages of all the
pgcopydb sub-commands. Start at [docs/README.md](docs/README.md).

```
$ pgcopydb help
Expand Down Expand Up @@ -246,6 +261,7 @@ dotenv pgcopydb clone
- **`PGCOPYDB_TARGET_PGURI`** - PostgreSQL connection string for the target database

Connection strings follow the standard PostgreSQL format:

```
postgres://[user[:password]@][host][:port][/dbname][?param1=value1&...]
```
Expand Down Expand Up @@ -280,11 +296,12 @@ PGSSLROOTCERT="/path/to/ca-cert.pem"
PGCOPYDB_SOURCE_PGURI="postgres://user@prod.example.com:5432/myapp?sslmode=verify-full"
PGCOPYDB_TARGET_PGURI="postgres://user@backup.example.com:5432/myapp?sslmode=verify-full"
```

See the included `.env.example` file for a complete configuration template.

## Installing pgcopydb

See our [documentation](https://pgcopydb.readthedocs.io/en/latest/install.html).
See [docs/install.md](docs/install.md).

## Design Considerations (why oh why)

Expand Down Expand Up @@ -407,7 +424,8 @@ of the operations. See documentation for `pgcopydb snapshot`.

## Authors

* [Dimitri Fontaine](https://github.com/dimitri)
- [PlanetScale team](https://github.com/planetscale) (2026 onwards)
- [Dimitri Fontaine](https://github.com/dimitri) (pre 2026)

## License

Expand Down
1 change: 0 additions & 1 deletion debian/control
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ Build-Depends:
libzstd-dev,
postgresql,
postgresql-server-dev-all (>= 158~),
python3-sphinx,
zlib1g-dev,
Standards-Version: 4.6.0
Homepage: https://github.com/dimitri/pgcopydb
Expand Down
5 changes: 0 additions & 5 deletions debian/rules
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,9 @@ VERSION = $(shell dpkg-parsechangelog --show-field version)
override_dh_auto_build:
$(MAKE) -C src/bin/pgcopydb GIT_VERSION=$(VERSION) git-version.h
$(MAKE) -C src/bin/pgcopydb
$(MAKE) -C docs man

override_dh_auto_clean:
$(MAKE) -C src/bin/pgcopydb clean
$(MAKE) -C docs clean

override_dh_auto_test:
PATH=$(CURDIR)/src/bin/pgcopydb:$(PATH) debian/tests/copydb
Expand All @@ -19,8 +17,5 @@ override_dh_auto_install:
DESTDIR=$(CURDIR)/debian/pgcopydb \
BINDIR=/usr/bin

override_dh_installman:
dh_installman docs/_build/man/*

%:
dh $@
20 changes: 0 additions & 20 deletions docs/Makefile

This file was deleted.

Loading
Loading