Skip to content

[PULP-1045] Add support for serving superseded publications for a retention period#7546

Open
pedro-psb wants to merge 1 commit intopulp:mainfrom
pedro-psb:feature/add-distributed-publication
Open

[PULP-1045] Add support for serving superseded publications for a retention period#7546
pedro-psb wants to merge 1 commit intopulp:mainfrom
pedro-psb:feature/add-distributed-publication

Conversation

@pedro-psb
Copy link
Copy Markdown
Member

@pedro-psb pedro-psb commented Mar 30, 2026

Introduces a DistributedPublication model to track which publications are associated with each distribution. When a new publication becomes the latest for a repository, older ones are retained and served by the content handler for a configurable grace period, allowing clients to continue fetching from superseded publications without interruption.

Handles publications with pass-trough=True|False. The behavior is sumarized:

Given:

  • CA=ContentArtifact,
  • PA=PublishedAritfact
  • PU=Publication
  • RV=RepositoryVersion
  • DP=DistributedPublication
  • DT=Distribution

Then CA is retrieved with:

  • pass-through: CA<-PA<-PU<-DP->DT
  • no-pass-through: CA<-RV<-PU<-DP->DT

Closes: #7514

📜 Checklist

  • Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
  • A changelog entry or entries has been added for any significant changes
  • Follows the Pulp policy on AI Usage
  • (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

@pedro-psb pedro-psb changed the title Add support for serving superseded publications for a retention period [PULP-1045] Add support for serving superseded publications for a retention period Mar 30, 2026
@pedro-psb pedro-psb force-pushed the feature/add-distributed-publication branch 5 times, most recently from 4b940b8 to 2ad945c Compare April 1, 2026 13:48
@pedro-psb pedro-psb force-pushed the feature/add-distributed-publication branch from 2ad945c to c87f525 Compare April 1, 2026 13:51
@pedro-psb pedro-psb marked this pull request as ready for review April 1, 2026 13:58
@mdellweg
Copy link
Copy Markdown
Member

mdellweg commented Apr 1, 2026

Can you make sure this doesn't collide with https://github.com/pulp/pulp_deb/blob/main/pulp_deb/app/models/publication.py#L136
And maybe coordinate an effort to superseed it there?

@pedro-psb
Copy link
Copy Markdown
Member Author

Can you make sure this doesn't collide with https://github.com/pulp/pulp_deb/blob/main/pulp_deb/app/models/publication.py#L136

I've tested with pulp_deb running alogside. There was a collision with the reverse relationship name, then I've simply renamed the reverse relationship. I'll create a test PR on deb to be sure.

And maybe coordinate an effort to superseed it there?

Yes, I'll open an issue there and start a conversation.

@pedro-psb
Copy link
Copy Markdown
Member Author

The test PR is fine: pulp/pulp_deb#1429

@pedro-psb
Copy link
Copy Markdown
Member Author

I've opened the issue for the superseeding with some options: pulp/pulp_deb#1430

@pedro-psb
Copy link
Copy Markdown
Member Author

I want a review of this, but I'd like pulp_deb folks feedback (on the general operation) before merging.

@pedro-psb pedro-psb requested review from gerrod3 and removed request for gerrod3 April 9, 2026 14:10
@pedro-psb pedro-psb force-pushed the feature/add-distributed-publication branch 3 times, most recently from a8afc90 to 85e7550 Compare April 13, 2026 15:00
Copy link
Copy Markdown
Member

@mdellweg mdellweg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite a complex thing, but looks good over all.

Comment thread docs/admin/reference/settings.md Outdated

Defaults to `False`.

### DISTRIBUTED\_PUBLICATION\_RETENTION\_PERIOD
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a global thing?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be a more complex setting and be defined per plugin, or not exposed at all. I could be convinced either way. I decided to start with this as a middle ground.

pub.delete()
assert not DistributedPublication.objects.filter(distribution=dist).exists()

def test_deleting_older_repository_version_doesnt_clear_dps(self, db):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's dps? (Actually there's no need for abbreviation in test case names. They don't have call sites.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah DistributedPublication(s). That's covered by the class name. How about:
test_does_not_happen_when_deleting_old_repository_version?

@pedro-psb pedro-psb requested a review from mdellweg April 15, 2026 15:03
@mdellweg
Copy link
Copy Markdown
Member

Can you squash before merging?

Comment thread docs/admin/reference/settings.md Outdated
Introduces a DistributedPublication model to track which publications are
associated with each distribution. When a new publication becomes the latest
for a repository, older ones are retained and served by the content handler
for a configurable grace period, allowing clients to continue fetching from
superseded publications without interruption.

Handles publications with pass-trough=True|False. The behavior is sumarized:

Given:
- CA=ContentArtifact,
- PA=PublishedAritfact
- PU=Publication
- RV=RepositoryVersion
- DP=DistributedPublication
- DT=Distribution

Then CA is retrieved with:
- no-pass-through: CA<-PA<-PU<-DP->DT
- pass-through:    CA<-RV<-PU<-DP->DT

Closes: pulp#7514
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@pedro-psb pedro-psb force-pushed the feature/add-distributed-publication branch from b34f6e6 to 1f7224a Compare April 16, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Distributed Publication Retention Cache

3 participants