fix(k8s): reconcile Cilium devices on join#892
Open
xtrusia wants to merge 2 commits into
Open
Conversation
Signed-off-by: Seyeong Kim <seyeong.kim@canonical.com>
Signed-off-by: Seyeong Kim <seyeong.kim@canonical.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commit 0a383c1 (0a383c1) binds the Kubernetes cluster endpoint to the internal space.
During a local control node join, this binding can change while Cilium on existing control nodes still uses the management interface.
EnsureCiliumDeviceByHostSteponly checked the joining node, so existing nodes kept the old device configuration until a full refresh.The local join plan now checks the joining node and existing control nodes that already have the Sunbeam hostname label.
FQDN-scoped reconciliation still avoids stale resource deletion.
Cilium reconciliation uses the existing
k8s-planlock.This serializes it with Terraform apply and other join or refresh commands.
Clusterd creates missing locks with create-only semantics so concurrent first requests cannot both succeed.
EnsureCiliumDeviceByHostStepcalculates and applies changes once while holding the lock and releases it fromfinally.QA steps
The regression was reproduced with two Ubuntu 24.04 LXD VMs.
Both VMs had management and internal interfaces, with the default route only on the management interface.
internalspace and install a test build of revision 996 that mapsNetworks.INTERNALto that space.InternalIP,CiliumNodeConfig, Cilium pod identity, and live Cilium device on node 1.sunbeam cluster refresh --force --no-hintsand check the same state again.During the join, the endpoint binding and node 1
InternalIPmoved to the internal network.Node 1 kept its original Cilium pod and management device.
The full refresh updated its
CiliumNodeConfig, replaced the pod, and moved Cilium to the internal device.To test this change, restore the
pre-endpoint-transitionsnapshots, install a snap built from this branch, and repeat the node 2 join.Node 1 should move to the internal device during the join without a later full refresh.
Links