Skip to content

net: mesh SitePeering with a manageCniPlugin=false Site is one-sided when its nodes have no PodCIDRs #992

Open

Description

Since #943, nodes in a Site with manageCniPlugin: false and no assignmentEnabled: true have no
PodCIDRs, as intended. In a SitePeering with meshNodes: true, peering then becomes one-sided:

  • The unmanaged-Site node keeps the remote Site's nodes as mesh peers (directly peered remote Sites
    are kept, cmd/unbounded-net-node/site_watch_reconcile.go:1812-1816) and routes their pod CIDRs
    into unbounded0.
  • The remote nodes drop it, because slice peers without PodCIDRs are skipped
    (site_watch_reconcile.go:1833-1837; tunnel_config.go:465 and :741 for BPF entries). In a kind
    lab the remote node has 0 peers and no geneve0, so the encapsulated packets are dropped at its
    eth0.

Before #943 these nodes picked up a stray PodCIDR, so host-network traffic to remote pods worked.
Nodes that already have one keep it (spec.podCIDR is immutable), so after an upgrade existing nodes
keep working and newly added nodes do not.

Lab: unmanaged Calico Site cluster, managed Site flex-site, SitePeering meshNodes: true over
GENEVE, no gateway pool.

Path main before #943 (93eced7) main (ae6dfab)
cluster node 1 (host) -> remote pod OK FAIL
cluster node 2 (host) -> remote pod OK FAIL
pod -> remote pod, either direction FAIL FAIL

Pod-to-pod fails in both columns; that is the return-path question in #921. The runs on current main
disable the #964 neighbor tuner, which kind's network namespace does not support.

The docs describe existing-CNI Sites as reaching other Sites through gateways
(docs/net/custom-resources.md:99-102), while the code keeps directly peered remote Sites as mesh
peers for them, and delegated IPAM with assignmentEnabled: true gives nodes PodCIDRs and so meshes
normally. Is mesh peering meant to work for nodes without PodCIDRs (then the docs need an update), or
only through gateways or delegated IPAM?

  1. Mesh requires a PodCIDR: the unmanaged node also skips remote mesh peers while it has none and
    logs why, plus a docs note that such Sites need a gateway pool or delegated IPAM.
  2. Host-only mesh peers: remote nodes keep PodCIDR-less peers from unmanaged Sites and program only
    their node /32 pins, which restores the host-network paths above.

Happy to send a PR for either, including the docs update.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions