Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ PastureStack is an independent community effort to preserve, audit, and moderniz

Earlier prerelease coordinates are retired from current release references;
their reviewed source commits remain in Git history. This source tree targets
the pure numeric coordinate `v0.3.10`; the GitHub tag and Release, rather than
the pure numeric coordinate `v0.3.11`; the GitHub tag and Release, rather than
this README, determine when it is published. Product identity is carried by
the repository, catalog metadata, and provenance rather than the version tag.

Expand Down Expand Up @@ -68,11 +68,14 @@ health-reporting, and encrypted-workload gates. The scheduler passed source,
build, security, public distribution, live Metadata, idempotent reservation,
managed allocation, and restart gates. Version `v0.8.15` additionally remained
healthy through repeated Metadata long-poll windows in production without a
second container start. Network Services version `6` moves to `v0.8.18`,
second container start. Network Services version `7` moves to `v0.8.19`,
rejects malformed per-host subnet labels before applying host firewall rules,
and preserves routed container source IPs between validated active peers. It
also fixes bidirectional VXLAN traffic when published host ports coexist with
the overlay. Layer 2 Flat Network version `4` moves to `v0.14.36` so the CNI
the overlay, binds forwarding rules to the exact managed bridge, and protects
bridge traffic from `route_localnet` loopback routing while preserving and
restoring the operator's original per-bridge setting. Layer 2 Flat Network
version `4` moves to `v0.14.36` so the CNI
preserves an operator-configured bridge address.
Restored-data provisioning, complete multi-host
scheduler lifecycle, and complete project-template upgrade and rollback remain
Expand All @@ -85,7 +88,7 @@ isolated Ubuntu 26.04.1 / Docker 29.8 hosts, the source-equivalent candidate
passed bidirectional workload ping and TCP, service DNS, egress, a published
host port, Docker restart, and both host reboots. The peer was explicitly
tested with native nftables, iptables-nft, and iptables-legacy, then restored
to its original native-nft configuration. This does not qualify every
to its original iptables-legacy configuration. This does not qualify every
existing deployment's upgrade or rollback path. The
alternative drivers are not installed automatically by the project template.

Expand Down
20 changes: 20 additions & 0 deletions catalog-images.json
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,26 @@
"critical": 0
}
},
{
"reference": "ghcr.io/pasturestack/network-plugin-manager:v0.8.19",
"manifestDigest": "sha256:c5b827c6cfe32a19b7ec0fe7377c81fdbe4ef225af1c2dc13bc74c016e970e67",
"sourceRepository": "https://github.com/PastureStack/network-plugin-manager",
"sourceCommit": "1cfebba285c74130690cb02121b2f4d9d34fd019",
"sourcePath": "package/Dockerfile",
"registryPage": "https://github.com/orgs/PastureStack/packages/container/package/network-plugin-manager",
"licenseBoundary": "Apache-2.0 source and image; bundled Alpine, Docker CLI, and other packages retain their upstream licenses and notices",
"reviewedAt": "2026-09-14",
"platforms": [
"linux/amd64"
],
"vulnerabilityScan": {
"scanner": "Trivy 0.74.0",
"reportCreatedAt": "2026-09-14",
"scope": "published runtime image",
"high": 0,
"critical": 0
}
},
{
"reference": "ghcr.io/pasturestack/network-diagnostics-agent:v0.2.0",
"sourceRepository": "https://github.com/PastureStack/network-diagnostics-agent",
Expand Down
39 changes: 39 additions & 0 deletions infra-templates/network-services/7/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<!-- SPDX-License-Identifier: MIT -->

# PastureStack Network Services

Version 7 uses Network Plugin Manager `v0.8.19`. It retains single-backend selection and unchanged Metadata Service and Internal DNS images. It also restores bidirectional VXLAN forwarding when published host ports coexist with the overlay. The manager rejects malformed or conflicting bridge metadata before touching host firewall rules, binds every managed forwarding rule to that exact bridge, and protects bridge traffic from `route_localnet` loopback routing. It records the original per-bridge setting under `/run`, applies the guard before enabling it, and restores the original value when that bridge no longer needs a host port. The optional per-host-subnet network preserves container source IPs across active peers and permits only peer-to-local-subnet forwarding. The image's release provenance and digest are recorded in `catalog-images.json`.

## Firewall backend

`FIREWALL_BACKEND` defaults to `auto`. It reads Docker's actual firewall driver: Docker's native `nftables` driver uses native nft rules, while Docker's `iptables` driver selects the frontend that owns Docker's active NAT chain. That may be `iptables-nft` or `iptables-legacy` on **any supported host**, including Ubuntu 26.04 and later. The OS release, installed executable, or unloaded kernel module alone never selects a backend. To pin one path, choose:

- `nftables`: Docker's native nftables firewall backend. This is **not** the same as the iptables-nft compatibility CLI.
- `iptables-nft`: xtables compatibility CLI backed by nf_tables, for Docker's iptables firewall driver.
- `iptables-legacy`: legacy xtables, only when the running Docker daemon actually owns the active rules through that frontend.

The manager refuses a mismatched or ambiguous selection and does not fall back, switch Docker's backend, or load legacy modules. An Ubuntu 26.04+ host already using `iptables-legacy` or `iptables-nft` must keep its live Docker path; do not turn on native nftables merely because the OS is new. A deliberate migration requires a separate host change, rollback point, and network lifecycle test.

This manager alone owns the host NAT and host-port `CATTLE_*` chains. Its
masquerade rules exclude destinations inside the managed overlay subnet in
all three backends; the IPsec host-XFRM router must not patch these chains.
For the per-host-subnet driver, the manager also excludes other active hosts'
validated subnets from masquerade and adds a bounded forwarding exception.
Inactive registrations are ignored; missing or overlapping labels on an active
host fail closed. This is a routed, unencrypted network; protect the host
transport separately.
Upgrade Network Services first and verify manager health on every host before
upgrading the matching IPsec Overlay version.

For Docker's native nftables driver, configure Docker itself with `"firewall-backend": "nftables"` and `"bridge-accept-fwmark": "0x1068/0x1068"` before upgrading this stack. Persist `net.ipv4.ip_forward=1` on the host and verify it remains enabled after a reboot: Docker's native nftables backend does not enable IPv4 forwarding for you. The mark allows Docker's bridge forwarding rules to accept the manager's published-host-port traffic; the template cannot configure the host daemon or kernel settings. Check and explicitly migrate any stale `iptables-nft` `FORWARD DROP` policy or previous platform hooks before switching Docker. The manager refuses that mixed state rather than changing the host's global firewall policy. Docker's native nftables backend remains an experimental Docker feature; qualify it against the installed Docker release before production use.

## Other configuration

- `DOCKER_BRIDGE`: host bridge for managed workload traffic.
- `DNS_RECURSER_TIMEOUT`, `TTL`: upstream DNS timeout and service-discovery cache time.
- `CPU_PERIOD`, `CPU_QUOTA`: Metadata Service CPU scheduling limits.
- `RELOAD_INTERVAL_LIMIT`, `ARP_SYNC_INTERVAL`: metadata reload and host ARP reconciliation intervals.

Network Plugin Manager still requires host networking, host PID visibility, the Docker socket, Docker state, kernel-module and runtime mounts, and the shared CNI volume. Metadata Service starts as root only to assign its link-local address, then drops to UID/GID 10001. Internal DNS shares its namespace. The `rancher-compose.yml` filename, `io.rancher.*` labels, `CATTLE_*` fallback variables, `/var/lib/rancher` CA path, and `rancher-cni-driver` volume are compatibility contracts, not a request to use legacy firewall rules.

These template files are MIT-licensed. The manager, metadata service, and internal DNS retain their Apache-2.0 licenses and bundled dependency notices. Verify image source and the recorded manifest digest in `catalog-images.json` before deployment.
30 changes: 30 additions & 0 deletions infra-templates/network-services/7/README.zh-TW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
<!-- SPDX-License-Identifier: MIT -->

# PastureStack 網路服務

第 7 版使用網路外掛管理器 `v0.8.19`,保留單一防火牆後端的選擇方式,以及相同的中繼資料服務與內部 DNS 映像;並修正發布主機連接埠與 VXLAN 並存時的雙向轉送。管理器會在修改主機防火牆規則前拒絕格式錯誤或互相衝突的網橋中繼資料,並把所有受管轉送規則限制在正確網橋。啟用 `route_localnet` 前會先阻擋來自該網橋、目的為 `127.0.0.0/8` 的流量;原始的每網橋設定會保存於 `/run`,不再需要主機連接埠時則恢復原值。選用每主機子網路時,會保留跨主機容器來源位址,並只允許已驗證的對端子網路轉送至本機子網路。映像發布來源與 digest 記錄於 `catalog-images.json`。

## 防火牆後端

`FIREWALL_BACKEND` 預設為 `auto`,依 Docker 實際防火牆驅動程式選擇單一路徑:Docker 原生 `nftables` 使用原生 nft 規則;Docker `iptables` 驅動程式則辨識哪一套前端擁有 Docker 現役 NAT 鏈。任何受支援主機(包括 Ubuntu 26.04 及更新版)都可能使用 `iptables-nft` 或 `iptables-legacy`;作業系統版本、執行檔存在或尚未載入的核心模組,均不足以決定後端。需要固定路徑時可選:

- `nftables`:Docker 原生 nftables 防火牆後端,**不是** iptables-nft 相容命令。
- `iptables-nft`:由 nf_tables 支援的 xtables 相容命令,搭配 Docker 的 iptables 防火牆驅動程式。
- `iptables-legacy`:只在現役 Docker 確實透過這套前端持有規則時選用。

選擇與 Docker 實際後端不符或無法判定時,管理器會拒絕啟動,不會自動降級、切換 Docker 後端或載入 legacy 模組。Ubuntu 26.04 及更新版若已使用 `iptables-legacy` 或 `iptables-nft`,就應維持現役 Docker 路徑;不能只因系統較新便替它切成原生 nftables。刻意遷移須另外準備主機變更、回復點及網路生命週期驗收。

主機 NAT 與主機連接埠的 `CATTLE_*` 規則鏈只由此管理器維護;三種後端的來源位址轉換規則都排除受管 overlay 子網路內的目的位址。每主機子網路還會排除其他有效主機的已驗證子網路,並加入限定來源與目的子網路的轉送例外;非現役主機不列入,現役主機若缺少標籤或子網路重疊則安全地拒絕套用。此網路只提供路由、不加密,須另行保護主機間傳輸。IPsec 主機 XFRM 路由器不得再插入補丁規則。升級時應先升級網路服務,逐台確認管理器健康,再升級相符的 IPsec 加密網路版本。

使用 Docker 原生 nftables 前,必須先在主機 Docker 設定加入 `"firewall-backend": "nftables"` 及 `"bridge-accept-fwmark": "0x1068/0x1068"`,再升級此堆疊。還須在主機持久設定 `net.ipv4.ip_forward=1`,並於重開機後確認仍啟用;Docker 原生 nftables 後端不會代為啟用 IPv4 轉送。此標記讓 Docker 網橋轉送規則接受管理器發布的主機連接埠流量;範本無法替主機設定 Docker daemon 或核心參數。切換前還須檢查並明確遷移殘留的 `iptables-nft FORWARD DROP` 全域政策與舊平台掛鉤。管理器遇到混用狀態會拒絕啟動,不會自行修改主機全域防火牆政策。Docker 原生 nftables 目前仍屬實驗性功能,正式環境使用前應針對安裝的 Docker 版本完成驗收。

## 其他設定

- `DOCKER_BRIDGE`:受管工作負載使用的主機網橋。
- `DNS_RECURSER_TIMEOUT`、`TTL`:上游 DNS 逾時與服務探索快取時間。
- `CPU_PERIOD`、`CPU_QUOTA`:中繼資料服務的 CPU 排程限制。
- `RELOAD_INTERVAL_LIMIT`、`ARP_SYNC_INTERVAL`:中繼資料重新載入與主機 ARP 協調間隔。

網路外掛管理器仍需主機網路、主機 PID、Docker Socket、Docker 狀態、核心模組與執行環境掛載,以及共用 CNI 磁碟區。中繼資料服務僅在指派連結本機位址時以 root 啟動,之後切換為 UID/GID 10001;內部 DNS 與其共用網路命名空間。`rancher-compose.yml`、`io.rancher.*`、`CATTLE_*` 備援變數、`/var/lib/rancher` CA 路徑及 `rancher-cni-driver` 磁碟區是既有協定的相容契約,不代表必須使用 legacy 防火牆規則。

範本檔案採 MIT 授權;管理器、中繼資料服務與內部 DNS 保留 Apache-2.0 授權及隨附相依套件聲明。部署前應以 `catalog-images.json` 核對映像來源與記錄的 manifest digest。
95 changes: 95 additions & 0 deletions infra-templates/network-services/7/docker-compose.yml.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# SPDX-License-Identifier: MIT
version: '2'

services:
network-plugin-manager:
image: ghcr.io/pasturestack/network-plugin-manager:v0.8.19
privileged: true
network_mode: host
pid: host
command:
- network-plugin-manager
- --metadata-url
- http://169.254.169.250/2016-07-29
- --arpsync-interval
- '${ARP_SYNC_INTERVAL}'
- --firewall-backend
- '${FIREWALL_BACKEND}'
environment:
DOCKER_BRIDGE: '${DOCKER_BRIDGE}'
METADATA_IP: 169.254.169.250
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker:/var/lib/docker
- /lib/modules:/lib/modules:ro
- /run:/run
- /var/run:/var/run
- rancher-cni-driver:/etc/cni
- rancher-cni-driver:/opt/cni
labels:
io.pasturestack.component: network-plugin-manager
io.rancher.scheduler.global: 'true'
logging:
driver: json-file
options:
max-size: 25m
max-file: '2'

metadata:
image: ghcr.io/pasturestack/metadata-service:v0.9.11
user: root
cap_add:
- NET_ADMIN
network_mode: bridge
command:
- /bin/bash
- -ec
- |
export PLATFORM_URL="$${PLATFORM_URL:-$${CATTLE_URL:-}}"
export PLATFORM_ACCESS_KEY="$${PLATFORM_ACCESS_KEY:-$${CATTLE_ACCESS_KEY:-}}"
export PLATFORM_SECRET_KEY="$${PLATFORM_SECRET_KEY:-$${CATTLE_SECRET_KEY:-}}"
exec metadata-service --reload-interval-limit="${RELOAD_INTERVAL_LIMIT}" --subscribe
environment:
PLATFORM_CA_ROOT: /var/lib/rancher/etc/ssl/ca.crt
labels:
io.pasturestack.component: metadata-service
io.rancher.sidekicks: dns
io.rancher.container.create_agent: 'true'
io.rancher.scheduler.global: 'true'
io.rancher.container.agent_service.metadata: 'true'
logging:
driver: json-file
options:
max-size: 25m
max-file: '2'
sysctls:
net.ipv4.conf.all.send_redirects: '0'
net.ipv4.conf.default.send_redirects: '0'
cpu_period: ${CPU_PERIOD}
cpu_quota: ${CPU_QUOTA}

dns:
image: ghcr.io/pasturestack/internal-dns:v0.17.11
network_mode: container:metadata
command:
- internal-dns
- --listen
- 169.254.169.250:53
- --recurser-timeout
- '${DNS_RECURSER_TIMEOUT}'
- --ttl
- '${TTL}'
environment:
PLATFORM_METADATA_ENABLED: 'true'
PLATFORM_METADATA_URL: http://localhost/2016-07-29
PLATFORM_METADATA_ANSWER: 169.254.169.250
NEVER_RECURSE_TO: 169.254.169.250
PLATFORM_DNS_ANSWERS_FILE: /etc/internal-dns/answers.json
labels:
io.pasturestack.component: internal-dns
io.rancher.scheduler.global: 'true'
logging:
driver: json-file
options:
max-size: 25m
max-file: '2'
77 changes: 77 additions & 0 deletions infra-templates/network-services/7/rancher-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# SPDX-License-Identifier: MIT
.catalog:
name: PastureStack Network Services
version: v0.3.5
description: Install host networking, metadata, and internal DNS services required by managed workloads.
minimum_rancher_version: v1.6.26-rc1
labels:
io.pasturestack.catalog.question.docker_bridge.label.zh-tw: 'Docker 網橋'
io.pasturestack.catalog.question.docker_bridge.description.zh-tw: '受管工作負載流量使用的主機網橋。'
io.pasturestack.catalog.question.firewall_backend.label.zh-tw: '主機防火牆後端'
io.pasturestack.catalog.question.firewall_backend.description.zh-tw: '依 Docker 現役後端自動選擇;不論 Ubuntu 版本,原生 nftables、iptables-nft 與 iptables-legacy 互不混用。切換原生 nftables 前須先設定 Docker bridge-accept-fwmark。'
io.pasturestack.catalog.question.dns_recurser_timeout.label.zh-tw: 'DNS 遞迴查詢逾時'
io.pasturestack.catalog.question.dns_recurser_timeout.description.zh-tw: '等待上游 DNS 查詢回應的秒數。'
io.pasturestack.catalog.question.ttl.label.zh-tw: '服務探索 DNS 紀錄存留時間'
io.pasturestack.catalog.question.ttl.description.zh-tw: '內部服務探索 DNS 回應可保留的秒數。'
io.pasturestack.catalog.question.cpu_period.label.zh-tw: '中繼資料服務 CPU 週期'
io.pasturestack.catalog.question.cpu_period.description.zh-tw: '分配給每個中繼資料服務執行個體的 CPU 排程週期。'
io.pasturestack.catalog.question.cpu_quota.label.zh-tw: '中繼資料服務 CPU 配額'
io.pasturestack.catalog.question.cpu_quota.description.zh-tw: '分配給每個中繼資料服務執行個體的 CPU 配額。'
io.pasturestack.catalog.question.reload_interval_limit.label.zh-tw: '中繼資料重新載入間隔'
io.pasturestack.catalog.question.reload_interval_limit.description.zh-tw: '兩次中繼資料設定重新載入之間的最短毫秒數。'
io.pasturestack.catalog.question.arp_sync_interval.label.zh-tw: 'ARP 同步間隔'
io.pasturestack.catalog.question.arp_sync_interval.description.zh-tw: '兩次主機 ARP 協調作業之間的秒數。'
questions:
- variable: DOCKER_BRIDGE
label: Docker bridge
description: Host bridge used for managed workload traffic.
type: string
default: docker0
required: true
- variable: FIREWALL_BACKEND
label: Host firewall backend
description: Follow Docker's active firewall backend on any supported Ubuntu version, or explicitly select native nftables, iptables-nft, or iptables-legacy. A mismatch fails safely; native nftables requires Docker bridge-accept-fwmark on the host.
type: enum
default: auto
required: true
options:
- auto
- nftables
- iptables-nft
- iptables-legacy
- variable: DNS_RECURSER_TIMEOUT
label: DNS recursion timeout
description: Seconds allowed for an upstream DNS query.
type: int
default: 2
required: true
- variable: TTL
label: Service discovery TTL
description: Seconds that internal service-discovery answers remain valid.
type: int
default: 1
required: true
- variable: CPU_PERIOD
label: Metadata CPU period
description: CPU scheduler period assigned to each metadata service instance.
type: int
default: 400000
required: true
- variable: CPU_QUOTA
label: Metadata CPU quota
description: CPU quota assigned to each metadata service instance.
type: int
default: 200000
required: true
- variable: RELOAD_INTERVAL_LIMIT
label: Metadata reload interval
description: Minimum milliseconds between metadata configuration reloads.
type: int
default: 1000
required: true
- variable: ARP_SYNC_INTERVAL
label: ARP synchronization interval
description: Seconds between host ARP reconciliation passes.
type: int
default: 5
required: true
2 changes: 1 addition & 1 deletion infra-templates/network-services/config.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: MIT
name: Network Services
description: Install host networking, metadata, and internal DNS services required by managed workloads.
version: v0.3.4
version: v0.3.5
category: Networking
maintainer: PastureStack contributors
license: MIT template; Apache-2.0 images and third-party package licenses apply
Expand Down
Loading