Static-mode IPv6 for Cuttlefish guests: ULA + NAT66 by default, opt-in routed /48, e2e tests by sferrogoo · Pull Request #3104 · google/android-cuttlefish

sferrogoo · 2026-08-27T20:38:04Z

Summary

Gives Cuttlefish guests IPv6 on Ethernet (eth1), Wi-Fi (wlan0, through OpenWrt) and mobile data (buried_eth0, through modem_simulator), set up by the host at boot like the existing IPv4 configuration. IPv4 behavior is unchanged.

Two modes, chosen in /etc/default/cuttlefish-host-resources:

Mode	When	Addresses	NAT
Private (default)	Any host. No upstream IPv6 needed to configure; upstream IPv6 needed to reach the internet.	ULA `fd00:cf:20::/44` (`fd00:cf:24::/64` Ethernet bridge, `fd00:cf:22::/64` Wi-Fi bridge, `fd00:cf:21:<i>::/64` mobile, `fd00:cf:23:<i>::/64` OpenWrt WAN, `fd00:cf:25::/64` OpenWrt LAN)	NAT66 (nftables masquerade)
Routed (opt-in, `ipv6_routed_prefix=P::/48`)	Host owns a routed /48 (lab router, ISP prefix delegation).	One global /64 per network from `P::/48` (`P:21NN`, `P:22`, `P:23NN`, `P:24`, `P:25NN`)	None

Routed mode exists because CTS ConnectivityManagerTest#testOpenConnection requires the address an external server sees to be on the network's link and to differ between Wi-Fi and cellular, which NAT cannot satisfy.

Commits

cuttlefish-host-resources: ULA prefixes, router advertisements (dnsmasq, separate from the IPv4 DHCP instances), RA guard on the bridges (including 802.1Q VID-0 frames), NAT66, clean stop, podcvd --sysctl for forwarding. Honors the new allocate_static_resources switch.
assemble_cvd, modem_simulator: the mobile IPv6 address is derived from the host cvd-mtap-NN address; +CGCONTRDP returns one line per address family (3GPP TS 27.007 §10.1.23).
Docs for the OpenWrt side.
e2e tests: static_resources_init_test (rootless sandbox, no KVM) and networking_tests (TestIPv6Provisioning, TestIPv6Nat66Egress).
Routed mode, openwrt_args.cpp kernel command line keys (wan_ip6addr, wan_ip6gw, lan_ip6prefix), TestIPv6RoutedEcho, a Docker upstream simulator (e2etests/ipv6_upstream_sim/) to exercise routed mode without a routed prefix, and fixes from review and testing.

Companion AOSP changes (same bug)

external/openwrt-prebuilts: Wi-Fi LAN ULA + masq6 by default; static WAN IPv6 and no NAT66 when lan_ip6prefix is on the kernel command line.
device/generic/goldfish radio HAL: read all +CGCONTRDP lines (dual-stack data calls).
packages/modules/Connectivity CTS: replace the Build.MODEL.contains("Cuttlefish") skips in DnsTest and testOpenConnection with capability checks.

Without the companion changes, this PR alone gives IPv6 on eth1 and keeps IPv4 everywhere; wlan0 and buried_eth0 IPv6 need the OpenWrt and radio HAL changes in the guest image.

Testing

bazel test in base/cvd for host/libs/config, assemble_cvd, modem_simulator (with format and tidy): 341/341 pass. Full //...: 1718/1720, the 2 failures are pre-existing environment issues (missing libclang for an unrelated camera test).
e2etests: //host_resources/... pass (15 top-level tests, including routed mode, invalid prefix fallback, start-twice, mode switch); //cvd/networking_tests/... builds; gofmt/vet/staticcheck clean; shellcheck findings on the init script 53 → 44.
Real host (Debian, packages built from this branch), Android 17 aosp_cf_x86_64_only_phone image with the companion changes:
- Private mode: all three guest interfaces get IPv6, ping6 to 2001:4860:4860::8888 on each, NAT66 counters increase; TestIPv6Provisioning, TestIPv6Nat66Egress, TestDeviceNetworking pass; bridged Wi-Fi mode pass.
- Official CTS (CtsNetTestCases, [instant], CtsTetheringTest, CtsNetSecConfig*, CtsVcnTestCases), unmodified APKs: 1,764 pass / 0 fail, per-module counts identical to the no-IPv6 baseline. Note: official CTS skips its IPv6 tests on Cuttlefish by model name; the Connectivity companion change removes those skips.
- With the companion CTS change, private mode: DnsTest passes with Wi-Fi as default network and with mobile as default network; testOpenConnection is skipped by the capability check.
- Routed mode (2001:db8:cf00::/48, isolated network namespace, Docker upstream simulator): no NAT66 rules; buried_eth0 gets 2001:db8:cf00:2101::2, wlan0 gets a 2001:db8:cf00:2501::/64 SLAAC address; TestIPv6RoutedEcho and TestDeviceNetworking pass; with the simulator's DNS and lab CA (lab setup, not official CTS) testOpenConnection and DnsTest pass on both networks.
- start twice without stop: no duplicate RA daemons or rules; stop leaves no IPv6 state.

Bug: b/549899406

sferrogoo · 2026-08-31T22:50:48Z

@google/android-cuttlefish @adelva1984 @jemoreira PTAL

- Remove duplicate nftables dependency from debian/control. - Roll IPv6 integration tests into existing networking_tests suite. - Factorize IPv6 address polling and connectivity assertions with timeouts. - Separate dual-stack connectivity from IPv6-only flush tests. - Add test host IPv6 bridge check to skip gracefully on IPv4-only hosts.

sferrogoo · 2026-09-10T20:45:41Z

Pushed commit 457a75c addressing all review comments:

Removed duplicate nftables dependency in debian/control.
Moved tests to e2etests/cvd/networking_tests/ipv6_test.go and integrated with Bazel.
Factorized polling and ping logic into reusable helpers with explicit timeouts.
Split into two tests: TestIPv6DualStackConnectivity and TestIPv6OnlyMode.
Added host bridge check to skip gracefully on IPv4-only test hosts.

sferrogoo · 2026-09-10T20:50:09Z

@dxapd @3405691582 PTAL, addressed all feedback in commit 457a75c.

…and NAT66 Replace opt-in bridge IPv6 provisioning in cuttlefish-host-resources with default-on Unique Local Address (ULA, RFC 4193) static-mode provisioning: - Configure default ULA prefixes under fd00:cf:20::/44: - cvd-mtap-XX: fd00:cf:21:<i in hex>::1/64 (routed; mobile guest IPv6 via RIL, no RA dnsmasq) - cvd-wbr: fd00:cf:22::1/64 (bridged; RA-only dnsmasq) - cvd-wifiap-XX: fd00:cf:23:<i in hex>::1/64 (routed OpenWrt WAN; RA-only dnsmasq) - cvd-ebr: fd00:cf:24::1/64 (bridged; RA-only dnsmasq) - Keep IPv4 setup and start_dnsmasq/stop_dnsmasq untouched; run start_ipv6 after all IPv4 setup when net.ipv6.conf.all.disable_ipv6=0 and track state via /run/cuttlefish/ipv6-enabled for symmetric teardown. - Run separate RA-only dnsmasq instances (cuttlefish-dnsmasq-ra-<iface>.pid) so IPv6/ICMPv6 issues cannot affect DHCPv4. - Set accept_ra=0 and autoconf=0 on all host cvd-* bridges and tap devices, guarding /proc/sys writes with [ -w ... ] and passing --sysctl net.ipv6.conf.all.forwarding=1 in podcvd for read-only /proc/sys rootless containers. - Add separate IPv6 nftables tables: - ip6 cuttlefish_nat6 postrouting masquerade for fd00:cf:20::/44 leaving non-cvd-* interfaces - bridge cuttlefish_ra_guard prerouting (priority -300) matching meta protocol ip6 meta l4proto ipv6-icmp (including 802.1Q VLAN-0 tagged frames) and dropping nd-router-advert and nd-redirect on cvd-etap-* and cvd-wtap-* - inet cuttlefish_ra_guard input (priority -300) dropping nd-router-advert and nd-redirect on cvd-mtap-* and cvd-wifiap-* Bug: b/549899406 Test: shellcheck -s sh base/debian/cuttlefish-base.cuttlefish-host-resources.init Test: bazel test //host_resources/static_resources_init_test:static_resources_init_test

Derive static-mode cellular IPv6 parameters (ril_ipv6_ipaddr, ril_ipv6_gateway, ril_ipv6_prefixlen) from the host mobile interface (cvd-mbr or cvd-mtap-XX) global/ULA IPv6 address in assemble_cvd via MobileIpv6ConfigFromHostAddress(), set ril_ipv6_dns to 2001:4860:4860::8888, and advertise dual-stack IPv4 + IPv6 PDP context parameters in modem_simulator DataService (AT+CGDCONT and AT+CGCONTRDP) when ril_ipv6_ipaddr is configured while falling back to IPv4-only when no host IPv6 address is present. Bug: b/549899406 Test: bazel test //cuttlefish/host/commands/assemble_cvd/... //cuttlefish/host/commands/modem_simulator/... //cuttlefish/host/libs/config/...

Add docs/networking/openwrt_ipv6_static.md describing the declarative OpenWrt UCI configuration in AOSP platform/external/openwrt-prebuilts (shared/config/{network,dhcp,firewall}) for static networking mode: - wan6 obtains a SLAAC address and default route (sourcefilter 0) from the host RA on cvd-wifiap-XX (fd00:cf:23:<hex-instance>::/64) or cvd-wbr (fd00:cf:22::/64). - globals.ula_prefix is set to fd00:cf:25::/48 with /64 assignments and odhcpd RA (ra_slaac 1, ra_default 1, dhcpv6 disabled) on wifi0/wifi1. - br-wifi0 and br-wifi1 use bridge_empty=1 with fixed MAC addresses instead of enslaving eth0.0/eth0.1 VLAN sub-interfaces. - firewall wan zone enables masq6=1 for NAT66 alongside IPv4 masq=1. Bug: b/549899406 Test: none (documentation only)

…ests - Add e2etests/cvd/networking_tests/ipv6_test.go and wire it into //cvd/networking_tests:networking_tests to validate the static-mode ULA prefix plan (fd00:cf:21:<i>::/64, fd00:cf:22::/64, fd00:cf:23:<i>::/64, fd00:cf:24::/64, fd00:cf:25::/48), absence of 2001:db8::/32 addresses, buried_eth0 IPv4 preservation, and optional RIL/OpenWrt/NAT66 egress checks behind --image_ril_ipv6, --image_openwrt_ipv6, and --image_manages_eth1 flags. - Update e2etests/host_resources/static_resources_init_test/main_test.go expectations for cuttlefish_nat6, bridge/inet cuttlefish_ra_guard, /run/cuttlefish/ipv6-enabled, and RA dnsmasq pidfiles. - Use json.RawMessage for nftMatch.Right in e2etests/host_resources/common/nft.go so rules with string right-hand sides (iifname/oifname) unmarshal cleanly. Bug: b/549899406 Test: bazel test //host_resources/static_resources_init_test:static_resources_init_test Test: bazel build //cvd/networking_tests:networking_tests && staticcheck ./...

Private mode (ULA fd00:cf:20::/44 + NAT66) stays the default so guests get IPv6 on any host. This adds an opt-in routed mode for hosts that own a routed /48, so that each guest network gets its own global /64 with no NAT66. That is what CTS ConnectivityManagerTest#testOpenConnection checks: the address an external server sees must be on the network's link and differ between Wi-Fi and cellular. Host init script and defaults: - New ipv6_routed_prefix=P::/48. Layout: cvd-mtap-NN P:21NN::/64, cvd-wbr P:22::/64, cvd-wifiap-NN P:23NN::/64, cvd-ebr P:24::/64, OpenWrt Wi-Fi LAN P:25NN::/64 routed via P:23NN::2. Anything that is not a /48 is rejected with a message and private mode is used. - New ipv6_nat=0|1 (default 0 in routed mode, 1 in private mode). Routes added in routed mode are recorded in /run/cuttlefish/ipv6-routes so stop removes them. - start_ipv6 now runs stop_ipv6 first, so a reinstall over a version that did not stop IPv6 no longer leaves duplicate RA dnsmasq processes. - Prefix settings are validated; the reason for the fixed fd00:cf prefix is documented in the defaults file. openwrt_args.cpp: when cvd-wifiap-NN carries P:23NN::1/64 (routed mode), pass wan_ip6addr, wan_ip6gw and lan_ip6prefix on the OpenWrt kernel command line. OpenWrt reads them in external/openwrt-prebuilts 0_default_config. No new launcher flag is needed. Tests and docs: - openwrt_args_test (new), routed-mode cases in network_flags_test and modem_simulator service_ipv6_test. - static_resources_init_test: routed-mode addressing, no NAT66, LAN routes, invalid prefix fallback, ipv6_nat override, start-twice, mode switch. - networking_tests: TestIPv6RoutedEcho (runs with --routed_echo_url), helper unit tests, bridged Wi-Fi prefix expectation fixed. - e2etests/ipv6_upstream_sim: Docker image that plays the upstream router, echo server and DNS for a routed /48, with a self-test, so routed mode can be exercised on a machine without a routed prefix. - docs/networking/ipv6_routed_mode.md; internal references removed from network_flags.cpp and the docs. Bug: 549899406 Test: bazel test //cuttlefish/host/libs/config/... //cuttlefish/host/commands/assemble_cvd/... //cuttlefish/host/commands/modem_simulator/... Test: e2etests: bazel test //host_resources/... ; bazel build //cvd/networking_tests/... Test: live: private mode, official CtsNetTestCases 1764 pass / 0 fail; routed mode with the simulator, TestIPv6RoutedEcho pass

…, and harden IPv6 lifecycle - Replace early-boot net.ipv6.conf.all.forwarding=1 in 90-cuttlefish-ip-forward.conf with net.ipv6.conf.default.accept_ra=2. In cuttlefish-host-resources.init enable_ipv6_forwarding(), promote accept_ra from 1 to 2 on default, non-cvd-* interfaces, and any interface with an RTF_ADDRCONF (0x40000) default route in /proc/net/ipv6_route before setting net.ipv6.conf.all.forwarding=1, so addrconf_fixup_forwarding() does not purge the host's SLAAC default route. - Add ip6fwd support to manage_nft_rule and insert handle-tracked iifname/oifname "cvd-*" counter accept rules into table ip6 filter chain FORWARD so nf_hook_slow accepts Cuttlefish IPv6 forwarding even when Docker sets ip6tables -P FORWARD DROP. - Remove '|| true' error suppression across the init script and ipv6_upstream_sim scripts. Check whether ip_forward / all.forwarding is already '1' before writing (preserving read-only /proc/sys in podcvd), gate forwarding sysctl setup on allocate_static_resources=1 after /.dockerenv device setup, log an explicit error to stderr on write failure, and abort start_ipv6 when IPv6 forwarding cannot be enabled. - Harden idempotency, IPv4-only hosts, multi-instance RA cleanup, and Debian packaging: * Delete existing handle before re-adding in manage_nft_rule. * Flush scope global IPv6 addresses on cvd-* bridges and taps and stop all /var/run/cuttlefish-dnsmasq-ra-*.pid daemons in stop_ipv6(). * Explicitly set disable_ipv6=0 on cvd-mtap-* and cvd-wifiap-* taps in start_ipv6() for hosts with net.ipv6.conf.default.disable_ipv6=1. * Clean up cuttlefish-dnsmasq*, routed IPv6 prefixes, nftables tables/handles, and /run/cuttlefish on purge in cuttlefish-base.postrm without TOCTOU signal races or unguarded jq. * Scope second dh_installsystemd call in base/debian/rules with --remaining-packages to prevent duplicate postinst/prerm snippets. Bug: 549899406

…ources - Add ParseDns6ServersFromDefaults() and ObtainMobileIpv6Dns() to network_flags.cpp so cellular RIL IPv6 DNS reads dns6_servers from /etc/default/cuttlefish-host-resources when --ril_dns does not already supply an IPv6 server, logging a warning for invalid tokens and falling back to 2001:4860:4860::8888,2001:4860:4860::8844 when unset or unparseable. - Add unit tests in network_flags_test.cc covering custom dns6_servers, commented/unset defaults, whitespace/quotes, mixed IPv4/IPv6 lists, and empty values. Bug: 549899406

…cker FORWARD, and purge - Add TestStaticIPv6PreservesUpstreamRA verifying that starting cuttlefish-host-resources with an active RTF_ADDRCONF default route promotes accept_ra to 2 and preserves the host's IPv6 default route. - Add TestStaticIPv6ForwardAcceptWithDockerDrop verifying that cuttlefish-host-resources adds iifname/oifname "cvd-*" counter accept rules in table ip6 filter chain FORWARD (both before and after Docker sets ip6tables -P FORWARD DROP), forwards IPv6 packets in both directions for cvd-* interfaces while dropping non-cvd-* traffic, and cleanly removes the rules on stop. - Add TestStaticIPv6ForwardingReadOnly verifying read-only IPv6 and IPv4 forwarding sysctl handling (including allocate_static_resources=0) and 90-cuttlefish-ip-forward.conf behavior. - Add TestStaticIPv6DefaultDisableIPv6 verifying tap/bridge IPv6 address and route assignment when net.ipv6.conf.default.disable_ipv6=1. - Add TestStaticIPv6ShrinkAccountsStopsOrphanRaDnsmasq verifying stop_ipv6 terminates all RA dnsmasq instances when num_cvd_accounts is reduced across start/stop. - Add TestStaticIPv6PostrmPurgeCleanup verifying cuttlefish-base.postrm purge removes routed IPv6 prefixes, kills cuttlefish-dnsmasq*, removes nftables rules/tables, and removes /run/cuttlefish. Bug: 549899406

Wire //cvd/networking_tests:networking_tests (hermetic IPv6 and routed echo helper suites) into the e2e-tests-orchestration-runner-special job in .github/workflows/presubmit.yaml. Note that static_resources_init_test is already executed by run-cvd-unit-tests under //host_tests/static_resources_init_test. Bug: 549899406

…s IPv6 egress On a host with kernel IPv6 enabled (disable_ipv6=0) but no IPv6 default route (typical IPv4-only corporate or home network), advertising default external IPv6 DNS servers (2001:4860:4860::8888/8844) in dnsmasq RAs on cvd-wifiap-XX/cvd-ebr/cvd-wbr and in RIL +CGCONTRDP causes OpenWrt dnsmasq and guest resolvers to send queries that blackhole at the host (failing CTS MultinetworkApiTest#testResNApi with -ETIMEDOUT). Detect host IPv6 egress at start_ipv6 time (ip -6 route show default non-empty, routed mode, or an explicit dns6_servers / ipv6_egress=1 override in /etc/default/cuttlefish-host-resources): - When the host has no IPv6 egress, log a message, omit option6:dns-server from start_ra_dnsmasq while keeping ULA addressing and RA prefix/route advertisements (so DnsTest#testDnsWorks continues to pass), and leave /run/cuttlefish/ipv6-egress absent. - When the host has IPv6 egress, write /run/cuttlefish/ipv6-egress and remove it in stop_ipv6. - In assemble_cvd (ResolveMobileIpv6Dns), omit ril_ipv6_dns when /run/cuttlefish/ipv6-egress is absent and dns6_servers is not explicitly configured in /etc/default/cuttlefish-host-resources. - Add hermetic coverage in static_resources_init_test (TestStaticIPv6EgressDetectionControlsDnsServerAndMarker) and network_flags_test (ResolveMobileIpv6Dns). Bug: 549899406

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from eee6c24 to c758369 Compare August 28, 2026 15:43

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from cd09095 to 4f5ccc3 Compare September 1, 2026 00:06

rmuthiah requested a review from dxapd September 1, 2026 04:03

0405ysj added the kokoro:force-run Trigger a presubmit build unconditionally. label Sep 1, 2026

GoogleCuttlefishTesterBot removed the kokoro:force-run Trigger a presubmit build unconditionally. label Sep 1, 2026

3405691582 requested changes Sep 3, 2026

View reviewed changes

dxapd reviewed Sep 3, 2026

View reviewed changes

Comment thread e2etests/cvd/network_tests/main_test.go Outdated

dxapd requested changes Sep 4, 2026

View reviewed changes

Comment thread base/debian/control Outdated

sferrogoo requested review from 3405691582 and dxapd September 10, 2026 20:49

3405691582 added the kokoro:run Run e2e tests. label Sep 16, 2026

GoogleCuttlefishTesterBot removed the kokoro:run Run e2e tests. label Sep 16, 2026

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from 1416eae to 8005a2d Compare September 22, 2026 16:16

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from 8005a2d to bd4af35 Compare October 6, 2026 16:37

sferrogoo changed the title ~~Enable IPv6 ULA, SLAAC Router Advertisements, NAT66, and E2E validation~~ Static-mode IPv6 for Cuttlefish guests: ULA + NAT66 by default, opt-in routed /48, e2e tests Oct 6, 2026

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from bd4af35 to dca0e84 Compare October 6, 2026 16:59

sferrogoo added 6 commits October 9, 2026 16:07

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from 73268a3 to b700835 Compare October 9, 2026 20:35

sferrogoo added 3 commits October 9, 2026 17:52

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from b700835 to 47ff17d Compare October 9, 2026 22:04

sferrogoo force-pushed the cuttlefish-ipv6-enablement branch from 47ff17d to 3744201 Compare October 9, 2026 23:38

Navigation Menu

Repository navigation

Static-mode IPv6 for Cuttlefish guests: ULA + NAT66 by default, opt-in routed /48, e2e tests - #3104

Static-mode IPv6 for Cuttlefish guests: ULA + NAT66 by default, opt-in routed /48, e2e tests#3104
sferrogoo wants to merge 10 commits into
google:mainfrom
sferrogoo:cuttlefish-ipv6-enablement

sferrogoo commented Aug 27, 2026 •

edited

Loading

Uh oh!

sferrogoo commented Aug 31, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sferrogoo commented Sep 10, 2026

Uh oh!

sferrogoo commented Sep 10, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

sferrogoo commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Commits

Companion AOSP changes (same bug)

Testing

Uh oh!

sferrogoo commented Aug 31, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sferrogoo commented Sep 10, 2026

Uh oh!

sferrogoo commented Sep 10, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sferrogoo commented Aug 27, 2026 •

edited

Loading