diff --git a/CHANGELOG.md b/CHANGELOG.md index 0c31249d..93b9976a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,8 @@ until 0.1.0. Introduced `tai64n_sub_duration`, rejected pre-boot replayed Inits on cold start, initialized rendezvous sequence numbers with wall-clock epoch ms, and rejected post-TTL `RegisterSigned` records. ### Performance +- **Configurable UMEM page pre-faulting (`MAP_POPULATE` vs lazy on-demand) & tunable `BatchUdpSocket` burst sizing (`crates/yip-io`, `bin/yipd`, `crates/yip-bench`, #173).** + Added `UmemPoolConfig` and `UmemPool::new_lazy` / `with_config` / `new_slabs_with_config` to eliminate startup latency and resident set size (RSS) bloat on memory-constrained devices. Allowing on-demand lazy paging drops UMEM initialization time from 1,590.98 µs down to **7.25 µs (219.5x faster startup)** while retaining optional page pre-faulting (`pre_fault: true`) and physical memory page locking (`libc::mlock`) for zero-jitter bare-metal environments. Generalized `BatchUdpSocket::recvmmsg_batch` and `sendmmsg_burst` across const-generic burst sizes, adding `BATCH_SIZE_64 = 64` and `BATCH_SIZE_128 = 128` constants to maximize single-queue syscall amortization at 40G+ line rates. Added runtime environment control in `bin/yipd` via `YIP_UMEM_LAZY` and `YIP_UMEM_MLOCK`. - **Per-core UMEM slab allocation & lock-free ring recycling for AF_XDP (`crates/yip-io`, `bin/yipd`, `crates/yip-bench`, #172).** Eliminated centralized free-list contention and cross-core MESI cache bouncing in AF_XDP kernel-bypass networking. Partitioned `UmemPool` into isolated, non-overlapping `UmemSlab`s (`UmemPool::new_slabs` / `create_slabs`) allowing worker threads to allocate and free chunks locally with zero atomic coordination. Added lock-free batch recycling (`UmemSlab::recycle_from_rings`) and batch population (`populate_fill_ring`), and vectorized `produce_batch` / `consume_batch` on all circular descriptor rings (`FillRing`, `CompletionRing`, `RxRing`, `TxRing`) using power-of-two mask and slice copying. Boosted circular ring batch throughput from 1,438 Mops/s to **4,057 Mops/s (0.25 ns/op)** and scaled 8-worker throughput to **19.60 Gbps (1.914 Mpps)** with 0 packet drops and 0 reordering. - **Zero-allocation FEC reassembler hot path, in-place RS repair & flat `ObjectTable` (`crates/yip-transport`, `crates/yip-bench`, #171, PR #176).** diff --git a/README.md b/README.md index d8bf4c28..438f68fe 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ Because throughput is inversely proportional to the square root of packet loss p - **Zero-Copy AF_XDP (XSK)**: Direct UMEM memory ring driver with per-core slab partitioning and lock-free ring recycling bypasses the Linux network stack, sustaining 40.27 Gbps and 3.93 Mpps across 8 CPU cores. - **Asynchronous `io_uring`**: High-performance multishot ring polling drives high packet rates without thread contention or syscall overhead. - **Multi-Queue TUN Sharding**: `IFF_MULTI_QUEUE` worker isolation maps packet flows to dedicated CPU cores via lock-free SPSC ring buffers. -- **Vectorized Socket I/O**: Batched `recvmmsg` and `sendmmsg` with `UDP_SEGMENT` Generic Segmentation Offload (GSO) optimize standard POSIX network paths. +- **Vectorized Socket I/O**: Batched `recvmmsg` and `sendmmsg` (tunable 32/64/128-packet bursts) with `UDP_SEGMENT` Generic Segmentation Offload (GSO) optimize standard POSIX network paths. - **Flat Open-Addressing Tables**: Replaces dynamic pointer maps with bounded flat slot tables using SplitMix64 fast hashing and backward-shift deletion with zero tombstones. ### 4. Post-Quantum Cryptography & Identity @@ -206,13 +206,13 @@ Deterministic benchmarks measured across multi-region cloud WANs (AWS `us-east-1 -*Two 1-vCPU Debian 13 VPSes, ~60.939 ms apart, 2026-10-26. yip trades raw clean-path throughput for loss resilience (see the FEC row).* +*Two 1-vCPU Debian 13 VPSes, ~54.365 ms apart, 2026-10-27. yip trades raw clean-path throughput for loss resilience (see the FEC row).* | | TCP 1-stream | UDP loss (200M) | effective loss @10% injected | |---|--:|--:|--:| -| **yip** | 274.2 Mbit/s | 0.00% | 1% | -| **WireGuard** | 451.8 Mbit/s | 0.01% | 11% | -| **OpenVPN** | 34.0 Mbit/s | 0.04% | 10% | +| **yip** | 253.7 Mbit/s | 0.00% | 0% | +| **WireGuard** | 453.1 Mbit/s | 0.00% | 7% | +| **OpenVPN** | 32.6 Mbit/s | 0.04% | 8% | diff --git a/ROADMAP.md b/ROADMAP.md index 57b91449..1c58bcd7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,7 +14,9 @@ maintainer. wire-auth 64-bit counter-bound FEC reassembly with pre-AEAD anti-replay gating, and bidirectional flow accounting in the traffic classifier (#11). - **Multi-core throughput & kernel-bypass** — multi-queue TUN worker sharding (`IFF_MULTI_QUEUE`), - core-pinned flow hashing, lock-free chunked nonce dispensers, circular replay windows, and + core-pinned flow hashing, lock-free chunked nonce dispensers, circular replay windows, + per-core UMEM slab partitioning with vectorized ring recycling (#172), configurable lazy on-demand paging vs `MAP_POPULATE` and `mlock` page pinning (#173), + tunable 32/64/128-datagram syscall burst sizing in `BatchUdpSocket` (#173), and AF_XDP zero-copy driver (over 40 Gbps aggregate throughput, 5.3x higher goodput than kernel WireGuard under 5% packet loss). - **Control plane & mesh routing** — multi-peer routing, self-certifying key-derived addresses diff --git a/assets/bench-fec-recovery.svg b/assets/bench-fec-recovery.svg index 686a9252..2e10a880 100644 --- a/assets/bench-fec-recovery.svg +++ b/assets/bench-fec-recovery.svg @@ -45,35 +45,35 @@ 10% Injected Network Loss (netem) Effective Ping Packet Loss % - - + + - - - - - + + + + + - - - - + + + + - - - - + + + + - - - 11% effective loss - - 1% loss (90% recovered) + + + 7% effective loss + + 0% loss (100% recovered) Cauchy RS GF(256): 100% recovery before kernel TCP timeout diff --git a/assets/bench-throughput-loss.svg b/assets/bench-throughput-loss.svg index 4f9435f2..64b2eb68 100644 --- a/assets/bench-throughput-loss.svg +++ b/assets/bench-throughput-loss.svg @@ -19,8 +19,8 @@ OpenVPN - - 173× to 193× higher throughput under loss + + 74× to 138× higher throughput under loss 0.5 @@ -53,42 +53,42 @@ 10% Netem Injected Packet Loss TCP Goodput (Mbit/s, log scale) - - - + + + - - - - - - + + + + + + - - - - + + + + - - - - - - - - - 251.3M - - 193× advantage (251.3 vs 1.3 Mbit/s) - - 86.6 Mbit/s - - 0.5 Mbit/s - - - - - 173× higher (86.6 vs 0.5) + + + + + + + + + 179.3M + + 138× advantage (179.3 vs 1.3 Mbit/s) + + 44.2 Mbit/s + + 0.6 Mbit/s + + + + + 74× higher (44.2 vs 0.6) NOTE Systematic Reed–Solomon FEC trades minor clean-path overhead for near-complete immunity against loss. diff --git a/bin/yipd/src/sharding.rs b/bin/yipd/src/sharding.rs index a444552e..51040c12 100644 --- a/bin/yipd/src/sharding.rs +++ b/bin/yipd/src/sharding.rs @@ -153,7 +153,7 @@ use std::os::fd::AsRawFd; use std::sync::atomic::{AtomicBool, Ordering}; use yip_io::af_xdp::{ - FillRing, UmemPool, UmemSlab, XskBindMode, XskDesc, XskSocket, UMEM_RING_SIZE, + FillRing, UmemPool, UmemPoolConfig, UmemSlab, XskBindMode, XskDesc, XskSocket, UMEM_RING_SIZE, }; use yip_io::poll::Dispatch; @@ -647,7 +647,19 @@ pub fn run_sharded(config: Config, num_shards: usize) -> io::Result<()> { // Allocate single shared UMEM pool partitioned into per-worker slabs for AF_XDP let mut slabs: Option> = if std::env::var("YIP_XDP_IFNAME").is_ok() { - UmemPool::new_slabs(2048 * num_shards, 4096, num_shards) + let lazy = std::env::var("YIP_UMEM_LAZY") + .map(|v| v == "1" || v.eq_ignore_ascii_case("true")) + .unwrap_or(false); + let mlock = std::env::var("YIP_UMEM_MLOCK") + .map(|v| v == "1" || v.eq_ignore_ascii_case("true")) + .unwrap_or(false); + let pool_cfg = UmemPoolConfig { + num_chunks: 2048 * num_shards, + chunk_size: 4096, + pre_fault: !lazy, + lock_memory: mlock, + }; + UmemPool::new_slabs_with_config(pool_cfg, num_shards) .ok() .map(|(_, s)| s) } else { diff --git a/crates/yip-bench/RESULTS.md b/crates/yip-bench/RESULTS.md index 03f29725..709c87c5 100644 --- a/crates/yip-bench/RESULTS.md +++ b/crates/yip-bench/RESULTS.md @@ -740,8 +740,8 @@ Regime F delivers: ### Conditions - **Hosts:** `EC2 us-east-1 (t3.small)` ↔ `EC2 us-west-2 (t3.small)` (two 1-vCPU / ~1 GB Debian 13 VPSes, kernel 6.12.63+deb13-cloud-amd64) -- **Baseline path:** ~60.939 ms RTT, single Gbit links -- **Date / commit:** 2026-10-26T22:26:35Z / `348fcc0` +- **Baseline path:** ~54.365 ms RTT, single Gbit links +- **Date / commit:** 2026-10-27T14:51:10Z / `4491d57` - **Host tuning (applied equally, all tunnels):** `net.core.rmem_max=net.core.wmem_max=16 MiB` (so yipd's userspace 4 MiB socket buffers take effect; in-kernel WireGuard is unaffected) - **All tunnels:** UDP outer transport, MTU 1380; yip `transport=raw` (epoll driver — kernel 6.12 has no io_uring multishot recv, #25); WireGuard kernel module; OpenVPN static-key AES-256-GCM @@ -751,10 +751,10 @@ Regime F delivers: | tunnel | TCP 1-stream | TCP 4-stream | UDP goodput | UDP loss | idle RTT | RTT under load | CPU (A, 1 vCPU) | |---|--:|--:|--:|--:|--:|--:|--:| -| baseline (no tunnel) | 383.8 Mbit/s | 922.7 Mbit/s | 200.0 Mbit/s | 0.00% | 60.939 ms | 60.982 ms | 1.3% | -| yip | 274.2 Mbit/s | 239.5 Mbit/s | 200.0 Mbit/s | 0.00% | 57.514 ms | 59.816 ms | 7.8% | -| WireGuard | 451.8 Mbit/s | 783.1 Mbit/s | 200.0 Mbit/s | 0.01% | 54.813 ms | 54.931 ms | 10.5% | -| OpenVPN | 34.0 Mbit/s | 47.5 Mbit/s | 200.0 Mbit/s | 0.04% | 54.835 ms | 54.955 ms | 4.3% | +| baseline (no tunnel) | 390.7 Mbit/s | 906.1 Mbit/s | 200.0 Mbit/s | 0.00% | 54.365 ms | 54.360 ms | 1.7% | +| yip | 253.7 Mbit/s | 350.7 Mbit/s | 200.0 Mbit/s | 0.00% | 55.288 ms | 56.051 ms | 12.5% | +| WireGuard | 453.1 Mbit/s | 1077.7 Mbit/s | 200.0 Mbit/s | 0.00% | 54.981 ms | 55.067 ms | 11.2% | +| OpenVPN | 32.6 Mbit/s | 35.6 Mbit/s | 200.0 Mbit/s | 0.04% | 54.745 ms | 54.661 ms | 3.7% | ### Loss resilience (netem injected on the A→B outer path) @@ -763,20 +763,20 @@ Effective end-to-end loss (ping) — lower is better; yip's FEC recovers what th | injected loss | baseline (no tunnel) | yip | WireGuard | OpenVPN | |---|--:|--:|--:|--:| | 0% | 0% | 0% | 0% | 0% | -| 1% | 1% | 0% | 0% | 0% | -| 3% | 2% | 0% | 0% | 4% | -| 5% | 2% | 0% | 4% | 1% | -| 10% | 7% | 1% | 11% | 10% | +| 1% | 0% | 0% | 0% | 0% | +| 3% | 0% | 0% | 2% | 1% | +| 5% | 1% | 0% | 2% | 4% | +| 10% | 9% | 0% | 7% | 8% | TCP goodput under the same injected loss (Mbit/s): | injected loss | baseline (no tunnel) | yip | WireGuard | OpenVPN | |---|--:|--:|--:|--:| -| 0% | 425.4 | 289.3 | 446.8 | 27.0 | -| 1% | 6.1 | 127.7 | 2.2 | 2.7 | -| 3% | 2.2 | 251.3 | 1.3 | 1.9 | -| 5% | 1.5 | 118.6 | 0.8 | 1.0 | -| 10% | 1.0 | 86.6 | 0.5 | 0.5 | +| 0% | 497.9 | 307.2 | 455.6 | 27.7 | +| 1% | 32.8 | 310.5 | 2.1 | 3.7 | +| 3% | 2.2 | 179.3 | 1.3 | 1.2 | +| 5% | 1.6 | 232.6 | 1.2 | 1.0 | +| 10% | 0.7 | 44.2 | 0.6 | 0.6 | ### Reproduce ```sh diff --git a/crates/yip-bench/benches/af_xdp_scale.rs b/crates/yip-bench/benches/af_xdp_scale.rs index 1331ab4c..ebbb1554 100644 --- a/crates/yip-bench/benches/af_xdp_scale.rs +++ b/crates/yip-bench/benches/af_xdp_scale.rs @@ -17,6 +17,7 @@ use std::collections::hash_map::DefaultHasher; use std::collections::HashMap; use std::hash::{Hash, Hasher}; +use std::net::UdpSocket; use std::sync::{Arc, Barrier}; use std::thread; use std::time::Instant; @@ -24,6 +25,7 @@ use std::time::Instant; use yip_bench::established_raw_keys; use yip_crypto::{ChaCha20Poly1305Cipher, ReplayWindow, Session}; use yip_io::af_xdp::{CompletionRing, FillRing, RxRing, TxRing, UmemPool, XskDesc}; +use yip_io::batch::{BatchUdpSocket, BATCH_SIZE, BATCH_SIZE_128, BATCH_SIZE_64}; /// Inner 5-tuple extracted from plaintext IP packets. #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] @@ -218,6 +220,36 @@ fn run_component_microbenchmarks() { ); } + // 1c. UMEM Startup Latency: Pre-Fault (MAP_POPULATE) vs Lazy (On-Demand) + { + let iters = 1_000; + let start_prefault = Instant::now(); + for _ in 0..iters { + let pool = UmemPool::new(1024, CHUNK_SIZE).expect("prefault pool"); + std::hint::black_box(pool); + } + let elapsed_prefault = start_prefault.elapsed().as_secs_f64(); + let us_prefault = (elapsed_prefault / iters as f64) * 1e6; + + let start_lazy = Instant::now(); + for _ in 0..iters { + let pool = UmemPool::new_lazy(1024, CHUNK_SIZE).expect("lazy pool"); + std::hint::black_box(pool); + } + let elapsed_lazy = start_lazy.elapsed().as_secs_f64(); + let us_lazy = (elapsed_lazy / iters as f64) * 1e6; + + println!( + " UMEM Init (MAP_POPULATE): {:6.2} µs/pool (full physical page commit)", + us_prefault + ); + println!( + " UMEM Init (Lazy On-Demand): {:6.2} µs/pool ({:.1}x faster startup)", + us_lazy, + us_prefault / us_lazy.max(0.001) + ); + } + // 2. Circular Ring Batch Enqueue / Dequeue Throughput { let batch_size = 32; @@ -289,6 +321,80 @@ fn run_component_microbenchmarks() { ); } + // 2c. Vectorized Syscall Burst Scaling: Batch 32 vs 64 vs 128 + { + let s1 = UdpSocket::bind("127.0.0.1:0").expect("bind s1"); + let s2 = UdpSocket::bind("127.0.0.1:0").expect("bind s2"); + s1.set_nonblocking(true).unwrap(); + s2.set_nonblocking(true).unwrap(); + let addr2 = s2.local_addr().unwrap(); + let mut b1 = BatchUdpSocket::new(&s1); + let mut b2 = BatchUdpSocket::new(&s2); + + let iters = 2_000; + let payload = [0x5au8; 1280]; + + // Batch 32 + let pkts_32: Vec<(&[u8], std::net::SocketAddr)> = + (0..BATCH_SIZE).map(|_| (&payload[..], addr2)).collect(); + let mut rx_buf_32 = [[0u8; yip_io::MAX_WIRE_DATAGRAM]; BATCH_SIZE]; + let mut rx_out_32 = [const { yip_io::batch::ReceivedDatagram::empty() }; BATCH_SIZE]; + + let start_32 = Instant::now(); + for _ in 0..iters { + let _ = b1.sendmmsg_burst::(&pkts_32); + let _ = b2.recvmmsg_batch::(&mut rx_buf_32, &mut rx_out_32); + } + let elapsed_32 = start_32.elapsed().as_secs_f64(); + let mpps_32 = (iters * BATCH_SIZE) as f64 / (elapsed_32 * 1e6); + + // Batch 64 + let pkts_64: Vec<(&[u8], std::net::SocketAddr)> = + (0..BATCH_SIZE_64).map(|_| (&payload[..], addr2)).collect(); + let mut rx_buf_64 = [[0u8; yip_io::MAX_WIRE_DATAGRAM]; BATCH_SIZE_64]; + let mut rx_out_64 = [const { yip_io::batch::ReceivedDatagram::empty() }; BATCH_SIZE_64]; + + let start_64 = Instant::now(); + for _ in 0..iters { + let _ = b1.sendmmsg_burst::(&pkts_64); + let _ = b2.recvmmsg_batch::(&mut rx_buf_64, &mut rx_out_64); + } + let elapsed_64 = start_64.elapsed().as_secs_f64(); + let mpps_64 = (iters * BATCH_SIZE_64) as f64 / (elapsed_64 * 1e6); + + // Batch 128 + let pkts_128: Vec<(&[u8], std::net::SocketAddr)> = + (0..BATCH_SIZE_128).map(|_| (&payload[..], addr2)).collect(); + let mut rx_buf_128 = [[0u8; yip_io::MAX_WIRE_DATAGRAM]; BATCH_SIZE_128]; + let mut rx_out_128 = [const { yip_io::batch::ReceivedDatagram::empty() }; BATCH_SIZE_128]; + + let start_128 = Instant::now(); + for _ in 0..iters { + let _ = b1.sendmmsg_burst::(&pkts_128); + let _ = b2.recvmmsg_batch::(&mut rx_buf_128, &mut rx_out_128); + } + let elapsed_128 = start_128.elapsed().as_secs_f64(); + let mpps_128 = (iters * BATCH_SIZE_128) as f64 / (elapsed_128 * 1e6); + + println!( + " Syscall Burst Batch 32: {:6.3} Mpps ({:.2} ns/packet)", + mpps_32, + (elapsed_32 / (iters * BATCH_SIZE) as f64) * 1e9 + ); + println!( + " Syscall Burst Batch 64: {:6.3} Mpps ({:.2} ns/packet, {:.1}x throughput)", + mpps_64, + (elapsed_64 / (iters * BATCH_SIZE_64) as f64) * 1e9, + mpps_64 / mpps_32.max(0.001) + ); + println!( + " Syscall Burst Batch 128: {:6.3} Mpps ({:.2} ns/packet, {:.1}x throughput)", + mpps_128, + (elapsed_128 / (iters * BATCH_SIZE_128) as f64) * 1e9, + mpps_128 / mpps_32.max(0.001) + ); + } + // 3. In-Place Zero-Copy AEAD Seal & Open Inside UMEM Chunks { let iters = 100_000; diff --git a/crates/yip-bench/wan/results/20261027T145055Z/wan.csv b/crates/yip-bench/wan/results/20261027T145055Z/wan.csv new file mode 100644 index 00000000..079d2106 --- /dev/null +++ b/crates/yip-bench/wan/results/20261027T145055Z/wan.csv @@ -0,0 +1,73 @@ +ts,host_pair,commit,tunnel,profile,param,metric,value,conditions +2026-10-27T14:51:10Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=1,mbit_s,374.2,clean +2026-10-27T14:51:23Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=4,mbit_s,741.0,clean +2026-10-27T14:51:33Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:51:33Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_jitter,offered=200M,ms,0.073,clean +2026-10-27T14:51:33Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:51:49Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,idle,rtt_ms,54.420,clean +2026-10-27T14:51:49Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,loaded,rtt_ms,54.417,saturated +2026-10-27T14:51:56Z,54.242.165.6->54.187.92.88,4491d57,baseline,cpu_used,tcp_saturate,pct,2.8,sender_host_A_1vcpu +2026-10-27T14:52:15Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:52:15Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=0pct,mbit_s,497.9,netem_0pct +2026-10-27T14:52:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=1pct,pct,1,netem_1pct +2026-10-27T14:52:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=1pct,mbit_s,3.0,netem_1pct +2026-10-27T14:52:55Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=3pct,pct,2,netem_3pct +2026-10-27T14:52:55Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=3pct,mbit_s,1.9,netem_3pct +2026-10-27T14:53:17Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=5pct,pct,6,netem_5pct +2026-10-27T14:53:17Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=5pct,mbit_s,1.6,netem_5pct +2026-10-27T14:53:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=10pct,pct,9,netem_10pct +2026-10-27T14:53:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=10pct,mbit_s,0.7,netem_10pct +2026-10-27T14:53:49Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=1,mbit_s,253.7,clean +2026-10-27T14:54:02Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=4,mbit_s,350.7,clean +2026-10-27T14:54:13Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:54:13Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T14:54:13Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:54:29Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,idle,rtt_ms,55.354,clean +2026-10-27T14:54:29Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,loaded,rtt_ms,62.208,saturated +2026-10-27T14:54:36Z,54.242.165.6->54.187.92.88,4491d57,yip,cpu_used,tcp_saturate,pct,18.2,sender_host_A_1vcpu +2026-10-27T14:54:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:54:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=0pct,mbit_s,307.2,netem_0pct +2026-10-27T14:55:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:55:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=1pct,mbit_s,102.6,netem_1pct +2026-10-27T14:55:35Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T14:55:35Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=3pct,mbit_s,176.6,netem_3pct +2026-10-27T14:55:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=5pct,pct,0,netem_5pct +2026-10-27T14:55:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=5pct,mbit_s,52.0,netem_5pct +2026-10-27T14:56:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=10pct,pct,0,netem_10pct +2026-10-27T14:56:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=10pct,mbit_s,28.1,netem_10pct +2026-10-27T14:56:28Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=1,mbit_s,424.5,clean +2026-10-27T14:56:41Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=4,mbit_s,770.9,clean +2026-10-27T14:56:53Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:56:53Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_jitter,offered=200M,ms,0.062,clean +2026-10-27T14:56:53Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_loss,offered=200M,pct,0.02,clean +2026-10-27T14:57:09Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,idle,rtt_ms,58.636,clean +2026-10-27T14:57:09Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,loaded,rtt_ms,59.093,saturated +2026-10-27T14:57:16Z,54.242.165.6->54.187.92.88,4491d57,wireguard,cpu_used,tcp_saturate,pct,11.6,sender_host_A_1vcpu +2026-10-27T14:57:34Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:57:34Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=0pct,mbit_s,430.5,netem_0pct +2026-10-27T14:57:54Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:57:54Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=1pct,mbit_s,1.9,netem_1pct +2026-10-27T14:58:13Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=3pct,pct,7,netem_3pct +2026-10-27T14:58:13Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=3pct,mbit_s,1.0,netem_3pct +2026-10-27T14:58:32Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=5pct,pct,5,netem_5pct +2026-10-27T14:58:32Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=5pct,mbit_s,1.2,netem_5pct +2026-10-27T14:58:52Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=10pct,pct,7,netem_10pct +2026-10-27T14:58:52Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=10pct,mbit_s,0.5,netem_10pct +2026-10-27T14:59:06Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=1,mbit_s,22.9,clean +2026-10-27T14:59:19Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=4,mbit_s,34.1,clean +2026-10-27T14:59:30Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:59:30Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_jitter,offered=200M,ms,0.064,clean +2026-10-27T14:59:30Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_loss,offered=200M,pct,0.04,clean +2026-10-27T14:59:46Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,idle,rtt_ms,54.745,clean +2026-10-27T14:59:46Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,loaded,rtt_ms,54.855,saturated +2026-10-27T14:59:52Z,54.242.165.6->54.187.92.88,4491d57,openvpn,cpu_used,tcp_saturate,pct,3.9,sender_host_A_1vcpu +2026-10-27T15:00:11Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:00:11Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=0pct,mbit_s,18.4,netem_0pct +2026-10-27T15:00:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T15:00:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=1pct,mbit_s,2.6,netem_1pct +2026-10-27T15:00:52Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=3pct,pct,4,netem_3pct +2026-10-27T15:00:52Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=3pct,mbit_s,1.2,netem_3pct +2026-10-27T15:01:12Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=5pct,pct,5,netem_5pct +2026-10-27T15:01:12Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=5pct,mbit_s,0.9,netem_5pct +2026-10-27T15:01:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=10pct,pct,14,netem_10pct +2026-10-27T15:01:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=10pct,mbit_s,0.6,netem_10pct diff --git a/crates/yip-bench/wan/results/20261027T150145Z/wan.csv b/crates/yip-bench/wan/results/20261027T150145Z/wan.csv new file mode 100644 index 00000000..962c60a9 --- /dev/null +++ b/crates/yip-bench/wan/results/20261027T150145Z/wan.csv @@ -0,0 +1,73 @@ +ts,host_pair,commit,tunnel,profile,param,metric,value,conditions +2026-10-27T15:02:00Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=1,mbit_s,390.7,clean +2026-10-27T15:02:13Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=4,mbit_s,906.1,clean +2026-10-27T15:02:24Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:02:24Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_jitter,offered=200M,ms,0.072,clean +2026-10-27T15:02:24Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T15:02:40Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,idle,rtt_ms,54.365,clean +2026-10-27T15:02:40Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,loaded,rtt_ms,54.360,saturated +2026-10-27T15:02:47Z,54.242.165.6->54.187.92.88,4491d57,baseline,cpu_used,tcp_saturate,pct,1.7,sender_host_A_1vcpu +2026-10-27T15:03:05Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:03:05Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=0pct,mbit_s,267.4,netem_0pct +2026-10-27T15:03:25Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T15:03:25Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=1pct,mbit_s,32.8,netem_1pct +2026-10-27T15:03:44Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T15:03:44Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=3pct,mbit_s,2.2,netem_3pct +2026-10-27T15:04:04Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=5pct,pct,1,netem_5pct +2026-10-27T15:04:04Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=5pct,mbit_s,1.5,netem_5pct +2026-10-27T15:04:25Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=10pct,pct,16,netem_10pct +2026-10-27T15:04:25Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=10pct,mbit_s,0.7,netem_10pct +2026-10-27T15:04:38Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=1,mbit_s,250.5,clean +2026-10-27T15:04:51Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=4,mbit_s,256.4,clean +2026-10-27T15:05:02Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:05:02Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T15:05:02Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_loss,offered=200M,pct,0.34,clean +2026-10-27T15:05:18Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,idle,rtt_ms,55.401,clean +2026-10-27T15:05:18Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,loaded,rtt_ms,58.233,saturated +2026-10-27T15:05:25Z,54.242.165.6->54.187.92.88,4491d57,yip,cpu_used,tcp_saturate,pct,13.8,sender_host_A_1vcpu +2026-10-27T15:05:43Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:05:43Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=0pct,mbit_s,290.5,netem_0pct +2026-10-27T15:06:04Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T15:06:04Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=1pct,mbit_s,130.9,netem_1pct +2026-10-27T15:06:24Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T15:06:24Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=3pct,mbit_s,179.3,netem_3pct +2026-10-27T15:06:43Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=5pct,pct,0,netem_5pct +2026-10-27T15:06:43Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=5pct,mbit_s,63.4,netem_5pct +2026-10-27T15:07:04Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=10pct,pct,2,netem_10pct +2026-10-27T15:07:04Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=10pct,mbit_s,5.2,netem_10pct +2026-10-27T15:07:17Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=1,mbit_s,448.2,clean +2026-10-27T15:07:30Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=4,mbit_s,1077.7,clean +2026-10-27T15:07:41Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:07:41Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_jitter,offered=200M,ms,0.062,clean +2026-10-27T15:07:41Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T15:07:58Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,idle,rtt_ms,55.048,clean +2026-10-27T15:07:58Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,loaded,rtt_ms,55.230,saturated +2026-10-27T15:08:04Z,54.242.165.6->54.187.92.88,4491d57,wireguard,cpu_used,tcp_saturate,pct,12.8,sender_host_A_1vcpu +2026-10-27T15:08:23Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:08:23Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=0pct,mbit_s,455.6,netem_0pct +2026-10-27T15:08:44Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=1pct,pct,1,netem_1pct +2026-10-27T15:08:44Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=1pct,mbit_s,1.9,netem_1pct +2026-10-27T15:09:04Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=3pct,pct,2,netem_3pct +2026-10-27T15:09:04Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=3pct,mbit_s,1.1,netem_3pct +2026-10-27T15:09:24Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=5pct,pct,11,netem_5pct +2026-10-27T15:09:24Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=5pct,mbit_s,1.0,netem_5pct +2026-10-27T15:09:44Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=10pct,pct,11,netem_10pct +2026-10-27T15:09:44Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=10pct,mbit_s,0.4,netem_10pct +2026-10-27T15:09:57Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=1,mbit_s,26.1,clean +2026-10-27T15:10:10Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=4,mbit_s,29.6,clean +2026-10-27T15:10:21Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:10:21Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_jitter,offered=200M,ms,0.050,clean +2026-10-27T15:10:21Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_loss,offered=200M,pct,0.06,clean +2026-10-27T15:10:38Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,idle,rtt_ms,54.748,clean +2026-10-27T15:10:38Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,loaded,rtt_ms,54.948,saturated +2026-10-27T15:10:44Z,54.242.165.6->54.187.92.88,4491d57,openvpn,cpu_used,tcp_saturate,pct,3.7,sender_host_A_1vcpu +2026-10-27T15:11:03Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:11:03Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=0pct,mbit_s,20.6,netem_0pct +2026-10-27T15:11:22Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=1pct,pct,3,netem_1pct +2026-10-27T15:11:22Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=1pct,mbit_s,2.9,netem_1pct +2026-10-27T15:11:42Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=3pct,pct,1,netem_3pct +2026-10-27T15:11:42Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=3pct,mbit_s,1.0,netem_3pct +2026-10-27T15:12:01Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=5pct,pct,4,netem_5pct +2026-10-27T15:12:01Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=5pct,mbit_s,1.0,netem_5pct +2026-10-27T15:12:22Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=10pct,pct,10,netem_10pct +2026-10-27T15:12:22Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=10pct,mbit_s,0.4,netem_10pct diff --git a/crates/yip-bench/wan/results/20261027T151228Z/wan.csv b/crates/yip-bench/wan/results/20261027T151228Z/wan.csv new file mode 100644 index 00000000..3a090b1a --- /dev/null +++ b/crates/yip-bench/wan/results/20261027T151228Z/wan.csv @@ -0,0 +1,73 @@ +ts,host_pair,commit,tunnel,profile,param,metric,value,conditions +2026-10-27T15:12:43Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=1,mbit_s,216.8,clean +2026-10-27T15:12:56Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=4,mbit_s,790.4,clean +2026-10-27T15:13:07Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:13:07Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_jitter,offered=200M,ms,0.062,clean +2026-10-27T15:13:07Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_loss,offered=200M,pct,0.39,clean +2026-10-27T15:13:23Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,idle,rtt_ms,54.821,clean +2026-10-27T15:13:23Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,loaded,rtt_ms,54.808,saturated +2026-10-27T15:13:30Z,54.242.165.6->54.187.92.88,4491d57,baseline,cpu_used,tcp_saturate,pct,1.9,sender_host_A_1vcpu +2026-10-27T15:13:49Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:13:49Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=0pct,mbit_s,280.7,netem_0pct +2026-10-27T15:14:10Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=1pct,pct,1,netem_1pct +2026-10-27T15:14:10Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=1pct,mbit_s,2.9,netem_1pct +2026-10-27T15:14:29Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=3pct,pct,1,netem_3pct +2026-10-27T15:14:29Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=3pct,mbit_s,1.8,netem_3pct +2026-10-27T15:14:50Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=5pct,pct,9,netem_5pct +2026-10-27T15:14:50Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=5pct,mbit_s,1.4,netem_5pct +2026-10-27T15:15:10Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=10pct,pct,10,netem_10pct +2026-10-27T15:15:10Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=10pct,mbit_s,0.6,netem_10pct +2026-10-27T15:15:24Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=1,mbit_s,95.1,clean +2026-10-27T15:15:37Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=4,mbit_s,318.3,clean +2026-10-27T15:15:48Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:15:48Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T15:15:48Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T15:16:04Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,idle,rtt_ms,55.288,clean +2026-10-27T15:16:04Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,loaded,rtt_ms,56.051,saturated +2026-10-27T15:16:10Z,54.242.165.6->54.187.92.88,4491d57,yip,cpu_used,tcp_saturate,pct,12.5,sender_host_A_1vcpu +2026-10-27T15:16:29Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:16:29Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=0pct,mbit_s,97.3,netem_0pct +2026-10-27T15:16:49Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T15:16:49Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=1pct,mbit_s,310.5,netem_1pct +2026-10-27T15:17:09Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T15:17:09Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=3pct,mbit_s,165.9,netem_3pct +2026-10-27T15:17:28Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=5pct,pct,0,netem_5pct +2026-10-27T15:17:28Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=5pct,mbit_s,232.6,netem_5pct +2026-10-27T15:17:47Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=10pct,pct,1,netem_10pct +2026-10-27T15:17:47Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=10pct,mbit_s,44.2,netem_10pct +2026-10-27T15:18:01Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=1,mbit_s,453.1,clean +2026-10-27T15:18:14Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=4,mbit_s,603.4,clean +2026-10-27T15:18:25Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:18:25Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T15:18:25Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_loss,offered=200M,pct,0.01,clean +2026-10-27T15:18:41Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,idle,rtt_ms,54.981,clean +2026-10-27T15:18:41Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,loaded,rtt_ms,55.067,saturated +2026-10-27T15:18:48Z,54.242.165.6->54.187.92.88,4491d57,wireguard,cpu_used,tcp_saturate,pct,11.2,sender_host_A_1vcpu +2026-10-27T15:19:06Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:19:06Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=0pct,mbit_s,398.4,netem_0pct +2026-10-27T15:19:26Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=1pct,pct,1,netem_1pct +2026-10-27T15:19:26Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=1pct,mbit_s,2.1,netem_1pct +2026-10-27T15:19:46Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=3pct,pct,3,netem_3pct +2026-10-27T15:19:46Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=3pct,mbit_s,1.3,netem_3pct +2026-10-27T15:20:05Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=5pct,pct,2,netem_5pct +2026-10-27T15:20:05Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=5pct,mbit_s,0.7,netem_5pct +2026-10-27T15:20:26Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=10pct,pct,14,netem_10pct +2026-10-27T15:20:26Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=10pct,mbit_s,0.6,netem_10pct +2026-10-27T15:20:39Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=1,mbit_s,32.6,clean +2026-10-27T15:20:52Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=4,mbit_s,35.6,clean +2026-10-27T15:21:03Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T15:21:03Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_jitter,offered=200M,ms,0.045,clean +2026-10-27T15:21:03Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_loss,offered=200M,pct,8.07,clean +2026-10-27T15:21:20Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,idle,rtt_ms,54.892,clean +2026-10-27T15:21:20Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,loaded,rtt_ms,54.661,saturated +2026-10-27T15:21:26Z,54.242.165.6->54.187.92.88,4491d57,openvpn,cpu_used,tcp_saturate,pct,3.9,sender_host_A_1vcpu +2026-10-27T15:21:45Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:21:45Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=0pct,mbit_s,27.7,netem_0pct +2026-10-27T15:22:04Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=1pct,pct,1,netem_1pct +2026-10-27T15:22:04Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=1pct,mbit_s,3.7,netem_1pct +2026-10-27T15:22:25Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=3pct,pct,3,netem_3pct +2026-10-27T15:22:25Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=3pct,mbit_s,1.0,netem_3pct +2026-10-27T15:22:44Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=5pct,pct,8,netem_5pct +2026-10-27T15:22:44Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=5pct,mbit_s,1.0,netem_5pct +2026-10-27T15:23:04Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=10pct,pct,8,netem_10pct +2026-10-27T15:23:04Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=10pct,mbit_s,0.4,netem_10pct diff --git a/crates/yip-bench/wan/results/best_of_3/wan.csv b/crates/yip-bench/wan/results/best_of_3/wan.csv index 6b0126bc..a3b0ec9d 100644 --- a/crates/yip-bench/wan/results/best_of_3/wan.csv +++ b/crates/yip-bench/wan/results/best_of_3/wan.csv @@ -1,73 +1,73 @@ ts,host_pair,commit,tunnel,profile,param,metric,value,conditions -2026-10-26T22:26:35Z,54.226.134.9->52.42.20.20,348fcc0,baseline,tcp_throughput,streams=1,mbit_s,383.8,clean -2026-10-26T22:26:49Z,54.226.134.9->52.42.20.20,348fcc0,baseline,tcp_throughput,streams=4,mbit_s,922.7,clean -2026-10-26T22:27:00Z,54.226.134.9->52.42.20.20,348fcc0,baseline,udp_throughput,offered=200M,mbit_s,200.0,clean -2026-10-26T22:27:00Z,54.226.134.9->52.42.20.20,348fcc0,baseline,udp_jitter,offered=200M,ms,0.008,clean -2026-10-26T22:27:00Z,54.226.134.9->52.42.20.20,348fcc0,baseline,udp_loss,offered=200M,pct,0.00,clean -2026-10-26T22:27:17Z,54.226.134.9->52.42.20.20,348fcc0,baseline,latency,idle,rtt_ms,60.939,clean -2026-10-26T22:27:17Z,54.226.134.9->52.42.20.20,348fcc0,baseline,latency,loaded,rtt_ms,60.982,saturated -2026-10-26T22:27:24Z,54.226.134.9->52.42.20.20,348fcc0,baseline,cpu_used,tcp_saturate,pct,1.3,sender_host_A_1vcpu -2026-10-26T22:27:43Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_effective,injected=0pct,pct,0,netem_0pct -2026-10-26T22:27:43Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_goodput,injected=0pct,mbit_s,425.4,netem_0pct -2026-10-26T22:28:06Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_effective,injected=1pct,pct,1,netem_1pct -2026-10-26T22:28:06Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_goodput,injected=1pct,mbit_s,6.1,netem_1pct -2026-10-26T22:28:27Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_effective,injected=3pct,pct,2,netem_3pct -2026-10-26T22:28:27Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_goodput,injected=3pct,mbit_s,2.2,netem_3pct -2026-10-26T22:28:48Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_effective,injected=5pct,pct,2,netem_5pct -2026-10-26T22:28:48Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_goodput,injected=5pct,mbit_s,1.5,netem_5pct -2026-10-26T22:29:11Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_effective,injected=10pct,pct,7,netem_10pct -2026-10-26T22:29:11Z,54.226.134.9->52.42.20.20,348fcc0,baseline,loss_goodput,injected=10pct,mbit_s,1.0,netem_10pct -2026-10-26T22:29:25Z,54.226.134.9->52.42.20.20,348fcc0,yip,tcp_throughput,streams=1,mbit_s,274.2,clean -2026-10-26T22:29:39Z,54.226.134.9->52.42.20.20,348fcc0,yip,tcp_throughput,streams=4,mbit_s,239.5,clean -2026-10-26T22:29:50Z,54.226.134.9->52.42.20.20,348fcc0,yip,udp_throughput,offered=200M,mbit_s,200.0,clean -2026-10-26T22:29:50Z,54.226.134.9->52.42.20.20,348fcc0,yip,udp_jitter,offered=200M,ms,0.010,clean -2026-10-26T22:29:50Z,54.226.134.9->52.42.20.20,348fcc0,yip,udp_loss,offered=200M,pct,0.00,clean -2026-10-26T22:30:07Z,54.226.134.9->52.42.20.20,348fcc0,yip,latency,idle,rtt_ms,57.514,clean -2026-10-26T22:30:07Z,54.226.134.9->52.42.20.20,348fcc0,yip,latency,loaded,rtt_ms,59.816,saturated -2026-10-26T22:30:15Z,54.226.134.9->52.42.20.20,348fcc0,yip,cpu_used,tcp_saturate,pct,7.8,sender_host_A_1vcpu -2026-10-26T22:30:35Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_effective,injected=0pct,pct,0,netem_0pct -2026-10-26T22:30:35Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_goodput,injected=0pct,mbit_s,289.3,netem_0pct -2026-10-26T22:30:56Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_effective,injected=1pct,pct,0,netem_1pct -2026-10-26T22:30:56Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_goodput,injected=1pct,mbit_s,127.7,netem_1pct -2026-10-26T22:31:16Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_effective,injected=3pct,pct,0,netem_3pct -2026-10-26T22:31:16Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_goodput,injected=3pct,mbit_s,251.3,netem_3pct -2026-10-26T22:31:38Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_effective,injected=5pct,pct,0,netem_5pct -2026-10-26T22:31:38Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_goodput,injected=5pct,mbit_s,118.6,netem_5pct -2026-10-26T22:31:58Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_effective,injected=10pct,pct,1,netem_10pct -2026-10-26T22:31:58Z,54.226.134.9->52.42.20.20,348fcc0,yip,loss_goodput,injected=10pct,mbit_s,86.6,netem_10pct -2026-10-26T22:32:13Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,tcp_throughput,streams=1,mbit_s,451.8,clean -2026-10-26T22:32:27Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,tcp_throughput,streams=4,mbit_s,783.1,clean -2026-10-26T22:32:38Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,udp_throughput,offered=200M,mbit_s,200.0,clean -2026-10-26T22:32:38Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,udp_jitter,offered=200M,ms,0.020,clean -2026-10-26T22:32:38Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,udp_loss,offered=200M,pct,0.01,clean -2026-10-26T22:32:55Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,latency,idle,rtt_ms,54.813,clean -2026-10-26T22:32:55Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,latency,loaded,rtt_ms,54.931,saturated -2026-10-26T22:33:02Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,cpu_used,tcp_saturate,pct,10.5,sender_host_A_1vcpu -2026-10-26T22:33:22Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_effective,injected=0pct,pct,0,netem_0pct -2026-10-26T22:33:22Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_goodput,injected=0pct,mbit_s,446.8,netem_0pct -2026-10-26T22:33:43Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_effective,injected=1pct,pct,0,netem_1pct -2026-10-26T22:33:43Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_goodput,injected=1pct,mbit_s,2.2,netem_1pct -2026-10-26T22:34:04Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_effective,injected=3pct,pct,0,netem_3pct -2026-10-26T22:34:04Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_goodput,injected=3pct,mbit_s,1.3,netem_3pct -2026-10-26T22:34:24Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_effective,injected=5pct,pct,4,netem_5pct -2026-10-26T22:34:24Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_goodput,injected=5pct,mbit_s,0.8,netem_5pct -2026-10-26T22:34:45Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_effective,injected=10pct,pct,11,netem_10pct -2026-10-26T22:34:45Z,54.226.134.9->52.42.20.20,348fcc0,wireguard,loss_goodput,injected=10pct,mbit_s,0.5,netem_10pct -2026-10-26T22:34:59Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,tcp_throughput,streams=1,mbit_s,34.0,clean -2026-10-26T22:35:12Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,tcp_throughput,streams=4,mbit_s,47.5,clean -2026-10-26T22:35:24Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,udp_throughput,offered=200M,mbit_s,200.0,clean -2026-10-26T22:35:24Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,udp_jitter,offered=200M,ms,0.039,clean -2026-10-26T22:35:24Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,udp_loss,offered=200M,pct,0.04,clean -2026-10-26T22:35:40Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,latency,idle,rtt_ms,54.835,clean -2026-10-26T22:35:40Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,latency,loaded,rtt_ms,54.955,saturated -2026-10-26T22:35:48Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,cpu_used,tcp_saturate,pct,4.3,sender_host_A_1vcpu -2026-10-26T22:36:08Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_effective,injected=0pct,pct,0,netem_0pct -2026-10-26T22:36:08Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_goodput,injected=0pct,mbit_s,27.0,netem_0pct -2026-10-26T22:36:29Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_effective,injected=1pct,pct,0,netem_1pct -2026-10-26T22:36:29Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_goodput,injected=1pct,mbit_s,2.7,netem_1pct -2026-10-26T22:36:53Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_effective,injected=3pct,pct,4,netem_3pct -2026-10-26T22:36:53Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_goodput,injected=3pct,mbit_s,1.9,netem_3pct -2026-10-26T22:37:14Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_effective,injected=5pct,pct,1,netem_5pct -2026-10-26T22:37:14Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_goodput,injected=5pct,mbit_s,1.0,netem_5pct -2026-10-26T22:37:35Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_effective,injected=10pct,pct,10,netem_10pct -2026-10-26T22:37:35Z,54.226.134.9->52.42.20.20,348fcc0,openvpn,loss_goodput,injected=10pct,mbit_s,0.5,netem_10pct +2026-10-27T14:51:10Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=1,mbit_s,390.7,clean +2026-10-27T14:51:23Z,54.242.165.6->54.187.92.88,4491d57,baseline,tcp_throughput,streams=4,mbit_s,906.1,clean +2026-10-27T14:51:33Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:51:33Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_jitter,offered=200M,ms,0.062,clean +2026-10-27T14:51:33Z,54.242.165.6->54.187.92.88,4491d57,baseline,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:51:49Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,idle,rtt_ms,54.365,clean +2026-10-27T14:51:49Z,54.242.165.6->54.187.92.88,4491d57,baseline,latency,loaded,rtt_ms,54.360,saturated +2026-10-27T14:51:56Z,54.242.165.6->54.187.92.88,4491d57,baseline,cpu_used,tcp_saturate,pct,1.7,sender_host_A_1vcpu +2026-10-27T14:52:15Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:52:15Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=0pct,mbit_s,497.9,netem_0pct +2026-10-27T14:52:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:52:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=1pct,mbit_s,32.8,netem_1pct +2026-10-27T14:52:55Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T14:52:55Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=3pct,mbit_s,2.2,netem_3pct +2026-10-27T14:53:17Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=5pct,pct,1,netem_5pct +2026-10-27T14:53:17Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=5pct,mbit_s,1.6,netem_5pct +2026-10-27T14:53:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_effective,injected=10pct,pct,9,netem_10pct +2026-10-27T14:53:36Z,54.242.165.6->54.187.92.88,4491d57,baseline,loss_goodput,injected=10pct,mbit_s,0.7,netem_10pct +2026-10-27T14:53:49Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=1,mbit_s,253.7,clean +2026-10-27T14:54:02Z,54.242.165.6->54.187.92.88,4491d57,yip,tcp_throughput,streams=4,mbit_s,350.7,clean +2026-10-27T14:54:13Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:54:13Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T14:54:13Z,54.242.165.6->54.187.92.88,4491d57,yip,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:54:29Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,idle,rtt_ms,55.288,clean +2026-10-27T14:54:29Z,54.242.165.6->54.187.92.88,4491d57,yip,latency,loaded,rtt_ms,56.051,saturated +2026-10-27T14:54:36Z,54.242.165.6->54.187.92.88,4491d57,yip,cpu_used,tcp_saturate,pct,12.5,sender_host_A_1vcpu +2026-10-27T14:54:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:54:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=0pct,mbit_s,307.2,netem_0pct +2026-10-27T14:55:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:55:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=1pct,mbit_s,310.5,netem_1pct +2026-10-27T14:55:35Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T14:55:35Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=3pct,mbit_s,179.3,netem_3pct +2026-10-27T14:55:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=5pct,pct,0,netem_5pct +2026-10-27T14:55:56Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=5pct,mbit_s,232.6,netem_5pct +2026-10-27T14:56:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_effective,injected=10pct,pct,0,netem_10pct +2026-10-27T14:56:15Z,54.242.165.6->54.187.92.88,4491d57,yip,loss_goodput,injected=10pct,mbit_s,44.2,netem_10pct +2026-10-27T14:56:28Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=1,mbit_s,453.1,clean +2026-10-27T14:56:41Z,54.242.165.6->54.187.92.88,4491d57,wireguard,tcp_throughput,streams=4,mbit_s,1077.7,clean +2026-10-27T14:56:53Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:56:53Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T14:56:53Z,54.242.165.6->54.187.92.88,4491d57,wireguard,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:57:09Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,idle,rtt_ms,54.981,clean +2026-10-27T14:57:09Z,54.242.165.6->54.187.92.88,4491d57,wireguard,latency,loaded,rtt_ms,55.067,saturated +2026-10-27T14:57:16Z,54.242.165.6->54.187.92.88,4491d57,wireguard,cpu_used,tcp_saturate,pct,11.2,sender_host_A_1vcpu +2026-10-27T14:57:34Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:57:34Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=0pct,mbit_s,455.6,netem_0pct +2026-10-27T14:57:54Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:57:54Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=1pct,mbit_s,2.1,netem_1pct +2026-10-27T14:58:13Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=3pct,pct,2,netem_3pct +2026-10-27T14:58:13Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=3pct,mbit_s,1.3,netem_3pct +2026-10-27T14:58:32Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=5pct,pct,2,netem_5pct +2026-10-27T14:58:32Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=5pct,mbit_s,1.2,netem_5pct +2026-10-27T14:58:52Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_effective,injected=10pct,pct,7,netem_10pct +2026-10-27T14:58:52Z,54.242.165.6->54.187.92.88,4491d57,wireguard,loss_goodput,injected=10pct,mbit_s,0.6,netem_10pct +2026-10-27T14:59:06Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=1,mbit_s,32.6,clean +2026-10-27T14:59:19Z,54.242.165.6->54.187.92.88,4491d57,openvpn,tcp_throughput,streams=4,mbit_s,35.6,clean +2026-10-27T14:59:30Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:59:30Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_jitter,offered=200M,ms,0.045,clean +2026-10-27T14:59:30Z,54.242.165.6->54.187.92.88,4491d57,openvpn,udp_loss,offered=200M,pct,0.04,clean +2026-10-27T14:59:46Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,idle,rtt_ms,54.745,clean +2026-10-27T14:59:46Z,54.242.165.6->54.187.92.88,4491d57,openvpn,latency,loaded,rtt_ms,54.661,saturated +2026-10-27T14:59:52Z,54.242.165.6->54.187.92.88,4491d57,openvpn,cpu_used,tcp_saturate,pct,3.7,sender_host_A_1vcpu +2026-10-27T15:00:11Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:00:11Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=0pct,mbit_s,27.7,netem_0pct +2026-10-27T15:00:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T15:00:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=1pct,mbit_s,3.7,netem_1pct +2026-10-27T15:00:52Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=3pct,pct,1,netem_3pct +2026-10-27T15:00:52Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=3pct,mbit_s,1.2,netem_3pct +2026-10-27T15:01:12Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=5pct,pct,4,netem_5pct +2026-10-27T15:01:12Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=5pct,mbit_s,1.0,netem_5pct +2026-10-27T15:01:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_effective,injected=10pct,pct,8,netem_10pct +2026-10-27T15:01:33Z,54.242.165.6->54.187.92.88,4491d57,openvpn,loss_goodput,injected=10pct,mbit_s,0.6,netem_10pct diff --git a/crates/yip-bench/wan/results/history.csv b/crates/yip-bench/wan/results/history.csv index 9c7582bb..ce7f0958 100644 --- a/crates/yip-bench/wan/results/history.csv +++ b/crates/yip-bench/wan/results/history.csv @@ -1223,3 +1223,75 @@ ts,host_pair,commit,tunnel,profile,param,metric,value,conditions 2026-10-26T22:37:14Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),348fcc0,openvpn,loss_goodput,injected=5pct,mbit_s,1.0,netem_5pct 2026-10-26T22:37:35Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),348fcc0,openvpn,loss_effective,injected=10pct,pct,10,netem_10pct 2026-10-26T22:37:35Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),348fcc0,openvpn,loss_goodput,injected=10pct,mbit_s,0.5,netem_10pct +2026-10-27T14:51:10Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,tcp_throughput,streams=1,mbit_s,390.7,clean +2026-10-27T14:51:23Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,tcp_throughput,streams=4,mbit_s,906.1,clean +2026-10-27T14:51:33Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:51:33Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,udp_jitter,offered=200M,ms,0.062,clean +2026-10-27T14:51:33Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:51:49Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,latency,idle,rtt_ms,54.365,clean +2026-10-27T14:51:49Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,latency,loaded,rtt_ms,54.360,saturated +2026-10-27T14:51:56Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,cpu_used,tcp_saturate,pct,1.7,sender_host_A_1vcpu +2026-10-27T14:52:15Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:52:15Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_goodput,injected=0pct,mbit_s,497.9,netem_0pct +2026-10-27T14:52:36Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:52:36Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_goodput,injected=1pct,mbit_s,32.8,netem_1pct +2026-10-27T14:52:55Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T14:52:55Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_goodput,injected=3pct,mbit_s,2.2,netem_3pct +2026-10-27T14:53:17Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_effective,injected=5pct,pct,1,netem_5pct +2026-10-27T14:53:17Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_goodput,injected=5pct,mbit_s,1.6,netem_5pct +2026-10-27T14:53:36Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_effective,injected=10pct,pct,9,netem_10pct +2026-10-27T14:53:36Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,baseline,loss_goodput,injected=10pct,mbit_s,0.7,netem_10pct +2026-10-27T14:53:49Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,tcp_throughput,streams=1,mbit_s,253.7,clean +2026-10-27T14:54:02Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,tcp_throughput,streams=4,mbit_s,350.7,clean +2026-10-27T14:54:13Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:54:13Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T14:54:13Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:54:29Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,latency,idle,rtt_ms,55.288,clean +2026-10-27T14:54:29Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,latency,loaded,rtt_ms,56.051,saturated +2026-10-27T14:54:36Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,cpu_used,tcp_saturate,pct,12.5,sender_host_A_1vcpu +2026-10-27T14:54:56Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:54:56Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_goodput,injected=0pct,mbit_s,307.2,netem_0pct +2026-10-27T14:55:15Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:55:15Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_goodput,injected=1pct,mbit_s,310.5,netem_1pct +2026-10-27T14:55:35Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_effective,injected=3pct,pct,0,netem_3pct +2026-10-27T14:55:35Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_goodput,injected=3pct,mbit_s,179.3,netem_3pct +2026-10-27T14:55:56Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_effective,injected=5pct,pct,0,netem_5pct +2026-10-27T14:55:56Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_goodput,injected=5pct,mbit_s,232.6,netem_5pct +2026-10-27T14:56:15Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_effective,injected=10pct,pct,0,netem_10pct +2026-10-27T14:56:15Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,yip,loss_goodput,injected=10pct,mbit_s,44.2,netem_10pct +2026-10-27T14:56:28Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,tcp_throughput,streams=1,mbit_s,453.1,clean +2026-10-27T14:56:41Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,tcp_throughput,streams=4,mbit_s,1077.7,clean +2026-10-27T14:56:53Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:56:53Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,udp_jitter,offered=200M,ms,0.061,clean +2026-10-27T14:56:53Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,udp_loss,offered=200M,pct,0.00,clean +2026-10-27T14:57:09Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,latency,idle,rtt_ms,54.981,clean +2026-10-27T14:57:09Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,latency,loaded,rtt_ms,55.067,saturated +2026-10-27T14:57:16Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,cpu_used,tcp_saturate,pct,11.2,sender_host_A_1vcpu +2026-10-27T14:57:34Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T14:57:34Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_goodput,injected=0pct,mbit_s,455.6,netem_0pct +2026-10-27T14:57:54Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T14:57:54Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_goodput,injected=1pct,mbit_s,2.1,netem_1pct +2026-10-27T14:58:13Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_effective,injected=3pct,pct,2,netem_3pct +2026-10-27T14:58:13Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_goodput,injected=3pct,mbit_s,1.3,netem_3pct +2026-10-27T14:58:32Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_effective,injected=5pct,pct,2,netem_5pct +2026-10-27T14:58:32Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_goodput,injected=5pct,mbit_s,1.2,netem_5pct +2026-10-27T14:58:52Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_effective,injected=10pct,pct,7,netem_10pct +2026-10-27T14:58:52Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,wireguard,loss_goodput,injected=10pct,mbit_s,0.6,netem_10pct +2026-10-27T14:59:06Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,tcp_throughput,streams=1,mbit_s,32.6,clean +2026-10-27T14:59:19Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,tcp_throughput,streams=4,mbit_s,35.6,clean +2026-10-27T14:59:30Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,udp_throughput,offered=200M,mbit_s,200.0,clean +2026-10-27T14:59:30Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,udp_jitter,offered=200M,ms,0.045,clean +2026-10-27T14:59:30Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,udp_loss,offered=200M,pct,0.04,clean +2026-10-27T14:59:46Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,latency,idle,rtt_ms,54.745,clean +2026-10-27T14:59:46Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,latency,loaded,rtt_ms,54.661,saturated +2026-10-27T14:59:52Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,cpu_used,tcp_saturate,pct,3.7,sender_host_A_1vcpu +2026-10-27T15:00:11Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_effective,injected=0pct,pct,0,netem_0pct +2026-10-27T15:00:11Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_goodput,injected=0pct,mbit_s,27.7,netem_0pct +2026-10-27T15:00:33Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_effective,injected=1pct,pct,0,netem_1pct +2026-10-27T15:00:33Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_goodput,injected=1pct,mbit_s,3.7,netem_1pct +2026-10-27T15:00:52Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_effective,injected=3pct,pct,1,netem_3pct +2026-10-27T15:00:52Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_goodput,injected=3pct,mbit_s,1.2,netem_3pct +2026-10-27T15:01:12Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_effective,injected=5pct,pct,4,netem_5pct +2026-10-27T15:01:12Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_goodput,injected=5pct,mbit_s,1.0,netem_5pct +2026-10-27T15:01:33Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_effective,injected=10pct,pct,8,netem_10pct +2026-10-27T15:01:33Z,EC2 us-east-1 (t3.small)->EC2 us-west-2 (t3.small),4491d57,openvpn,loss_goodput,injected=10pct,mbit_s,0.6,netem_10pct diff --git a/crates/yip-bench/wan/results/readme-snippet.md b/crates/yip-bench/wan/results/readme-snippet.md index 31b77b33..40a3cd77 100644 --- a/crates/yip-bench/wan/results/readme-snippet.md +++ b/crates/yip-bench/wan/results/readme-snippet.md @@ -1,8 +1,8 @@ -*Two 1-vCPU Debian 13 VPSes, ~60.939 ms apart, 2026-10-26. yip trades raw clean-path throughput for loss resilience (see the FEC row).* +*Two 1-vCPU Debian 13 VPSes, ~54.365 ms apart, 2026-10-27. yip trades raw clean-path throughput for loss resilience (see the FEC row).* | | TCP 1-stream | UDP loss (200M) | effective loss @10% injected | |---|--:|--:|--:| -| **yip** | 274.2 Mbit/s | 0.00% | 1% | -| **WireGuard** | 451.8 Mbit/s | 0.01% | 11% | -| **OpenVPN** | 34.0 Mbit/s | 0.04% | 10% | +| **yip** | 253.7 Mbit/s | 0.00% | 0% | +| **WireGuard** | 453.1 Mbit/s | 0.00% | 7% | +| **OpenVPN** | 32.6 Mbit/s | 0.04% | 8% | diff --git a/crates/yip-io/src/af_xdp.rs b/crates/yip-io/src/af_xdp.rs index 12b7269c..412cdcbc 100644 --- a/crates/yip-io/src/af_xdp.rs +++ b/crates/yip-io/src/af_xdp.rs @@ -17,6 +17,38 @@ pub const UMEM_CHUNK_SIZE: usize = 2048; /// Default ring size in descriptors (must be a power of two). pub const UMEM_RING_SIZE: u32 = 2048; +/// Configuration parameters for allocating an AF_XDP [`UmemPool`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct UmemPoolConfig { + /// Number of chunk buffers in the pool. + pub num_chunks: usize, + /// Size of each chunk in bytes (typically 2048 or 4096). + pub chunk_size: usize, + /// Pre-fault all memory pages during allocation via `MAP_POPULATE`. + /// + /// When `true` (the default), eliminates page fault jitter during packet I/O + /// at the expense of higher startup latency and immediate RSS allocation. + /// When `false`, pages are faulted lazily on-demand, reducing startup time + /// and memory footprint on resource-constrained hosts. + pub pre_fault: bool, + /// Pin pages in physical RAM via `libc::mlock` to prevent swapping. + /// + /// Requires `CAP_IPC_LOCK` or sufficient `RLIMIT_MEMLOCK`. If `true` and `mlock` + /// fails, an error is returned. + pub lock_memory: bool, +} + +impl Default for UmemPoolConfig { + fn default() -> Self { + Self { + num_chunks: 4096, + chunk_size: 2048, + pre_fault: true, + lock_memory: false, + } + } +} + /// Pre-allocated page-aligned shared memory pool for AF_XDP kernel-bypass I/O. pub struct UmemPool { area: *mut libc::c_void, @@ -27,19 +59,46 @@ pub struct UmemPool { impl UmemPool { /// Allocates an anonymous, page-aligned shared memory region for `num_chunks` chunks - /// of size `chunk_size`. + /// of size `chunk_size` with default pre-faulting (`MAP_POPULATE`) enabled. pub fn new(num_chunks: usize, chunk_size: usize) -> io::Result { - if num_chunks == 0 || chunk_size == 0 { + Self::with_config(UmemPoolConfig { + num_chunks, + chunk_size, + pre_fault: true, + lock_memory: false, + }) + } + + /// Allocates a new `UmemPool` with lazy on-demand page allocation (`MAP_POPULATE` disabled), + /// minimizing startup latency and initial RSS footprint. + pub fn new_lazy(num_chunks: usize, chunk_size: usize) -> io::Result { + Self::with_config(UmemPoolConfig { + num_chunks, + chunk_size, + pre_fault: false, + lock_memory: false, + }) + } + + /// Allocates a new `UmemPool` according to the provided [`UmemPoolConfig`]. + pub fn with_config(config: UmemPoolConfig) -> io::Result { + if config.num_chunks == 0 || config.chunk_size == 0 { return Err(io::Error::new( io::ErrorKind::InvalidInput, "num_chunks and chunk_size must be positive", )); } - let size = num_chunks - .checked_mul(chunk_size) + let size = config + .num_chunks + .checked_mul(config.chunk_size) .ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "UMEM size overflow"))?; + let mut flags = libc::MAP_ANONYMOUS | libc::MAP_SHARED; + if config.pre_fault { + flags |= libc::MAP_POPULATE; + } + // SAFETY: We request an anonymous, shared, page-aligned memory mapping of `size` bytes. // The address is NULL so the kernel selects the starting address. // We verify that the returned pointer is not `MAP_FAILED` before proceeding. @@ -48,7 +107,7 @@ impl UmemPool { std::ptr::null_mut(), size, libc::PROT_READ | libc::PROT_WRITE, - libc::MAP_ANONYMOUS | libc::MAP_SHARED | libc::MAP_POPULATE, + flags, -1, 0, ) @@ -58,16 +117,29 @@ impl UmemPool { return Err(io::Error::last_os_error()); } + if config.lock_memory { + // SAFETY: `area` points to `size` bytes of freshly mapped valid memory. + let lock_ret = unsafe { libc::mlock(area, size) }; + if lock_ret != 0 { + let err = io::Error::last_os_error(); + // SAFETY: On mlock failure, unmap the allocated area before returning error. + unsafe { + libc::munmap(area, size); + } + return Err(err); + } + } + // Populates free_chunks so popping yields 0, chunk_size, 2*chunk_size, ... - let free_chunks: Vec = (0..num_chunks) + let free_chunks: Vec = (0..config.num_chunks) .rev() - .map(|i| (i * chunk_size) as u64) + .map(|i| (i * config.chunk_size) as u64) .collect(); Ok(Self { area, size, - chunk_size, + chunk_size: config.chunk_size, free_chunks, }) } @@ -241,13 +313,29 @@ impl UmemPool { Ok(slabs) } - /// Allocates a new `UmemPool` and partitions it into `num_slabs` disjoint slabs. + /// Allocates a new `UmemPool` with default configuration and partitions it into `num_slabs` disjoint slabs. pub fn new_slabs( num_chunks: usize, chunk_size: usize, num_slabs: usize, ) -> io::Result<(Arc, Vec)> { - let pool = Arc::new(Self::new(num_chunks, chunk_size)?); + Self::new_slabs_with_config( + UmemPoolConfig { + num_chunks, + chunk_size, + pre_fault: true, + lock_memory: false, + }, + num_slabs, + ) + } + + /// Allocates a new `UmemPool` using the specified [`UmemPoolConfig`] and partitions it into `num_slabs` disjoint slabs. + pub fn new_slabs_with_config( + config: UmemPoolConfig, + num_slabs: usize, + ) -> io::Result<(Arc, Vec)> { + let pool = Arc::new(Self::with_config(config)?); let slabs = pool.create_slabs(num_slabs)?; Ok((pool, slabs)) } diff --git a/crates/yip-io/src/batch.rs b/crates/yip-io/src/batch.rs index 879fb594..b7816eaa 100644 --- a/crates/yip-io/src/batch.rs +++ b/crates/yip-io/src/batch.rs @@ -10,9 +10,15 @@ use std::os::fd::{AsRawFd, RawFd}; use crate::MAX_WIRE_DATAGRAM; -/// Maximum number of datagrams transferred in a single batched syscall. +/// Maximum number of datagrams transferred in a single default batched syscall. pub const BATCH_SIZE: usize = 32; +/// Expanded 64-datagram burst size for high-packet-rate 10G/25G interfaces. +pub const BATCH_SIZE_64: usize = 64; + +/// Ultra-high 128-datagram burst size for maximum syscall amortization at 40G+ line rate. +pub const BATCH_SIZE_128: usize = 128; + /// Metadata for a single datagram received via [`BatchUdpSocket::recvmmsg_batch`]. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct ReceivedDatagram { @@ -67,7 +73,7 @@ impl BatchUdpSocket { self.fd } - /// Receive a burst of up to [`BATCH_SIZE`] datagrams into `buffers` via `libc::recvmmsg`. + /// Receive a burst of up to `N` datagrams into `buffers` via `libc::recvmmsg`. /// /// For each received datagram: /// - The datagram payload bytes are written into `buffers[i]`. @@ -79,26 +85,30 @@ impl BatchUdpSocket { /// /// Returns an [`io::Error`] if the underlying `recvmmsg` syscall fails with any error /// other than `EAGAIN` or `EWOULDBLOCK`. - pub fn recvmmsg_batch( + pub fn recvmmsg_batch( &mut self, - buffers: &mut [[u8; MAX_WIRE_DATAGRAM]; BATCH_SIZE], - out: &mut [ReceivedDatagram; BATCH_SIZE], + buffers: &mut [[u8; MAX_WIRE_DATAGRAM]; N], + out: &mut [ReceivedDatagram; N], ) -> io::Result { - let mut storages: [libc::sockaddr_storage; BATCH_SIZE] = unsafe { + if N == 0 { + return Ok(0); + } + + let mut storages: [libc::sockaddr_storage; N] = unsafe { // SAFETY: `sockaddr_storage` is a plain-old-data struct consisting of integer // and byte array fields. The all-zero bit pattern is a valid initial state that // `libc::recvmmsg` overwrites upon receipt. std::mem::zeroed() }; let addrlens = [libc::socklen_t::try_from(std::mem::size_of::()) - .expect("sockaddr_storage size fits socklen_t"); BATCH_SIZE]; + .expect("sockaddr_storage size fits socklen_t"); N]; let mut iovecs = [libc::iovec { iov_base: std::ptr::null_mut(), iov_len: 0, - }; BATCH_SIZE]; - let mut msgs = [crate::empty_mmsghdr(); BATCH_SIZE]; + }; N]; + let mut msgs = [crate::empty_mmsghdr(); N]; - for i in 0..BATCH_SIZE { + for i in 0..N { iovecs[i].iov_base = buffers[i].as_mut_ptr().cast::(); iovecs[i].iov_len = MAX_WIRE_DATAGRAM; msgs[i].msg_hdr.msg_iov = &raw mut iovecs[i]; @@ -118,7 +128,7 @@ impl BatchUdpSocket { libc::recvmmsg( self.fd, msgs.as_mut_ptr(), - u32::try_from(BATCH_SIZE).expect("BATCH_SIZE fits u32"), + u32::try_from(N).expect("N fits u32"), libc::MSG_DONTWAIT as crate::MsgFlags, std::ptr::null_mut(), ) @@ -137,10 +147,7 @@ impl BatchUdpSocket { } let received = usize::try_from(ret).expect("recvmmsg return fits usize"); - assert!( - received <= BATCH_SIZE, - "recvmmsg returned count exceeding BATCH_SIZE" - ); + assert!(received <= N, "recvmmsg returned count exceeding N"); for i in 0..received { let len = usize::try_from(msgs[i].msg_len).expect("msg_len fits usize"); @@ -155,7 +162,7 @@ impl BatchUdpSocket { } } - /// Send a burst of up to [`BATCH_SIZE`] datagrams in a single `libc::sendmmsg` syscall. + /// Send a burst of up to [`BATCH_SIZE`] (32) datagrams in a single `libc::sendmmsg` syscall. /// /// If `packets.len() > BATCH_SIZE`, only the first [`BATCH_SIZE`] packets are submitted. /// Returns the number of datagrams accepted by the kernel (0 if the socket would block). @@ -164,22 +171,37 @@ impl BatchUdpSocket { /// /// Returns an [`io::Error`] if the underlying `sendmmsg` syscall fails with an unrecoverable error. pub fn sendmmsg_batch(&mut self, packets: &[(&[u8], SocketAddr)]) -> io::Result { - let count = packets.len().min(BATCH_SIZE); + self.sendmmsg_burst::(packets) + } + + /// Send a burst of up to `N` datagrams in a single `libc::sendmmsg` syscall. + /// + /// If `packets.len() > N`, only the first `N` packets are submitted. + /// Returns the number of datagrams accepted by the kernel (0 if the socket would block). + /// + /// # Errors + /// + /// Returns an [`io::Error`] if the underlying `sendmmsg` syscall fails with an unrecoverable error. + pub fn sendmmsg_burst( + &mut self, + packets: &[(&[u8], SocketAddr)], + ) -> io::Result { + let count = packets.len().min(N); if count == 0 { return Ok(0); } - let mut storages: [libc::sockaddr_storage; BATCH_SIZE] = unsafe { + let mut storages: [libc::sockaddr_storage; N] = unsafe { // SAFETY: `sockaddr_storage` is plain-old-data. An all-zero value is a valid // initial state that is overwritten for the first `count` items below. std::mem::zeroed() }; - let mut addrlens = [0 as libc::socklen_t; BATCH_SIZE]; + let mut addrlens = [0 as libc::socklen_t; N]; let mut iovecs = [libc::iovec { iov_base: std::ptr::null_mut(), iov_len: 0, - }; BATCH_SIZE]; - let mut msgs = [crate::empty_mmsghdr(); BATCH_SIZE]; + }; N]; + let mut msgs = [crate::empty_mmsghdr(); N]; for (i, &(payload, dst)) in packets[..count].iter().enumerate() { let (storage, addr_len) = crate::addr::std_to_sockaddr(dst); diff --git a/crates/yip-io/tests/batch_io_test.rs b/crates/yip-io/tests/batch_io_test.rs index 921a07a8..006067e3 100644 --- a/crates/yip-io/tests/batch_io_test.rs +++ b/crates/yip-io/tests/batch_io_test.rs @@ -159,3 +159,109 @@ fn test_udp_gso_segment_and_superpacket() { let res = yip_io::batch::send_gso_superpacket(&s1, &payload, 1420, addr2); let _ = res; } + +#[test] +fn test_batch_size_64_burst() { + use yip_io::batch::BATCH_SIZE_64; + + let s1 = UdpSocket::bind("127.0.0.1:0").expect("bind s1"); + let s2 = UdpSocket::bind("127.0.0.1:0").expect("bind s2"); + s1.set_nonblocking(true).unwrap(); + s2.set_nonblocking(true).unwrap(); + + let addr1 = s1.local_addr().unwrap(); + let addr2 = s2.local_addr().unwrap(); + let mut b1 = BatchUdpSocket::new(&s1); + let mut b2 = BatchUdpSocket::new(&s2); + + let mut payloads = Vec::with_capacity(BATCH_SIZE_64); + let mut to_send = Vec::with_capacity(BATCH_SIZE_64); + for i in 0..BATCH_SIZE_64 { + payloads.push(format!("payload-64-burst-{i}").into_bytes()); + } + for p in &payloads { + to_send.push((p.as_slice(), addr2)); + } + + let sent = b1 + .sendmmsg_burst::(&to_send) + .expect("sendmmsg 64"); + assert_eq!(sent, BATCH_SIZE_64); + + std::thread::sleep(std::time::Duration::from_millis(25)); + + let mut buffers = [[0u8; yip_io::MAX_WIRE_DATAGRAM]; BATCH_SIZE_64]; + let mut out = [const { yip_io::batch::ReceivedDatagram::empty() }; BATCH_SIZE_64]; + + let mut total_recvd = 0; + while total_recvd < BATCH_SIZE_64 { + let recvd = b2 + .recvmmsg_batch(&mut buffers, &mut out) + .expect("recvmmsg 64"); + if recvd == 0 { + std::thread::sleep(std::time::Duration::from_millis(10)); + continue; + } + for i in 0..recvd { + let idx = total_recvd + i; + assert_eq!(out[i].len, payloads[idx].len()); + assert_eq!(&buffers[i][..out[i].len], payloads[idx].as_slice()); + assert_eq!(out[i].src, addr1); + } + total_recvd += recvd; + } + assert_eq!(total_recvd, BATCH_SIZE_64); +} + +#[test] +fn test_batch_size_128_burst() { + use yip_io::batch::BATCH_SIZE_128; + + let s1 = UdpSocket::bind("127.0.0.1:0").expect("bind s1"); + let s2 = UdpSocket::bind("127.0.0.1:0").expect("bind s2"); + s1.set_nonblocking(true).unwrap(); + s2.set_nonblocking(true).unwrap(); + + let addr1 = s1.local_addr().unwrap(); + let addr2 = s2.local_addr().unwrap(); + let mut b1 = BatchUdpSocket::new(&s1); + let mut b2 = BatchUdpSocket::new(&s2); + + let mut payloads = Vec::with_capacity(BATCH_SIZE_128); + let mut to_send = Vec::with_capacity(BATCH_SIZE_128); + for i in 0..BATCH_SIZE_128 { + payloads.push(format!("payload-128-burst-{i}").into_bytes()); + } + for p in &payloads { + to_send.push((p.as_slice(), addr2)); + } + + let sent = b1 + .sendmmsg_burst::(&to_send) + .expect("sendmmsg 128"); + assert_eq!(sent, BATCH_SIZE_128); + + std::thread::sleep(std::time::Duration::from_millis(30)); + + let mut buffers = [[0u8; yip_io::MAX_WIRE_DATAGRAM]; BATCH_SIZE_128]; + let mut out = [const { yip_io::batch::ReceivedDatagram::empty() }; BATCH_SIZE_128]; + + let mut total_recvd = 0; + while total_recvd < BATCH_SIZE_128 { + let recvd = b2 + .recvmmsg_batch(&mut buffers, &mut out) + .expect("recvmmsg 128"); + if recvd == 0 { + std::thread::sleep(std::time::Duration::from_millis(10)); + continue; + } + for i in 0..recvd { + let idx = total_recvd + i; + assert_eq!(out[i].len, payloads[idx].len()); + assert_eq!(&buffers[i][..out[i].len], payloads[idx].as_slice()); + assert_eq!(out[i].src, addr1); + } + total_recvd += recvd; + } + assert_eq!(total_recvd, BATCH_SIZE_128); +} diff --git a/crates/yip-io/tests/umem_test.rs b/crates/yip-io/tests/umem_test.rs index 029d9509..7c3dd47f 100644 --- a/crates/yip-io/tests/umem_test.rs +++ b/crates/yip-io/tests/umem_test.rs @@ -1,7 +1,62 @@ use yip_io::af_xdp::{ - CompletionRing, FillRing, UmemPool, UmemSlab, UMEM_CHUNK_SIZE, UMEM_RING_SIZE, + CompletionRing, FillRing, UmemPool, UmemPoolConfig, UmemSlab, UMEM_CHUNK_SIZE, UMEM_RING_SIZE, }; +#[test] +fn test_umem_pool_lazy_allocation() { + let mut pool = UmemPool::new_lazy(64, UMEM_CHUNK_SIZE).expect("allocate lazy UMEM pool"); + assert_eq!(pool.free_chunk_count(), 64); + assert_eq!(pool.size(), 64 * UMEM_CHUNK_SIZE); + + let c1 = pool.alloc_chunk().expect("alloc lazy chunk 1"); + let c2 = pool.alloc_chunk().expect("alloc lazy chunk 2"); + assert_ne!(c1, c2); + + // Fault in page on demand via memory write + { + let slice = pool.chunk_slice_mut(c1, 256); + slice[0] = 0xAA; + slice[255] = 0x55; + } + + let read_slice = pool.chunk_slice(c1, 256); + assert_eq!(read_slice[0], 0xAA); + assert_eq!(read_slice[255], 0x55); + + pool.free_chunk(c1); + pool.free_chunk(c2); + assert_eq!(pool.free_chunk_count(), 64); +} + +#[test] +fn test_umem_pool_with_config_and_slabs() { + let config = UmemPoolConfig { + num_chunks: 32, + chunk_size: UMEM_CHUNK_SIZE, + pre_fault: false, + lock_memory: false, + }; + let (_pool, mut slabs) = + UmemPool::new_slabs_with_config(config, 4).expect("new_slabs_with_config"); + assert_eq!(slabs.len(), 4); + + for (i, slab) in slabs.iter_mut().enumerate() { + assert_eq!(slab.slab_id(), i); + assert_eq!(slab.num_chunks(), 8); + assert_eq!(slab.free_chunk_count(), 8); + + let chunk = slab.alloc_chunk().expect("alloc from slab"); + assert!(slab.contains_addr(chunk)); + + let slice = slab.chunk_slice_mut(chunk, 128); + slice[0] = (i + 1) as u8; + assert_eq!(slab.chunk_slice(chunk, 128)[0], (i + 1) as u8); + + slab.free_chunk(chunk); + assert_eq!(slab.free_chunk_count(), 8); + } +} + #[test] fn test_umem_pool_allocation_and_slicing() { let mut pool = UmemPool::new(64, UMEM_CHUNK_SIZE).expect("allocate UMEM pool"); diff --git a/docs/03-architecture.md b/docs/03-architecture.md index 56ba257e..d4bd492c 100644 --- a/docs/03-architecture.md +++ b/docs/03-architecture.md @@ -421,7 +421,7 @@ On multi-socket NUMA hardware (e.g., dual AMD EPYC or Intel Xeon systems), cross The standard vectorized engine ([`crates/yip-io/src/batch.rs`](../crates/yip-io/src/batch.rs)) runs on any generic Linux distribution without hardware-specific requirements: -1. **Syscall Amortization**: Collects bursts of up to `BATCH_SIZE = 32` datagrams and transfers them in a single `recvmmsg(2)` or `sendmmsg(2)` invocation, amortizing CPU privilege boundary transitions. +1. **Syscall Amortization**: Collects bursts of datagrams (default `BATCH_SIZE = 32`, expandable to `BATCH_SIZE_64 = 64` and `BATCH_SIZE_128 = 128` via const-generic `sendmmsg_burst` and `recvmmsg_batch`) and transfers them in a single `recvmmsg(2)` or `sendmmsg(2)` invocation, amortizing CPU privilege boundary transitions. 2. **Fate-Safe Generic Segmentation Offload (GSO)**: Uses the Linux `UDP_SEGMENT` socket control message to hand the kernel aggregated super-buffers up to 64 KB. The kernel or NIC hardware splits the buffer into physical MTU-sized segments. #### Fate-Safety Rule @@ -463,8 +463,9 @@ The AF_XDP engine ([`crates/yip-io/src/af_xdp.rs`](../crates/yip-io/src/af_xdp.r #### Shared Memory UMEM Pool Architecture Packet memory is managed in a pre-allocated, page-aligned shared memory pool (`UmemPool`): -- Allocated via `libc::mmap` with `MAP_ANONYMOUS | MAP_SHARED | MAP_POPULATE`. +- Allocated via `libc::mmap` with `MAP_ANONYMOUS | MAP_SHARED`. - Chunk size is 2,048 bytes (`UMEM_CHUNK_SIZE`), aligning with standard MTU envelopes and cache lines. +- **Configurable Page Faulting (`UmemPoolConfig`)**: Supports both zero-tail-latency physical page pre-faulting (`pre_fault: true` with `MAP_POPULATE`) and instant on-demand lazy paging (`pre_fault: false` via `UmemPool::new_lazy`, cutting pool initialization time from ~1,590 µs to ~7 µs, a 219x reduction). Optional `lock_memory: true` calls `libc::mlock` to pin pages in physical RAM and prevent paging under memory pressure. - **Per-Core `UmemSlab` Partitioning**: To eliminate centralized free-list serialization and cross-core cache invalidation under multi-worker scaling, `UmemPool` partitions chunks into disjoint per-core `UmemSlab`s (`UmemPool::new_slabs`). Each worker thread allocates and frees chunks locally from its dedicated slab without atomic locks or cross-core coordination. - **Lock-Free Batch Ring Recycling**: Completed TX chunk descriptors are recycled directly from the `CompletionRing` into the `FillRing` via `slab.recycle_from_rings`, spilling surplus chunks to the thread-local slab without heap allocations. - **Vectorized Ring Operations**: Circular descriptor rings (`FillRing`, `CompletionRing`, `RxRing`, `TxRing`) execute batch produce and consume using power-of-two masking and contiguous slice copies, sustaining over 4,000 Mops/s. diff --git a/docs/05-benchmarking-and-performance.md b/docs/05-benchmarking-and-performance.md index d12c67b2..159f1c3b 100644 --- a/docs/05-benchmarking-and-performance.md +++ b/docs/05-benchmarking-and-performance.md @@ -264,7 +264,8 @@ The Dedicated Multi-Core Server testbed isolates raw datapath throughput and eva - **Kernel-Bypass Subsystem:** Linux kernel 6.12 with AF_XDP (XSK) driver: - eBPF driver running in `XDP_DRV` native mode, steering matched UDP tunnel datagrams directly to AF_XDP UMEM queues via `bpf_redirect_map`. - 8 pinned worker threads running unprivileged event loops with core isolation (`taskset -c 0-7`). - - Page-aligned `UmemPool` chunk allocator with power-of-two circular descriptor queues (`FillRing`, `RxRing`, `TxRing`, `CompletionRing`). + - Page-aligned `UmemPool` chunk allocator with configurable pre-faulting (`MAP_POPULATE`, 1,590 µs) versus lazy on-demand allocation (7.25 µs, **219.5x faster startup**), power-of-two circular descriptor queues (`FillRing`, `RxRing`, `TxRing`, `CompletionRing`), and per-core `UmemSlab` partitioning. + - Tunable `BatchUdpSocket` syscall burst sizing (32, 64, and 128 datagrams per syscall) amortizing kernel transitions under high packet rates. - Zero-copy in-place AEAD sealing (`seal_in_place` and `open_in_place_with_window`) inside pinned UMEM chunks. - **Concurrency & Workload:** 64 concurrent TCP streams generated by hardware packet blast engines (`cargo bench --bench af_xdp_scale`), pushing 60,000 1280-byte IPv4 packets per core. - **Empirical Results:**