Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,15 @@ until 0.1.0.
- **Time-based anti-replay hardening & clock epoch initialization (#103, #104).**
Introduced `tai64n_sub_duration`, rejected pre-boot replayed Inits on cold start, initialized rendezvous sequence numbers with wall-clock epoch ms, and rejected post-TTL `RegisterSigned` records.

### Performance
- **Zero-allocation FEC reassembler hot path, in-place RS repair & flat `ObjectTable` (`crates/yip-transport`, `crates/yip-bench`, #171, PR #176).**
Eliminated all per-packet heap allocations, `HashMap` lookups, and intermediate `Vec<Vec<u8>>` cloning from `FecReassembler::push()`. Pre-allocated contiguous object buffers with a 256-bit bitset (`seen_mask: [u64; 4]`), delivering incoming systematic source shards directly into pre-allocated memory slices (`payload[idx * shard_len ..]`) and extracting completed objects via `std::mem::take` and `truncate` with zero dynamic allocations. Introduced in-place Reed-Solomon packet loss repair (`reconstruct_missing_sources` & `decode_source_into` in `crates/yip-transport/src/rs.rs`), solving missing source shards directly into the pre-allocated payload buffer using a single scratch slice and eliminating the `Vec<Vec<u8>>` heap matrix. Replaced standard library `HashMap<u64, ObjState>` with `ObjectTable`, a flat contiguous open-addressing table featuring SplitMix64 fast hashing and backward-shift deletion with zero tombstones and zero runtime allocations during packet ingress and eviction. Added Criterion microbenchmark suite in `crates/yip-bench/benches/fec_reassemble.rs`, verifying a 3.1x speedup on clean $K=1$ packets (117.11 ns vs 361.98 ns, -67.6% latency), 4.0x speedup on clean $K=3$ objects (173.12 ns vs 690.73 ns, -74.9% latency), and 1.65x speedup on loss recovery under $K=3, R=1$ (631.83 ns vs 1.0401 µs, -39.3% latency).

### Added
- **FEC reassembly Criterion microbenchmark suite (`crates/yip-bench/benches/fec_reassemble.rs`, #171, PR #176).**
Added isolated microbenchmark harnesses measuring clean-path reassembly latency ($K=1$ sub-symbol MTU-trimmed packets and $K=3$ multi-symbol objects) and packet-loss Reed-Solomon reconstruction latency ($K=3, R=1$).
- **Comprehensive documentation landing page & modular curriculum overhaul (`docs/`, `README.md`, #170, PR #175).**
Overhauled the project documentation and root `README.md` to provide an accessible beginner onboarding pathway alongside deep technical specifications. Restored the comprehensive "Why yip Exists: The Problem with Traditional VPNs" motivation narrative explaining Mathis' formula for TCP throughput collapse under packet loss ($T \le \frac{MSS}{RTT} \times \frac{C}{\sqrt{p}}$) and yip's proactive in-flight Galois Field erasure repair. Added structured "When to Use yip / When Not to Use" guidelines, simplified 5-minute quickstarts, and audience-specific navigation paths (Operators, Developers, Researchers, Security Reviewers). Restructured technical internals into 6 standalone numbered guides in `docs/` (`01-getting-started.md`, `02-configuration.md`, `03-architecture.md`, `04-security-and-threat-model.md`, `05-benchmarking-and-performance.md`, `06-troubleshooting.md`, and `motivation.md`).
- **Zero-friction distribution & packaging suite (`contrib/`, `Dockerfile`, `install.sh`, `Cargo.toml`, `.github/workflows/release.yml`, #147).**
Added standalone, statically linked musl distribution binaries (`x86_64-unknown-linux-musl` and `aarch64-unknown-linux-musl`) with zero runtime dynamic dependencies. Built a multi-stage production Dockerfile supporting both single-binary target builds (`yipd`, `yip-rendezvous`) with minimal alpine/musl layers. Implemented a turnkey POSIX-compliant 1-line installer (`install.sh`) featuring architecture detection (`x86_64`, `aarch64`), cryptographic SHA-256 checksum verification against official release manifests, non-root `sudo`/`doas` privilege handling, safe fallback directory overrides, and automated init service detection and configuration. Added a production-grade multi-init service suite in `contrib/` providing hardened systemd units (`yipd.service` and `yip-rendezvous.service`) with `LimitMEMLOCK=infinity`, `CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_RAW CAP_BPF CAP_IPC_LOCK`, `DeviceAllow=/dev/net/tun rw`, and strict sandboxing (`ProtectSystem=strict`, `ProtectHome=true`, `PrivateTmp=true`), alongside OpenRC service scripts (`contrib/openrc/`) and runit service directories (`contrib/runit/`). Configured `cargo-binstall` metadata in binary package manifests (`bin/*/Cargo.toml`) for direct binary installation from GitHub Releases. Created multi-arch GitHub Actions and Forgejo automated release pipelines (`.github/workflows/release.yml`, `.forgejo/workflows/release.yml`) cross-compiling static binaries, assembling tarballs, computing detached `SHA256SUMS`, and publishing artifacts on semver git tags.
- **Traffic-analysis defense configuration schema, micro-jitter queue & DPI validation (`bin/yipd`, `crates/yip-obf`, #11).**
Expand Down Expand Up @@ -347,6 +355,8 @@ until 0.1.0.
failed deterministically on stock-configured machines.

### Changed
- **Sanitized cryptographic key examples and standardized default port 443 (#170, PR #175).**
Replaced all repetitive dummy key examples (`11111111...`, `22222222...`, `33333333...`) across all configuration files, markdown guides, and CLI references with cryptographically realistic high-entropy hex sequences and explicit zero-trust `yip-ca` certificate workflows. Standardized the default daemon listen port to UDP 443 across all configurations, scripts, and documentation, ensuring out-of-the-box traffic blending with HTTPS/QUIC and evading middlebox port filters targeting standard VPN ports (such as WireGuard's 51820), with automated unprivileged fallback to port 8443. Added live animated terminal telemetry dashboard GIF (`assets/terminal-telemetry.gif`) demonstrating real-time multi-transport mesh status and peer statistics. Recorded best-of-3 automated AWS multi-region WAN benchmarks between `us-east-1` and `us-west-2` under 0% to 10% packet loss.
- Rendezvous relay-forward counter moved off the server's mutex onto a
shared `AtomicU64` (#68, PR #93) — a pure perf fix (removes lock
contention on the hot forwarding path under load); no behavior change to
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ Because throughput is inversely proportional to the square root of packet loss p
- **Systematic Cauchy RS + RLNC Streaming**: In-flight erasure coding over $GF(2^8)$ pairs Cauchy Reed-Solomon block codes with Random Linear Network Coding (RLNC) streaming fountain codes.
- **Poly-Vector SIMD Acceleration**: Hardware-accelerated matrix multiplication across AVX-512BW, Intel GFNI, AVX2, SSSE3, ARM NEON, and WebAssembly SIMD128.
- **Adaptive Repair Scaling**: Real-time sliding window telemetry dynamically adjusts repair symbol overhead based on observed channel loss, eliminating goodput waste on clean links.
- **Zero-Allocation Hot-Path Reassembly**: Pre-allocated contiguous object buffers and in-place Cauchy RS repair (`reconstruct_missing_sources`) deliver clean packets in **117 ns** with zero heap allocations.

### 2. Zero-Trust PKI & Decentralized Mesh Membership
- **Cryptographic Authority (`yip-ca`)**: Offline or operator-governed root-of-trust issuing signed node membership certificates (`Cert`).
Expand All @@ -70,6 +71,7 @@ Because throughput is inversely proportional to the square root of packet loss p
- **Asynchronous `io_uring`**: High-performance multishot ring polling drives high packet rates without thread contention or syscall overhead.
- **Multi-Queue TUN Sharding**: `IFF_MULTI_QUEUE` worker isolation maps packet flows to dedicated CPU cores via lock-free SPSC ring buffers.
- **Vectorized Socket I/O**: Batched `recvmmsg` and `sendmmsg` with `UDP_SEGMENT` Generic Segmentation Offload (GSO) optimize standard POSIX network paths.
- **Flat Open-Addressing Tables**: Replaces dynamic pointer maps with bounded flat slot tables using SplitMix64 fast hashing and backward-shift deletion with zero tombstones.

### 4. Post-Quantum Cryptography & Identity
- **Single-RTT Hybrid Handshake**: Blends Noise_IKpsk2 (Curve25519) with NIST FIPS 203 ML-KEM-512 (Kyber) within compact datagrams strictly under the 1200-byte internet MTU floor.
Expand Down
2 changes: 2 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ maintainer.
- **Data-plane windowed loss, 8-contender harness & bench polish (#11)** — exact per-class windowed loss fractions ($L_{\text{class}} = M_{\text{class}} / S_{\text{class}}$ in $(H_{\text{prev}}..H_{\text{curr}}]$), 4096-entry synchronized loss detection window matching sent-log capacity, bounded spurious NACK drop invariants, 8-contender comparison matrix and netns runners (yip, WireGuard, AmneziaWG, OpenVPN, n2n, ZeroTier, Tailscale, NetBird), dual TCP & UDP-under-loss evaluation, and configurable 1000-ping variance tightening.
- **Traffic-analysis defense (DAITA & AmneziaWG recipes, #11 / Sub-project #4)** — DAITA discrete packet size bucketing (`round_to_bucket`), stochastic decoy bursts with zero-allocation wire generation (`obf_scratch`), bounded egress micro-jitter queue (up to 1,024 packets with flush-oldest overrun protection), and event-loop poll timeout clamping (`next_scheduled_egress_delay_ms`).
- **Zero-friction distribution & packaging (#147)** — standalone, statically linked musl binaries (`x86_64-unknown-linux-musl` and `aarch64-unknown-linux-musl`) with zero dynamic runtime dependencies, multi-arch GitHub Actions and Forgejo automated release pipelines generating tarballs with detached `SHA256SUMS`, production multi-stage Dockerfile (`ghcr.io/femboyisp/yipd` and `yip-rendezvous`), turnkey POSIX `install.sh` with architecture detection, checksum verification, and init auto-detection, hardened multi-init service suite in `contrib/` (systemd with `LimitMEMLOCK=infinity`, capability bounding, and strict sandboxing; OpenRC; and runit), and `cargo-binstall` package metadata integration.
- **Zero-allocation FEC reassembler hot path (#171, PR #176)** — eliminated per-packet heap allocations, `HashMap` lookups, and intermediate `Vec<Vec<u8>>` cloning in `FecReassembler::push()`. Pre-allocated contiguous object buffers with a 256-bit bitset (`seen_mask: [u64; 4]`), zero-alloc in-place systematic delivery via buffer slicing (`mem::take` & `truncate`), in-place Cauchy Reed-Solomon packet loss repair (`reconstruct_missing_sources` & `decode_source_into`) avoiding `Vec<Vec<u8>>` heap matrix allocations, and bounded flat open-addressing slot table (`ObjectTable`) with SplitMix64 fast hashing and backward-shift deletion with zero tombstones. Achieved 3.1x faster reassembly on clean $K=1$ (117 ns), 4.0x on clean $K=3$ (173 ns), and 1.65x on packet loss recovery (632 ns).
- **Comprehensive documentation overhaul & landing page architecture (#170, PR #175)** — transformed repository landing page into an accessible, structured guide with clear "What is yip?", "When to use / when not to use", simplified quickstart, audience-specific pathways, terminal telemetry demo GIF, sanitized production-grade zero-trust CA key management, standardized default port 443 (with 8443 unprivileged fallback), best-of-3 AWS WAN benchmark measurements, and 6 standalone guides in `docs/` (`01-getting-started.md`, `02-configuration.md`, `03-architecture.md`, `04-security-and-threat-model.md`, `05-benchmarking-and-performance.md`, `06-troubleshooting.md`, plus `motivation.md`).

## Now

Expand Down
4 changes: 4 additions & 0 deletions crates/yip-bench/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,9 @@ harness = false
name = "rlnc_bench"
harness = false

[[bench]]
name = "fec_reassemble"
harness = false

[lints]
workspace = true
65 changes: 65 additions & 0 deletions crates/yip-bench/benches/fec_reassemble.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;
use yip_transport::{FecEncoder, FecReassembler, FlowClass};

fn bench_fec_reassemble(c: &mut Criterion) {
let mut enc = FecEncoder::new();

// 1. Clean-path K=1 (typical WAN / internet MTU packet: 1200 bytes, 1 shard)
let k1_data = vec![0x42u8; 1200];
let k1_params = FlowClass::Default.params();
let k1_syms = enc.encode(&k1_data, k1_params, 0);
assert_eq!(k1_syms.len(), 1);

c.bench_function("fec_reassemble_clean_k1", |bn| {
let mut re = FecReassembler::new(k1_params.symbol_size, 256);
let mut key: u64 = 1;
bn.iter(|| {
let res = re.push(black_box(key), black_box(&k1_syms[0]), 1000, 5000);
key = key.wrapping_add(1);
black_box(res)
})
});

// 2. Clean-path K=3 (3600 bytes, 3 source shards, 0 loss)
let k3_data = vec![0x42u8; 3600];
let k3_params = FlowClass::Bulk.params();
let k3_syms = enc.encode(&k3_data, k3_params, 0);
assert_eq!(k3_syms.len(), 3);

c.bench_function("fec_reassemble_clean_k3", |bn| {
let mut re = FecReassembler::new(k3_params.symbol_size, 256);
let mut key: u64 = 1;
bn.iter(|| {
let cur_key = key;
key = key.wrapping_add(1);
let _s0 = re.push(cur_key, &k3_syms[0], 1000, 5000);
let _s1 = re.push(cur_key, &k3_syms[1], 1000, 5000);
let res = re.push(cur_key, &k3_syms[2], 1000, 5000);
black_box(res)
})
});

// 3. Loss recovery K=3, R=1 (shard 1 missing, recovered via repair shard 3)
let k3_r1_syms = enc.encode(&k3_data, k3_params, 1);
assert_eq!(k3_r1_syms.len(), 4);
let s0 = &k3_r1_syms[0];
let s2 = &k3_r1_syms[2];
let repair3 = &k3_r1_syms[3];

c.bench_function("fec_reassemble_loss_k3_r1", |bn| {
let mut re = FecReassembler::new(k3_params.symbol_size, 256);
let mut key: u64 = 1;
bn.iter(|| {
let cur_key = key;
key = key.wrapping_add(1);
let _s0 = re.push(cur_key, s0, 1000, 5000);
let _s2 = re.push(cur_key, s2, 1000, 5000);
let res = re.push(cur_key, repair3, 1000, 5000);
black_box(res)
})
});
}

criterion_group!(benches, bench_fec_reassemble);
criterion_main!(benches);
Loading
Loading