diff --git a/README.md b/README.md index 2e63628..476c265 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ A Linux UDP datagram engine in C++26: batched send and receive, all memory carved once at startup, and a protocol boundary that hands out nothing but `std::span` and timing. -Built on `recvmmsg` / `sendmmsg`, ancillary data (`IP_PKTINFO`, ECN), hardware +Built on `recvmmsg` / `sendmmsg`, ancillary data (`IP_PKTINFO`, DSCP and ECN), hardware segmentation offload (`UDP_GRO` / `UDP_SEGMENT`) and transmit pacing (`SO_TXTIME`). It is not a general network library and has no TCP: the whole design turns on a batch being reclaimable in one step, which a byte stream is @@ -65,7 +65,7 @@ compiled and run as a test, so none of them can rot. | `receive_batch` | `recvmmsg` argument block carved from a resource, plus a lazy view over what arrived. | | `transmit_batch` | `sendmmsg` argument block. Stages by reference, so echoing costs no copy. | | `features` | Compile-time set of ancillary-data features; the control buffer is sized as the sum over it. | -| `pktinfo` / `ecn` | Which local address a datagram arrived on, and its ECN marking. Both families. | +| `pktinfo` / `ecn` / `traffic_class` | Which local address a datagram arrived on, and its ECN and DSCP marking. Both families. | | `gro` / `segment` | Hardware segmentation offload: many datagrams per slot in, one large buffer out. | | `txtime` / `pacer` | Per-datagram departure times via `SO_TXTIME`, and drift-free rate arithmetic. | | `flow_table` / `route` | Open-addressing demultiplexing to protocol state, keyed by a caller-supplied projection and hashed under a per-table SipHash seed, so peers cannot flood it. | @@ -104,10 +104,11 @@ for (const auto& d : batch->datagrams() | std::views::filter(dgram::is_intact)) control buffer cannot fall out of step with what the parser expects. ```cpp -using metadata_set = dgram::features; +using rx_set = dgram::features; +using tx_set = dgram::features; const auto meta{d.meta()}; -if (const auto marking{meta.get()}) { reply.set(*marking); } +if (const auto marking{meta.get()}) { reply.set({.ecn = *marking}); } ``` ## Building @@ -162,7 +163,7 @@ const auto counts{dgram::route(batch, dgram::by_payload_id<1, 8>{}, flows, now)} ## Status All five phases are done and tested: the memory model, the batching syscalls, -ancillary data (destination address and ECN, both families), segmentation +ancillary data (destination address, DSCP and ECN, both families), segmentation offload (`UDP_GRO` / `UDP_SEGMENT`), transmit pacing (`SO_TXTIME`), demultiplexing, and protocol timers. diff --git a/docs/metadata.md b/docs/metadata.md index 1a85b95..50a0dfe 100644 --- a/docs/metadata.md +++ b/docs/metadata.md @@ -1,8 +1,8 @@ # Ancillary data Datagrams carry more than payload and peer address. The kernel reports which -local address one arrived on, and what its ECN marking was, as *control -messages* alongside the data. This layer turns those into typed values and +local address one arrived on, and what its DSCP and ECN marking were, as +*control messages* alongside the data. This layer turns those into typed values and attaches them on the way out. ## The feature set @@ -12,12 +12,16 @@ determines three things that must agree: how large the control buffer is, what the parser looks for, and what can be attached on transmit. ```cpp -using metadata_set = dgram::features; +using rx_set = dgram::features; +using tx_set = dgram::features; -using rx = dgram::receive_batch<64, 2048, metadata_set>; -using tx = dgram::transmit_batch<64, 0, metadata_set>; +using rx = dgram::receive_batch<64, 2048, rx_set>; +using tx = dgram::transmit_batch<64, 0, tx_set>; ``` +Not every feature goes both ways, so a receive and a transmit batch usually take +different sets. ECN is read with `ecn` and sent with `traffic_class`. + | Member | Description | |--------|-------------| | `count` | How many features. | @@ -68,8 +72,8 @@ short parse rather than a read past the end. The loss is reported separately, as ## Writing it ```cpp -dgram::control reply{}; -reply.set(dgram::ecn_codepoint::ect0); +dgram::control reply{}; +reply.set({.dscp = dgram::dscp::ef, .ecn = dgram::ecn_codepoint::ect0}); (void)tx.stage(payload, peer, reply); ``` @@ -103,9 +107,19 @@ Sized for the larger of the two families' payloads, because the receive path cannot know which will arrive: a dual-stack socket reports `IPV6_PKTINFO` even for v4-mapped traffic. -## `ecn` +## `ecn` and `traffic_class` + +Both read `IP_TOS` / `IPV6_TCLASS`, the traffic-class byte: six bits of DSCP +and two of ECN. `ecn` reduces it to the ECN bits and is **receive-only**. +`traffic_class` carries the whole byte as a `marking {dscp, ecn}` and goes both +ways. -`IP_TOS` / `IPV6_TCLASS`, reduced to the two ECN bits of RFC 3168. +`ecn` cannot be sent because the kernel takes a per-datagram traffic class as +the whole byte. Marking the ECN bits alone would zero the DSCP, so every +ECN-capable datagram would leave as best effort. Sending goes through +`traffic_class`, which makes the DSCP part of the value. That value also +replaces any DSCP the socket was given with `setsockopt(IP_TOS)`: a socket-level +DSCP only applies to datagrams that carry no `traffic_class`. | `ecn_codepoint` | Meaning | |-----------------|---------| @@ -114,8 +128,12 @@ for v4-mapped traffic. | `ect0` | ECN-capable, codepoint 0. | | `ce` | Congestion experienced. | -An unmarked datagram reads as `not_ect`, not as absent, provided reporting was -enabled. +`dscp` names the standard classes (`df`, `le`, `cs1` to `cs7`, `af11` to +`af43`, `voice_admit`, `ef`), and any value up to 63 is valid through a cast. + +An unmarked datagram reads as `not_ect` (and `dscp::df`), not as absent, +provided reporting was enabled. A receive set can hold both features; they read +the same message. On a v6 socket, enabling it sets both `IPV6_RECVTCLASS` and `IP_RECVTOS`. IPv4 peers of a dual-stack socket arrive through the kernel's IPv4 path, which reports diff --git a/docs/offload.md b/docs/offload.md index 1e6e487..252d802 100644 --- a/docs/offload.md +++ b/docs/offload.md @@ -12,7 +12,7 @@ each take their own: ```cpp using rx_features = dgram::features; -using tx_features = dgram::features; +using tx_features = dgram::features; using rx = dgram::receive_batch<16, 1 << 16, rx_features>; using tx = dgram::transmit_batch<16, 0, tx_features>; diff --git a/docs/pacing.md b/docs/pacing.md index 9e279d5..8ca9714 100644 --- a/docs/pacing.md +++ b/docs/pacing.md @@ -54,7 +54,7 @@ option later. unpaced entries: ```cpp -using tx_features = dgram::features; +using tx_features = dgram::features; dgram::control ancillary{}; ancillary.set(when); diff --git a/docs/testing.md b/docs/testing.md index 5a3fa51..0919fda 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -19,7 +19,7 @@ checkout. | `doc_examples.cpp` | Every snippet printed in `README.md` and `docs/`, compiled and run. A snippet in a document is untested code that looks authoritative; this is what stops one rotting. | | `compile_checks.cpp` | Pure `static_assert`. Feature concepts, control-buffer sizing, batch geometry, which resources may be carved from, which metadata is readable, and SipHash against its published vectors, split at every word boundary. Compiling is the test; it is never run. | | `loopback_tests.cpp` | Batch reuse, truncation reporting (including a caller-supplied `MSG_TRUNC`), nonblocking receive, staging limits, a failed flush leaving the batch staged, the error combinators. | -| `metadata_tests.cpp` | Destination address and ECN over loopback, both families, including a feature that was never enabled and an undersized control buffer. Dual-stack sockets: ECN from and to IPv4 peers, and a reply from the arrival address. | +| `metadata_tests.cpp` | Destination address and ECN over loopback, both families, including a feature that was never enabled and an undersized control buffer. DSCP and ECN together in both families, and a per-datagram marking replacing the socket's DSCP. Dual-stack sockets: ECN from and to IPv4 peers, and a reply from the arrival address. | | `malformed_control_tests.cpp` | Control buffers the kernel would never write. Every case here was found by the fuzzer first. | | `offload_tests.cpp` | GRO and GSO over loopback, and the segmentation walk on its own: short tail, exact multiple, stride of zero, stride past the end, empty payload. | | `demux_tests.cpp` | Key equality and hash agreement (including the flow-label and padding cases), keys crafted to collide under one seed spreading under another, the table bucketing by its own seed, the flow table under churn and at capacity, projections, routing a GRO slot whose datagrams belong to different flows, an `unmatched` callback claiming or declining a datagram, arguments forwarded to a sink that takes them and ignored by one that does not, and the per-slot count. | diff --git a/examples/echo.cpp b/examples/echo.cpp index f947461..ec65a0b 100644 --- a/examples/echo.cpp +++ b/examples/echo.cpp @@ -21,9 +21,10 @@ constexpr std::size_t batch_capacity{16}; constexpr std::size_t slot_bytes{1 << 16}; // The two directions carry different features: UDP_GRO is only ever received -// and UDP_SEGMENT only ever sent. +// and UDP_SEGMENT only ever sent. ECN is read with `ecn` and sent with +// `traffic_class`, which carries the DSCP alongside it. using rx_features = dgram::features; -using tx_features = dgram::features; +using tx_features = dgram::features; using rx_batch = dgram::receive_batch; @@ -82,7 +83,7 @@ int main(const int argc, const char* const* argv) { // Reflect the sender's ECN codepoint rather than sending unmarked. dgram::control reply{}; if (const auto marking{meta.get()}) { - reply.set(*marking); + reply.set({.dscp = dgram::dscp::df, .ecn = *marking}); } if (!reported) { diff --git a/fuzz/fuzz_cmsg_parse.cpp b/fuzz/fuzz_cmsg_parse.cpp index 8fbfa21..273a833 100644 --- a/fuzz/fuzz_cmsg_parse.cpp +++ b/fuzz/fuzz_cmsg_parse.cpp @@ -28,7 +28,7 @@ import dgram; namespace { -using both = dgram::features; +using combined = dgram::features; using only_pktinfo = dgram::features; using only_ecn = dgram::features; @@ -74,6 +74,11 @@ template void parse_as(const std::span byte (void)std::to_underlying(*marking); } } + if constexpr (Features::template contains) { + if (const auto& marking{meta.template get()}) { + (void)marking->byte(); + } + } ::operator delete(control, len, std::align_val_t{alignof(::cmsghdr)}); } @@ -89,7 +94,7 @@ extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t* data, std::size_t size switch (selector % 3) { case 0: - parse_as(body); + parse_as(body); break; case 1: parse_as(body); diff --git a/src/dgram/metadata.cppm b/src/dgram/metadata.cppm index a1f9683..5d5e42c 100644 --- a/src/dgram/metadata.cppm +++ b/src/dgram/metadata.cppm @@ -117,7 +117,7 @@ export struct pktinfo { }; /* ============================================================================ - * Explicit Congestion Notification + * Traffic class: DSCP and ECN * ============================================================================ */ /** @brief The two ECN bits of the traffic class, as RFC 3168 defines them. */ @@ -128,45 +128,91 @@ export enum class ecn_codepoint : std::uint8_t { ce = IPTOS_ECN_CE, ///< congestion experienced }; +/** @brief The six DSCP bits of the traffic class. Any value up to 63 is valid; these are the named ones. */ +export enum class dscp : std::uint8_t { + df = 0, ///< default forwarding, best effort + le = 1, ///< lower effort, RFC 8622 + cs1 = 8, ///< class selector 1 + af11 = 10, ///< assured forwarding, class 1 + af12 = 12, ///< assured forwarding, class 1 + af13 = 14, ///< assured forwarding, class 1 + cs2 = 16, ///< class selector 2 + af21 = 18, ///< assured forwarding, class 2 + af22 = 20, ///< assured forwarding, class 2 + af23 = 22, ///< assured forwarding, class 2 + cs3 = 24, ///< class selector 3 + af31 = 26, ///< assured forwarding, class 3 + af32 = 28, ///< assured forwarding, class 3 + af33 = 30, ///< assured forwarding, class 3 + cs4 = 32, ///< class selector 4 + af41 = 34, ///< assured forwarding, class 4 + af42 = 36, ///< assured forwarding, class 4 + af43 = 38, ///< assured forwarding, class 4 + cs5 = 40, ///< class selector 5 + voice_admit = 44, ///< capacity-admitted voice, RFC 5865 + ef = 46, ///< expedited forwarding + cs6 = 48, ///< class selector 6, network control + cs7 = 56, ///< class selector 7 +}; + +/** @brief The whole traffic-class byte: DSCP in the high six bits, ECN in the low two. */ +export struct marking { + dgram::dscp dscp{}; + ecn_codepoint ecn{}; + + /** @brief The byte on the wire. A DSCP above 63 loses its high bits. */ + [[nodiscard]] constexpr std::uint8_t byte() const noexcept { + return static_cast(((std::to_underlying(dscp) & 0x3FU) << 2) | (std::to_underlying(ecn) & IPTOS_ECN_MASK)); + } + + [[nodiscard]] static constexpr marking from_byte(const std::uint8_t byte) noexcept { + return {static_cast(byte >> 2), static_cast(byte & IPTOS_ECN_MASK)}; + } + + // Not `= default`, for the GCC 16.2 crash noted on `hash_seed`. + [[nodiscard]] friend constexpr bool operator==(const marking& a, const marking& b) noexcept { return a.byte() == b.byte(); } +}; + +namespace detail { + /** - * @brief `IP_TOS` / `IPV6_TCLASS`, reduced to the ECN bits. + * @brief `IP_TOS` / `IPV6_TCLASS`: the one message behind `ecn` and `traffic_class`. * * The widths differ and the difference is not documented anywhere obvious: * `IP_TOS` arrives as a single byte, `IPV6_TCLASS` as a four-byte `int`. Reading * the wrong one overreads the control buffer, so `parse` branches on the length * the kernel actually wrote rather than on the family. */ -export struct ecn { - using value_type = ecn_codepoint; - - static constexpr std::size_t space{std::max(detail::space_for(sizeof(std::uint8_t)), detail::space_for(sizeof(int)))}; +struct tos_message { + static constexpr std::size_t space{std::max(space_for(sizeof(std::uint8_t)), space_for(sizeof(int)))}; [[nodiscard]] static bool matches(const int level, const int type) noexcept { return (level == IPPROTO_IP && type == IP_TOS) || (level == IPPROTO_IPV6 && type == IPV6_TCLASS); } - [[nodiscard]] static std::optional parse(const ::cmsghdr* c) noexcept { - const auto width{detail::payload_size(c)}; + [[nodiscard]] static std::optional parse(const ::cmsghdr* c) noexcept { + const auto width{payload_size(c)}; if (width >= sizeof(int)) { - return from_traffic_class(detail::read_payload(c)); + return static_cast(read_payload(c)); } if (width >= sizeof(std::uint8_t)) { - return from_traffic_class(detail::read_payload(c)); + return read_payload(c); } return std::nullopt; } /** - * @brief Mark an outgoing datagram. + * @brief Written as an `int` for both families; the kernel accepts either width for `IP_TOS`. * - * Written at the width each family expects, mirroring the receive side. + * The kernel takes the value as the datagram's whole traffic-class byte, + * replacing any DSCP the socket was given with `setsockopt`. */ - [[nodiscard]] static std::size_t build(::cmsghdr* dst, const value_type value, const family fam) noexcept { - const int bits{static_cast(std::to_underlying(value))}; + [[nodiscard]] static std::size_t build(::cmsghdr* dst, const std::uint8_t byte, const family fam) noexcept { + const int value{byte}; if (fam == family::inet4) { - return detail::write_message(dst, IPPROTO_IP, IP_TOS, bits); + return write_message(dst, IPPROTO_IP, IP_TOS, value); } - return detail::write_message(dst, IPPROTO_IPV6, IPV6_TCLASS, bits); + return write_message(dst, IPPROTO_IPV6, IPV6_TCLASS, value); } /** @@ -177,21 +223,56 @@ export struct ecn { * `IPV6_TCLASS`. The option is accepted on a v6-only socket too. */ [[nodiscard]] static result<> enable(const int fd, const family fam) noexcept { - const auto report_tos = [fd] { return detail::set_option(fd, IPPROTO_IP, IP_RECVTOS, 1); }; + const auto report_tos = [fd] { return set_option(fd, IPPROTO_IP, IP_RECVTOS, 1); }; if (fam == family::inet4) { return report_tos(); } - return detail::set_option(fd, IPPROTO_IPV6, IPV6_RECVTCLASS, 1) | then(report_tos); + return set_option(fd, IPPROTO_IPV6, IPV6_RECVTCLASS, 1) | then(report_tos); } +}; + +} // namespace detail -private: - [[nodiscard]] static constexpr ecn_codepoint from_traffic_class(const std::integral auto tclass) noexcept { - return static_cast(static_cast(tclass) & IPTOS_ECN_MASK); +/** + * @brief The ECN bits of a received datagram. Receive-only. + * + * Not sendable on purpose: the kernel takes a per-datagram traffic class as the + * whole byte, so marking ECN alone would zero the DSCP. Send `traffic_class`. + */ +export struct ecn { + using value_type = ecn_codepoint; + + static constexpr std::size_t space{detail::tos_message::space}; + + [[nodiscard]] static bool matches(const int level, const int type) noexcept { return detail::tos_message::matches(level, type); } + + [[nodiscard]] static std::optional parse(const ::cmsghdr* c) noexcept { + return detail::tos_message::parse(c).transform([](const std::uint8_t byte) { return marking::from_byte(byte).ecn; }); } + + [[nodiscard]] static result<> enable(const int fd, const family fam) noexcept { return detail::tos_message::enable(fd, fam); } +}; + +/** @brief The whole traffic class, DSCP and ECN, in both directions. */ +export struct traffic_class { + using value_type = marking; + + static constexpr std::size_t space{detail::tos_message::space}; + + [[nodiscard]] static bool matches(const int level, const int type) noexcept { return detail::tos_message::matches(level, type); } + + [[nodiscard]] static std::optional parse(const ::cmsghdr* c) noexcept { return detail::tos_message::parse(c).transform(marking::from_byte); } + + [[nodiscard]] static std::size_t build(::cmsghdr* dst, const value_type& value, const family fam) noexcept { + return detail::tos_message::build(dst, value.byte(), fam); + } + + [[nodiscard]] static result<> enable(const int fd, const family fam) noexcept { return detail::tos_message::enable(fd, fam); } }; static_assert(cmsg_feature && sendable_feature && receivable_feature); -static_assert(cmsg_feature && sendable_feature && receivable_feature); +static_assert(receivable_feature && !sendable_feature, "sending the ECN bits alone would zero the DSCP; send traffic_class"); +static_assert(receivable_feature && sendable_feature); /* ============================================================================ * Enabling a whole set at open time diff --git a/tests/compile_checks.cpp b/tests/compile_checks.cpp index 87ce023..2664860 100644 --- a/tests/compile_checks.cpp +++ b/tests/compile_checks.cpp @@ -17,9 +17,21 @@ namespace { static_assert(dgram::cmsg_feature); static_assert(dgram::cmsg_feature); static_assert(dgram::sendable_feature); -static_assert(dgram::sendable_feature); static_assert(dgram::receivable_feature); static_assert(dgram::receivable_feature); +static_assert(dgram::sendable_feature && dgram::receivable_feature); + +/* The kernel takes a per-datagram traffic class as the whole byte, so sending + the ECN bits alone would zero the DSCP. Only traffic_class sends. */ +static_assert(!dgram::sendable_feature); + +/* DSCP in the high six bits, ECN in the low two. */ +static_assert(dgram::marking{dgram::dscp::ef, dgram::ecn_codepoint::ect0}.byte() == 0xBA); +static_assert(dgram::marking{dgram::dscp::cs7, dgram::ecn_codepoint::ce}.byte() == 0xE3); +static_assert(dgram::marking{}.byte() == 0); +static_assert(dgram::marking::from_byte(0xBA) == dgram::marking{dgram::dscp::ef, dgram::ecn_codepoint::ect0}); +static_assert(dgram::marking{static_cast(0xFF), dgram::ecn_codepoint::not_ect}.byte() == 0xFC, "a DSCP above 63 cannot spill into the ECN bits"); +static_assert(dgram::traffic_class::space == dgram::ecn::space); /* A type carrying only a size is not a feature: it can contribute to the buffer but never be parsed out of it, which is exactly the mismatch to prevent. */ @@ -68,8 +80,8 @@ static_assert(dgram::sendable_feature && !dgram::parseable_featur /* Four distinct cmsg payload widths are now in play, so every control-buffer term is derived from the feature rather than written down. */ -using paced = dgram::features; -static_assert(paced::control_space == dgram::txtime::space + dgram::ecn::space + dgram::segment::space); +using paced = dgram::features; +static_assert(paced::control_space == dgram::txtime::space + dgram::traffic_class::space + dgram::segment::space); static_assert(dgram::txtime::space == dgram::detail::space_for(sizeof(std::uint64_t))); static_assert(dgram::segment::space == dgram::detail::space_for(sizeof(std::uint16_t))); static_assert(dgram::gro::space == dgram::detail::space_for(sizeof(int))); @@ -160,7 +172,8 @@ static_assert(std::is_same_v>( template concept settable = requires(dgram::control& c, typename F::value_type v) { c.template set(v); }; -static_assert(settable); +static_assert(settable, dgram::traffic_class>); +static_assert(!settable, "ecn is receive-only"); static_assert(!settable); /* The control buffer's size is carried by the type, not checked at runtime. diff --git a/tests/concurrency_tests.cpp b/tests/concurrency_tests.cpp index aa2080b..e570f1f 100644 --- a/tests/concurrency_tests.cpp +++ b/tests/concurrency_tests.cpp @@ -23,8 +23,9 @@ constexpr std::size_t worker_count{4}; constexpr std::size_t datagrams_per_worker{64}; using meta_set = dgram::features; +using tx_set = dgram::features; using rx_batch = dgram::receive_batch; -using tx_batch = dgram::transmit_batch; +using tx_batch = dgram::transmit_batch; std::span bytes_of(std::string_view s) noexcept { return {reinterpret_cast(s.data()), s.size()}; @@ -64,8 +65,8 @@ TEST(Concurrency, WorkersShareNothing) { } for (std::size_t sent{}; sent < datagrams_per_worker; ++sent) { - dgram::control marked{}; - marked.set(dgram::ecn_codepoint::ect0); + dgram::control marked{}; + marked.set({.ecn = dgram::ecn_codepoint::ect0}); const auto text{std::format("w-{}-{}", w, sent)}; if (!tx->stage_copy(bytes_of(text), target, marked)) { continue; diff --git a/tests/metadata_tests.cpp b/tests/metadata_tests.cpp index 7c5febf..862e08a 100644 --- a/tests/metadata_tests.cpp +++ b/tests/metadata_tests.cpp @@ -20,8 +20,9 @@ constexpr std::size_t capacity{8}; constexpr std::size_t slot{512}; using meta_set = dgram::features; +using tx_set = dgram::features; using rx_batch = dgram::receive_batch; -using tx_batch = dgram::transmit_batch; +using tx_batch = dgram::transmit_batch; std::span bytes_of(std::string_view s) noexcept { return {reinterpret_cast(s.data()), s.size()}; @@ -99,8 +100,8 @@ TEST_P(EcnRoundTrip, MarkedCodepointSurvives) { ASSERT_TRUE(rx.has_value() && tx.has_value()); for (const auto marked : {dgram::ecn_codepoint::ect0, dgram::ecn_codepoint::ect1, dgram::ecn_codepoint::ce}) { - dgram::control ancillary{}; - ancillary.set(marked); + dgram::control ancillary{}; + ancillary.set({.ecn = marked}); ASSERT_TRUE(tx->stage_copy(bytes_of("marked"), net.target, ancillary)); ASSERT_TRUE(tx->flush(net.sender).has_value()); @@ -132,6 +133,88 @@ TEST(Metadata, UnmarkedDatagramReadsAsNotEct) { EXPECT_EQ(*got, dgram::ecn_codepoint::not_ect); } +/* ============================================================================ + * Traffic class: DSCP travels with ECN + * ============================================================================ */ + +using class_set = dgram::features; +using class_rx = dgram::receive_batch; + +/** A receiver reporting the traffic class, and a plain sender aimed at it. */ +struct class_fixture { + libmem::arena arena{class_rx::footprint() + tx_batch::footprint()}; + dgram::socket receiver; + dgram::socket sender; + dgram::endpoint target{}; + + explicit class_fixture(const dgram::family f) + : receiver{std::move(*dgram::socket::open>(f))}, sender{std::move(*dgram::socket::open<>(f))} { + EXPECT_TRUE(receiver.bind(dgram::endpoint::any(f, 0)).has_value()); + target = *dgram::endpoint::parse(f, f == dgram::family::inet4 ? "127.0.0.1" : "::1", receiver.local_address()->port()); + } + + /** The DSCP a socket applies to datagrams that carry no traffic class of their own. */ + void set_socket_dscp(const dgram::family f, const dgram::dscp d) const { + const int byte{dgram::marking{d, dgram::ecn_codepoint::not_ect}.byte()}; + const int level{f == dgram::family::inet4 ? IPPROTO_IP : IPPROTO_IPV6}; + const int name{f == dgram::family::inet4 ? IP_TOS : IPV6_TCLASS}; + ASSERT_EQ(::setsockopt(sender.native(), level, name, &byte, sizeof(byte)), 0); + } +}; + +class TrafficClass : public testing::TestWithParam {}; + +/* The bug this guards: marking ECN used to write the ECN bits as the whole byte, + so every ECN-marked datagram went out as best effort. */ +TEST_P(TrafficClass, DscpAndEcnArriveTogether) { + class_fixture net{GetParam()}; + auto rx{class_rx::carve(net.arena)}; + auto tx{tx_batch::carve(net.arena)}; + ASSERT_TRUE(rx.has_value() && tx.has_value()); + + for (const auto sent : {dgram::marking{dgram::dscp::ef, dgram::ecn_codepoint::ect0}, dgram::marking{dgram::dscp::af41, dgram::ecn_codepoint::ce}, + dgram::marking{dgram::dscp::cs1, dgram::ecn_codepoint::not_ect}}) { + dgram::control ancillary{}; + ancillary.set(sent); + ASSERT_TRUE(tx->stage_copy(bytes_of("marked"), net.target, ancillary)); + ASSERT_TRUE(tx->flush(net.sender).has_value()); + + ASSERT_EQ(*rx->receive(net.receiver), 1u); + const auto meta{(*rx->datagrams().begin()).meta()}; + ASSERT_TRUE(meta.get().has_value()); + EXPECT_EQ(meta.get()->byte(), sent.byte()); + ASSERT_TRUE(meta.get().has_value()) << "ecn reads the same message"; + EXPECT_EQ(*meta.get(), sent.ecn); + } +} + +/* The per-datagram byte owns the whole traffic class: a socket-level DSCP applies + only to datagrams that carry none. */ +TEST_P(TrafficClass, PerDatagramMarkingReplacesTheSocketDscp) { + class_fixture net{GetParam()}; + net.set_socket_dscp(GetParam(), dgram::dscp::ef); + auto rx{class_rx::carve(net.arena)}; + auto tx{tx_batch::carve(net.arena)}; + ASSERT_TRUE(rx.has_value() && tx.has_value()); + + ASSERT_TRUE(tx->stage_copy(bytes_of("socket default"), net.target)); + ASSERT_TRUE(tx->flush(net.sender).has_value()); + ASSERT_EQ(*rx->receive(net.receiver), 1u); + EXPECT_EQ((*rx->datagrams().begin()).meta().get()->dscp, dgram::dscp::ef); + + dgram::control ancillary{}; + ancillary.set({.ecn = dgram::ecn_codepoint::ect0}); + ASSERT_TRUE(tx->stage_copy(bytes_of("own marking"), net.target, ancillary)); + ASSERT_TRUE(tx->flush(net.sender).has_value()); + ASSERT_EQ(*rx->receive(net.receiver), 1u); + const auto got{*(*rx->datagrams().begin()).meta().get()}; + EXPECT_EQ(got.dscp, dgram::dscp::df) << "the datagram said df, and that wins over the socket"; + EXPECT_EQ(got.ecn, dgram::ecn_codepoint::ect0); +} + +INSTANTIATE_TEST_SUITE_P(BothFamilies, TrafficClass, testing::Values(dgram::family::inet4, dgram::family::inet6), + [](const auto& p) { return p.param == dgram::family::inet4 ? "IPv4" : "IPv6"; }); + /* ============================================================================ * Dual-stack sockets: IPv4 peers on a v6 socket * ============================================================================ */ @@ -162,8 +245,8 @@ TEST(DualStack, ReportsEcnForIPv4Arrivals) { ASSERT_TRUE(rx.has_value() && tx.has_value()); for (const auto marked : marked_codepoints) { - dgram::control ancillary{}; - ancillary.set(marked); + dgram::control ancillary{}; + ancillary.set({.ecn = marked}); ASSERT_TRUE(tx->stage_copy(bytes_of("from v4"), target, ancillary)); ASSERT_TRUE(tx->flush(net.sender).has_value()); @@ -190,8 +273,8 @@ TEST(DualStack, MarksEcnOnDatagramsToV4MappedPeers) { ASSERT_TRUE(rx.has_value() && tx.has_value()); for (const auto marked : marked_codepoints) { - dgram::control ancillary{}; - ancillary.set(marked); + dgram::control ancillary{}; + ancillary.set({.ecn = marked}); ASSERT_TRUE(tx->stage_copy(bytes_of("to v4-mapped"), target, ancillary)); const auto sent{tx->flush(sender)}; ASSERT_TRUE(sent.has_value()) << dgram::describe(sent.error()); @@ -220,8 +303,8 @@ TEST(DualStack, RepliesToAnIPv4PeerFromTheArrivalAddress) { ASSERT_TRUE(client_rx.has_value() && client_tx.has_value() && server_rx.has_value() && server_tx.has_value()); // The client's receive socket sends too, so the reply comes back to it. - dgram::control request{}; - request.set(dgram::ecn_codepoint::ect1); + dgram::control request{}; + request.set({.ecn = dgram::ecn_codepoint::ect1}); ASSERT_TRUE(client_tx->stage_copy(bytes_of("ping"), server_target, request)); ASSERT_TRUE(client_tx->flush(client.receiver).has_value()); @@ -230,9 +313,9 @@ TEST(DualStack, RepliesToAnIPv4PeerFromTheArrivalAddress) { const auto meta{arrival.meta()}; ASSERT_TRUE(meta.get().has_value() && meta.get().has_value()); - dgram::control reply{}; + dgram::control reply{}; reply.set(*meta.get()); - reply.set(*meta.get()); + reply.set({.ecn = *meta.get()}); ASSERT_TRUE(server_tx->stage_copy(bytes_of("pong"), arrival.from(), reply)); const auto sent{server_tx->flush(server)}; ASSERT_TRUE(sent.has_value()) << dgram::describe(sent.error()); @@ -321,8 +404,8 @@ TEST(Metadata, StaleControlLengthDoesNotLeakBetweenDatagrams) { auto tx{tx_batch::carve(net.arena)}; ASSERT_TRUE(rx.has_value() && tx.has_value()); - dgram::control marked{}; - marked.set(dgram::ecn_codepoint::ce); + dgram::control marked{}; + marked.set({.ecn = dgram::ecn_codepoint::ce}); ASSERT_TRUE(tx->stage_copy(bytes_of("marked"), net.target, marked)); ASSERT_TRUE(tx->flush(net.sender).has_value()); diff --git a/tests/offload_tests.cpp b/tests/offload_tests.cpp index 42f4546..5581f02 100644 --- a/tests/offload_tests.cpp +++ b/tests/offload_tests.cpp @@ -24,7 +24,7 @@ constexpr std::size_t capacity{8}; constexpr std::size_t gro_slot{1 << 16}; using rx_set = dgram::features; -using tx_set = dgram::features; +using tx_set = dgram::features; using rx_batch = dgram::receive_batch; using tx_batch = dgram::transmit_batch; @@ -187,7 +187,7 @@ TEST(Offload, SegmentAndEcnTravelTogether) { std::vector payload(3 * mtu); dgram::control ancillary{}; ancillary.set(mtu); - ancillary.set(dgram::ecn_codepoint::ect0); + ancillary.set({.ecn = dgram::ecn_codepoint::ect0}); ASSERT_TRUE(tx->stage(payload, net.target, ancillary)); ASSERT_TRUE(tx->flush(net.sender).has_value()); diff --git a/tests/pacing_tests.cpp b/tests/pacing_tests.cpp index 38d7b64..fb75eca 100644 --- a/tests/pacing_tests.cpp +++ b/tests/pacing_tests.cpp @@ -27,7 +27,7 @@ constexpr std::size_t capacity{8}; constexpr std::size_t slot{2048}; constexpr dgram::pacer::rate_type gigabit{125'000'000}; // bytes/s -using tx_set = dgram::features; +using tx_set = dgram::features; using tx_batch = dgram::transmit_batch; using rx_batch = dgram::receive_batch; @@ -216,7 +216,7 @@ TEST(TransmitTime, PacingAndEcnTravelTogether) { dgram::control ancillary{}; ancillary.set(1'000'000ns); - ancillary.set(dgram::ecn_codepoint::ect0); + ancillary.set({.ecn = dgram::ecn_codepoint::ect0}); const auto target{*dgram::endpoint::parse(dgram::family::inet4, "127.0.0.1", 9)}; EXPECT_TRUE(tx->stage(bytes_of("x"), target, ancillary));