Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ A Linux UDP datagram engine in C++26: batched send and receive, all memory
carved once at startup, and a protocol boundary that hands out nothing but
`std::span` and timing.

Built on `recvmmsg` / `sendmmsg`, ancillary data (`IP_PKTINFO`, ECN), hardware
Built on `recvmmsg` / `sendmmsg`, ancillary data (`IP_PKTINFO`, DSCP and ECN), hardware
segmentation offload (`UDP_GRO` / `UDP_SEGMENT`) and transmit pacing
(`SO_TXTIME`). It is not a general network library and has no TCP: the whole
design turns on a batch being reclaimable in one step, which a byte stream is
Expand Down Expand Up @@ -65,7 +65,7 @@ compiled and run as a test, so none of them can rot.
| `receive_batch` | `recvmmsg` argument block carved from a resource, plus a lazy view over what arrived. |
| `transmit_batch` | `sendmmsg` argument block. Stages by reference, so echoing costs no copy. |
| `features` | Compile-time set of ancillary-data features; the control buffer is sized as the sum over it. |
| `pktinfo` / `ecn` | Which local address a datagram arrived on, and its ECN marking. Both families. |
| `pktinfo` / `ecn` / `traffic_class` | Which local address a datagram arrived on, and its ECN and DSCP marking. Both families. |
| `gro` / `segment` | Hardware segmentation offload: many datagrams per slot in, one large buffer out. |
| `txtime` / `pacer` | Per-datagram departure times via `SO_TXTIME`, and drift-free rate arithmetic. |
| `flow_table` / `route` | Open-addressing demultiplexing to protocol state, keyed by a caller-supplied projection and hashed under a per-table SipHash seed, so peers cannot flood it. |
Expand Down Expand Up @@ -104,10 +104,11 @@ for (const auto& d : batch->datagrams() | std::views::filter(dgram::is_intact))
control buffer cannot fall out of step with what the parser expects.

```cpp
using metadata_set = dgram::features<dgram::pktinfo, dgram::ecn>;
using rx_set = dgram::features<dgram::pktinfo, dgram::ecn>;
using tx_set = dgram::features<dgram::pktinfo, dgram::traffic_class>;

const auto meta{d.meta()};
if (const auto marking{meta.get<dgram::ecn>()}) { reply.set<dgram::ecn>(*marking); }
if (const auto marking{meta.get<dgram::ecn>()}) { reply.set<dgram::traffic_class>({.ecn = *marking}); }
```

## Building
Expand Down Expand Up @@ -162,7 +163,7 @@ const auto counts{dgram::route(batch, dgram::by_payload_id<1, 8>{}, flows, now)}
## Status

All five phases are done and tested: the memory model, the batching syscalls,
ancillary data (destination address and ECN, both families), segmentation
ancillary data (destination address, DSCP and ECN, both families), segmentation
offload (`UDP_GRO` / `UDP_SEGMENT`), transmit pacing (`SO_TXTIME`),
demultiplexing, and protocol timers.

Expand Down
40 changes: 29 additions & 11 deletions docs/metadata.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Ancillary data

Datagrams carry more than payload and peer address. The kernel reports which
local address one arrived on, and what its ECN marking was, as *control
messages* alongside the data. This layer turns those into typed values and
local address one arrived on, and what its DSCP and ECN marking were, as
*control messages* alongside the data. This layer turns those into typed values and
attaches them on the way out.

## The feature set
Expand All @@ -12,12 +12,16 @@ determines three things that must agree: how large the control buffer is, what
the parser looks for, and what can be attached on transmit.

```cpp
using metadata_set = dgram::features<dgram::pktinfo, dgram::ecn>;
using rx_set = dgram::features<dgram::pktinfo, dgram::ecn>;
using tx_set = dgram::features<dgram::pktinfo, dgram::traffic_class>;

using rx = dgram::receive_batch<64, 2048, metadata_set>;
using tx = dgram::transmit_batch<64, 0, metadata_set>;
using rx = dgram::receive_batch<64, 2048, rx_set>;
using tx = dgram::transmit_batch<64, 0, tx_set>;
```

Not every feature goes both ways, so a receive and a transmit batch usually take
different sets. ECN is read with `ecn` and sent with `traffic_class`.

| Member | Description |
|--------|-------------|
| `count` | How many features. |
Expand Down Expand Up @@ -68,8 +72,8 @@ short parse rather than a read past the end. The loss is reported separately, as
## Writing it

```cpp
dgram::control<metadata_set> reply{};
reply.set<dgram::ecn>(dgram::ecn_codepoint::ect0);
dgram::control<tx_set> reply{};
reply.set<dgram::traffic_class>({.dscp = dgram::dscp::ef, .ecn = dgram::ecn_codepoint::ect0});
(void)tx.stage(payload, peer, reply);
```

Expand Down Expand Up @@ -103,9 +107,19 @@ Sized for the larger of the two families' payloads, because the receive path
cannot know which will arrive: a dual-stack socket reports `IPV6_PKTINFO` even
for v4-mapped traffic.

## `ecn`
## `ecn` and `traffic_class`

Both read `IP_TOS` / `IPV6_TCLASS`, the traffic-class byte: six bits of DSCP
and two of ECN. `ecn` reduces it to the ECN bits and is **receive-only**.
`traffic_class` carries the whole byte as a `marking {dscp, ecn}` and goes both
ways.

`IP_TOS` / `IPV6_TCLASS`, reduced to the two ECN bits of RFC 3168.
`ecn` cannot be sent because the kernel takes a per-datagram traffic class as
the whole byte. Marking the ECN bits alone would zero the DSCP, so every
ECN-capable datagram would leave as best effort. Sending goes through
`traffic_class`, which makes the DSCP part of the value. That value also
replaces any DSCP the socket was given with `setsockopt(IP_TOS)`: a socket-level
DSCP only applies to datagrams that carry no `traffic_class`.

| `ecn_codepoint` | Meaning |
|-----------------|---------|
Expand All @@ -114,8 +128,12 @@ for v4-mapped traffic.
| `ect0` | ECN-capable, codepoint 0. |
| `ce` | Congestion experienced. |

An unmarked datagram reads as `not_ect`, not as absent, provided reporting was
enabled.
`dscp` names the standard classes (`df`, `le`, `cs1` to `cs7`, `af11` to
`af43`, `voice_admit`, `ef`), and any value up to 63 is valid through a cast.

An unmarked datagram reads as `not_ect` (and `dscp::df`), not as absent,
provided reporting was enabled. A receive set can hold both features; they read
the same message.

On a v6 socket, enabling it sets both `IPV6_RECVTCLASS` and `IP_RECVTOS`. IPv4
peers of a dual-stack socket arrive through the kernel's IPv4 path, which reports
Expand Down
2 changes: 1 addition & 1 deletion docs/offload.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ each take their own:

```cpp
using rx_features = dgram::features<dgram::pktinfo, dgram::ecn, dgram::gro>;
using tx_features = dgram::features<dgram::ecn, dgram::segment>;
using tx_features = dgram::features<dgram::traffic_class, dgram::segment>;

using rx = dgram::receive_batch<16, 1 << 16, rx_features>;
using tx = dgram::transmit_batch<16, 0, tx_features>;
Expand Down
2 changes: 1 addition & 1 deletion docs/pacing.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ option later.
unpaced entries:

```cpp
using tx_features = dgram::features<dgram::txtime, dgram::ecn>;
using tx_features = dgram::features<dgram::txtime, dgram::traffic_class>;

dgram::control<tx_features> ancillary{};
ancillary.set<dgram::txtime>(when);
Expand Down
2 changes: 1 addition & 1 deletion docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ checkout.
| `doc_examples.cpp` | Every snippet printed in `README.md` and `docs/`, compiled and run. A snippet in a document is untested code that looks authoritative; this is what stops one rotting. |
| `compile_checks.cpp` | Pure `static_assert`. Feature concepts, control-buffer sizing, batch geometry, which resources may be carved from, which metadata is readable, and SipHash against its published vectors, split at every word boundary. Compiling is the test; it is never run. |
| `loopback_tests.cpp` | Batch reuse, truncation reporting (including a caller-supplied `MSG_TRUNC`), nonblocking receive, staging limits, a failed flush leaving the batch staged, the error combinators. |
| `metadata_tests.cpp` | Destination address and ECN over loopback, both families, including a feature that was never enabled and an undersized control buffer. Dual-stack sockets: ECN from and to IPv4 peers, and a reply from the arrival address. |
| `metadata_tests.cpp` | Destination address and ECN over loopback, both families, including a feature that was never enabled and an undersized control buffer. DSCP and ECN together in both families, and a per-datagram marking replacing the socket's DSCP. Dual-stack sockets: ECN from and to IPv4 peers, and a reply from the arrival address. |
| `malformed_control_tests.cpp` | Control buffers the kernel would never write. Every case here was found by the fuzzer first. |
| `offload_tests.cpp` | GRO and GSO over loopback, and the segmentation walk on its own: short tail, exact multiple, stride of zero, stride past the end, empty payload. |
| `demux_tests.cpp` | Key equality and hash agreement (including the flow-label and padding cases), keys crafted to collide under one seed spreading under another, the table bucketing by its own seed, the flow table under churn and at capacity, projections, routing a GRO slot whose datagrams belong to different flows, an `unmatched` callback claiming or declining a datagram, arguments forwarded to a sink that takes them and ignored by one that does not, and the per-slot count. |
Expand Down
7 changes: 4 additions & 3 deletions examples/echo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,10 @@ constexpr std::size_t batch_capacity{16};
constexpr std::size_t slot_bytes{1 << 16};

// The two directions carry different features: UDP_GRO is only ever received
// and UDP_SEGMENT only ever sent.
// and UDP_SEGMENT only ever sent. ECN is read with `ecn` and sent with
// `traffic_class`, which carries the DSCP alongside it.
using rx_features = dgram::features<dgram::pktinfo, dgram::ecn, dgram::gro>;
using tx_features = dgram::features<dgram::ecn>;
using tx_features = dgram::features<dgram::traffic_class>;

using rx_batch = dgram::receive_batch<batch_capacity, slot_bytes, rx_features>;

Expand Down Expand Up @@ -82,7 +83,7 @@ int main(const int argc, const char* const* argv) {
// Reflect the sender's ECN codepoint rather than sending unmarked.
dgram::control<tx_features> reply{};
if (const auto marking{meta.get<dgram::ecn>()}) {
reply.set<dgram::ecn>(*marking);
reply.set<dgram::traffic_class>({.dscp = dgram::dscp::df, .ecn = *marking});
}

if (!reported) {
Expand Down
9 changes: 7 additions & 2 deletions fuzz/fuzz_cmsg_parse.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import dgram;

namespace {

using both = dgram::features<dgram::pktinfo, dgram::ecn>;
using combined = dgram::features<dgram::pktinfo, dgram::ecn, dgram::traffic_class>;
using only_pktinfo = dgram::features<dgram::pktinfo>;
using only_ecn = dgram::features<dgram::ecn>;

Expand Down Expand Up @@ -74,6 +74,11 @@ template <typename Features> void parse_as(const std::span<const std::byte> byte
(void)std::to_underlying(*marking);
}
}
if constexpr (Features::template contains<dgram::traffic_class>) {
if (const auto& marking{meta.template get<dgram::traffic_class>()}) {
(void)marking->byte();
}
}

::operator delete(control, len, std::align_val_t{alignof(::cmsghdr)});
}
Expand All @@ -89,7 +94,7 @@ extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t* data, std::size_t size

switch (selector % 3) {
case 0:
parse_as<both>(body);
parse_as<combined>(body);
break;
case 1:
parse_as<only_pktinfo>(body);
Expand Down
125 changes: 103 additions & 22 deletions src/dgram/metadata.cppm
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ export struct pktinfo {
};

/* ============================================================================
* Explicit Congestion Notification
* Traffic class: DSCP and ECN
* ============================================================================ */

/** @brief The two ECN bits of the traffic class, as RFC 3168 defines them. */
Expand All @@ -128,45 +128,91 @@ export enum class ecn_codepoint : std::uint8_t {
ce = IPTOS_ECN_CE, ///< congestion experienced
};

/** @brief The six DSCP bits of the traffic class. Any value up to 63 is valid; these are the named ones. */
export enum class dscp : std::uint8_t {
df = 0, ///< default forwarding, best effort
le = 1, ///< lower effort, RFC 8622
cs1 = 8, ///< class selector 1
af11 = 10, ///< assured forwarding, class 1
af12 = 12, ///< assured forwarding, class 1
af13 = 14, ///< assured forwarding, class 1
cs2 = 16, ///< class selector 2
af21 = 18, ///< assured forwarding, class 2
af22 = 20, ///< assured forwarding, class 2
af23 = 22, ///< assured forwarding, class 2
cs3 = 24, ///< class selector 3
af31 = 26, ///< assured forwarding, class 3
af32 = 28, ///< assured forwarding, class 3
af33 = 30, ///< assured forwarding, class 3
cs4 = 32, ///< class selector 4
af41 = 34, ///< assured forwarding, class 4
af42 = 36, ///< assured forwarding, class 4
af43 = 38, ///< assured forwarding, class 4
cs5 = 40, ///< class selector 5
voice_admit = 44, ///< capacity-admitted voice, RFC 5865
ef = 46, ///< expedited forwarding
cs6 = 48, ///< class selector 6, network control
cs7 = 56, ///< class selector 7
};

/** @brief The whole traffic-class byte: DSCP in the high six bits, ECN in the low two. */
export struct marking {
dgram::dscp dscp{};
ecn_codepoint ecn{};

/** @brief The byte on the wire. A DSCP above 63 loses its high bits. */
[[nodiscard]] constexpr std::uint8_t byte() const noexcept {
return static_cast<std::uint8_t>(((std::to_underlying(dscp) & 0x3FU) << 2) | (std::to_underlying(ecn) & IPTOS_ECN_MASK));
}

[[nodiscard]] static constexpr marking from_byte(const std::uint8_t byte) noexcept {
return {static_cast<dgram::dscp>(byte >> 2), static_cast<ecn_codepoint>(byte & IPTOS_ECN_MASK)};
}

// Not `= default`, for the GCC 16.2 crash noted on `hash_seed`.
[[nodiscard]] friend constexpr bool operator==(const marking& a, const marking& b) noexcept { return a.byte() == b.byte(); }
};

namespace detail {

/**
* @brief `IP_TOS` / `IPV6_TCLASS`, reduced to the ECN bits.
* @brief `IP_TOS` / `IPV6_TCLASS`: the one message behind `ecn` and `traffic_class`.
*
* The widths differ and the difference is not documented anywhere obvious:
* `IP_TOS` arrives as a single byte, `IPV6_TCLASS` as a four-byte `int`. Reading
* the wrong one overreads the control buffer, so `parse` branches on the length
* the kernel actually wrote rather than on the family.
*/
export struct ecn {
using value_type = ecn_codepoint;

static constexpr std::size_t space{std::max(detail::space_for(sizeof(std::uint8_t)), detail::space_for(sizeof(int)))};
struct tos_message {
static constexpr std::size_t space{std::max(space_for(sizeof(std::uint8_t)), space_for(sizeof(int)))};

[[nodiscard]] static bool matches(const int level, const int type) noexcept {
return (level == IPPROTO_IP && type == IP_TOS) || (level == IPPROTO_IPV6 && type == IPV6_TCLASS);
}

[[nodiscard]] static std::optional<value_type> parse(const ::cmsghdr* c) noexcept {
const auto width{detail::payload_size(c)};
[[nodiscard]] static std::optional<std::uint8_t> parse(const ::cmsghdr* c) noexcept {
const auto width{payload_size(c)};
if (width >= sizeof(int)) {
return from_traffic_class(detail::read_payload<int>(c));
return static_cast<std::uint8_t>(read_payload<int>(c));
}
if (width >= sizeof(std::uint8_t)) {
return from_traffic_class(detail::read_payload<std::uint8_t>(c));
return read_payload<std::uint8_t>(c);
}
return std::nullopt;
}

/**
* @brief Mark an outgoing datagram.
* @brief Written as an `int` for both families; the kernel accepts either width for `IP_TOS`.
*
* Written at the width each family expects, mirroring the receive side.
* The kernel takes the value as the datagram's whole traffic-class byte,
* replacing any DSCP the socket was given with `setsockopt`.
*/
[[nodiscard]] static std::size_t build(::cmsghdr* dst, const value_type value, const family fam) noexcept {
const int bits{static_cast<int>(std::to_underlying(value))};
[[nodiscard]] static std::size_t build(::cmsghdr* dst, const std::uint8_t byte, const family fam) noexcept {
const int value{byte};
if (fam == family::inet4) {
return detail::write_message(dst, IPPROTO_IP, IP_TOS, bits);
return write_message(dst, IPPROTO_IP, IP_TOS, value);
}
return detail::write_message(dst, IPPROTO_IPV6, IPV6_TCLASS, bits);
return write_message(dst, IPPROTO_IPV6, IPV6_TCLASS, value);
}

/**
Expand All @@ -177,21 +223,56 @@ export struct ecn {
* `IPV6_TCLASS`. The option is accepted on a v6-only socket too.
*/
[[nodiscard]] static result<> enable(const int fd, const family fam) noexcept {
const auto report_tos = [fd] { return detail::set_option(fd, IPPROTO_IP, IP_RECVTOS, 1); };
const auto report_tos = [fd] { return set_option(fd, IPPROTO_IP, IP_RECVTOS, 1); };
if (fam == family::inet4) {
return report_tos();
}
return detail::set_option(fd, IPPROTO_IPV6, IPV6_RECVTCLASS, 1) | then(report_tos);
return set_option(fd, IPPROTO_IPV6, IPV6_RECVTCLASS, 1) | then(report_tos);
}
};

} // namespace detail

private:
[[nodiscard]] static constexpr ecn_codepoint from_traffic_class(const std::integral auto tclass) noexcept {
return static_cast<ecn_codepoint>(static_cast<std::uint8_t>(tclass) & IPTOS_ECN_MASK);
/**
* @brief The ECN bits of a received datagram. Receive-only.
*
* Not sendable on purpose: the kernel takes a per-datagram traffic class as the
* whole byte, so marking ECN alone would zero the DSCP. Send `traffic_class`.
*/
export struct ecn {
using value_type = ecn_codepoint;

static constexpr std::size_t space{detail::tos_message::space};

[[nodiscard]] static bool matches(const int level, const int type) noexcept { return detail::tos_message::matches(level, type); }

[[nodiscard]] static std::optional<value_type> parse(const ::cmsghdr* c) noexcept {
return detail::tos_message::parse(c).transform([](const std::uint8_t byte) { return marking::from_byte(byte).ecn; });
}

[[nodiscard]] static result<> enable(const int fd, const family fam) noexcept { return detail::tos_message::enable(fd, fam); }
};

/** @brief The whole traffic class, DSCP and ECN, in both directions. */
export struct traffic_class {
using value_type = marking;

static constexpr std::size_t space{detail::tos_message::space};

[[nodiscard]] static bool matches(const int level, const int type) noexcept { return detail::tos_message::matches(level, type); }

[[nodiscard]] static std::optional<value_type> parse(const ::cmsghdr* c) noexcept { return detail::tos_message::parse(c).transform(marking::from_byte); }

[[nodiscard]] static std::size_t build(::cmsghdr* dst, const value_type& value, const family fam) noexcept {
return detail::tos_message::build(dst, value.byte(), fam);
}

[[nodiscard]] static result<> enable(const int fd, const family fam) noexcept { return detail::tos_message::enable(fd, fam); }
};

static_assert(cmsg_feature<pktinfo> && sendable_feature<pktinfo> && receivable_feature<pktinfo>);
static_assert(cmsg_feature<ecn> && sendable_feature<ecn> && receivable_feature<ecn>);
static_assert(receivable_feature<ecn> && !sendable_feature<ecn>, "sending the ECN bits alone would zero the DSCP; send traffic_class");
static_assert(receivable_feature<traffic_class> && sendable_feature<traffic_class>);

/* ============================================================================
* Enabling a whole set at open time
Expand Down
Loading
Loading