Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This is a comprehensive knowledge base built with **MkDocs Material** that conta

- **Static Site Generator**: MkDocs Material
- **Content Format**: Markdown with extensions
- **Dependency Management**: Poetry (Python 3.11+)
- **Dependency Management**: uv (Python 3.11+)
- **Deployment**: GitHub Actions → GitHub Pages
- **Code Quality**: Pre-commit hooks

Expand Down Expand Up @@ -84,7 +84,7 @@ function_call()

1. **Clone Repository**: Use standard git clone workflow

1. **Install Dependencies**: Use Poetry for dependency management
1. **Install Dependencies**: Use uv for dependency management

1. **Start Development Server**: Use MkDocs serve command for local development

Expand Down Expand Up @@ -213,7 +213,7 @@ When helping with this project:

### Commands for Debugging

Use the appropriate Poetry and MkDocs commands for:
Use the appropriate uv and MkDocs commands for:

- Configuration validation
- Verbose build output
Expand Down
24 changes: 5 additions & 19 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,31 +23,17 @@ jobs:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
enable-cache: true
python-version: '3.11'

- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}

- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
run: uv sync --locked

- name: Build MkDocs site
run: poetry run mkdocs build --clean
run: uv run mkdocs build --clean

- name: Upload artifact
uses: actions/upload-pages-artifact@v3
Expand Down
10 changes: 4 additions & 6 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,12 @@ jobs:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
enable-cache: true
python-version: '3.11'

- name: Install pre-commit
run: pip install pre-commit

- name: Cache pre-commit environments
uses: actions/cache@v3
with:
Expand All @@ -30,4 +28,4 @@ jobs:
pre-commit-${{ runner.os }}-

- name: Run pre-commit
run: pre-commit run --all-files
run: uvx pre-commit run --all-files
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,13 @@ Organized by area of interest, with a focus on machine learning and AI:
1. **Install dependencies**:

```bash
poetry install
uv sync
```

1. **Start the development server**:

```bash
poetry run mkdocs serve
uv run mkdocs serve
```

1. **Open your browser** to `http://127.0.0.1:8000`
Expand Down
4 changes: 2 additions & 2 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,8 @@ of just a pile of independent files.
## 7. Preview Locally Before Committing

```bash
poetry install
poetry run mkdocs serve
uv sync
uv run mkdocs serve
```

Then open `http://127.0.0.1:8000` — the site reloads automatically as you edit. Worth
Expand Down
12 changes: 3 additions & 9 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,26 +39,20 @@ To preview changes before publishing:
### Prerequisites

- Python 3.11+
- Poetry installed
- [uv](https://docs.astral.sh/uv/) installed

### Steps

1. **Install dependencies**:

```bash
poetry install
```

1. **Activate the virtual environment**:

```bash
poetry shell
uv sync
```

1. **Start the development server**:

```bash
mkdocs serve
uv run mkdocs serve
```

1. **Open your browser** to `http://127.0.0.1:8000`
Expand Down
3 changes: 2 additions & 1 deletion docs/troubleshooting/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,4 +92,5 @@ or just "now I know to look here first."
## Entries

- [Browser Keeps Prompting to Log In on Every Article (polyfill.io)](polyfill-io-login-prompt.md)
- [WiFi Connection Collapses Only When Streaming to a LAN Device (Expo, Steam Link)](wifi-bufferbloat-lan-streaming-drops.md)
- [WiFi Bufferbloat Collapses the Connection When Streaming to a LAN Device (Expo, Steam Link)](wifi-bufferbloat-lan-streaming-drops.md)
- [WiFi Card Drops Connection Under Load Due to PCIe Power Management (mt7925e Forced Reassociation)](wifi-card-pcie-power-reassociation-drops.md)
162 changes: 59 additions & 103 deletions docs/troubleshooting/wifi-bufferbloat-lan-streaming-drops.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,20 @@ tags:

---

# WiFi Connection Collapses Only When Streaming to a LAN Device (Expo, Steam Link)
# WiFi Bufferbloat Collapses the Connection When Streaming to a LAN Device (Expo, Steam Link)

Internet felt "unstable" only in specific situations — running an Expo dev server for a
phone to connect to, or streaming games to a phone via Steam Link. Regular browsing was
fine most of the time. Turned out to be **two separate, compounding bugs**, both only
triggered by sustained local WiFi throughput: classic **bufferbloat** (no active queue
management on the wireless interface, so a big transfer buried everything else behind
it), and a **driver/firmware instability** where the WiFi chipset forced its own
reassociation under load and sometimes came back degraded. Fixing the first made things
much better; the second explains why it wasn't perfect until both were addressed.
fine most of the time. Turned out to be classic **bufferbloat**: no active queue
management on the wireless interface, so a big sustained transfer buried everything else
(pings, DNS, keepalives) behind it.

!!! note
This machine also hit a second, unrelated WiFi instability during the same
investigation — see
[WiFi Card Drops Connection Under Load Due to PCIe Power Management](wifi-card-pcie-power-reassociation-drops.md).
That bug persisted even after this bufferbloat fix was applied, so don't assume
fixing bufferbloat alone rules out further WiFi instability.

## Environment

Expand Down Expand Up @@ -196,102 +200,6 @@ tc qdisc show dev wlp4s0

Swap `wlp4s0` for your own interface name (`ip -br addr` to find it) if different.

## A Second, Separate Bug: PCIe Power Management Confusing the WiFi Firmware

`fq_codel` fixed the bufferbloat collapse, but a follow-up Expo session still hit a real
outage — not jitter this time, actual `Destination Host Unreachable` for ~13 seconds,
followed by a kernel-logged forced reassociation:

```text
wlp4s0: deauthenticating from 2e:70:4f:18:e8:d8 by local choice (Reason: 3=DEAUTH_LEAVING)
wlp4s0: authenticate with 2e:70:4f:18:e8:d8 (try 1/3)
wlp4s0: associated
```

The radio recovered for about 17 seconds (clean 1-4ms pings), then degraded back into
sustained heavy jitter (100ms-1500ms) that didn't fully resolve for the rest of the
session — worse than plain bufferbloat, and `tc qdisc show` confirmed `fq_codel` was
still active throughout, so queue management wasn't the issue this time.

### Root Cause (Second Bug)

`by local choice` means the *driver* initiated the disconnect, not the router. This is a
known class of instability on the MediaTek `mt7925e` chipset (and several other WiFi
chipsets): under sustained load, the driver/firmware can get confused by the PCIe link
transitioning through power-saving states (ASPM L0s/L1) or by PCIe runtime autosuspend
kicking in, and forces a reassociation to recover — sometimes coming back in a degraded
state rather than cleanly.

```bash
$ cat /sys/module/pcie_aspm/parameters/policy
default [performance] powersave powersupersave # was previously "default" (active)

$ cat /sys/bus/pci/devices/0000:04:00.0/power/control
on # was "auto" (runtime autosuspend allowed)
```

Checked first whether a newer firmware fixed it — it didn't apply here: `linux-firmware`
was already at Ubuntu's latest packaged version, and `fwupdmgr` doesn't manage this
chip's firmware (it ships as part of the kernel firmware package, not as a
`fwupd`-updatable device). So the fix targets the power-management trigger directly
instead of waiting on an upstream firmware update.

### Fix (Second Bug)

**Important — scope matters here.** `/sys/module/pcie_aspm/parameters/policy` is a
**global** kernel setting: it affects every PCIe device (NVMe SSD, GPU, everything), not
just WiFi. Setting it to `performance` system-wide measurably increases idle power draw
and heat (worse battery life, more fan noise), especially with a discrete GPU in the
picture. Prefer the WiFi-scoped version below unless the machine is always plugged in.

**Recommended — scoped to the WiFi card only:**

```bash
# Disable PCIe runtime autosuspend for just the WiFi device
sudo sh -c 'echo on > /sys/bus/pci/devices/0000:04:00.0/power/control'

# Disable ASPM (L0s/L1 low-power link states) on just the WiFi device's PCIe link,
# by clearing bits 0-1 of its Link Control register (CAP_EXP+0x10) — leaves every
# other PCIe device's power management untouched
sudo setpci -s 04:00.0 CAP_EXP+10.w=0000:0003
```

Verify:

```bash
$ cat /sys/bus/pci/devices/0000:04:00.0/power/control
on

$ sudo setpci -s 04:00.0 CAP_EXP+10.w
0040 # low nibble even (0, 4, 8, or c) = ASPM bits clear = disabled
```

**To reverse (restore default power-saving behavior on the WiFi card):**

```bash
sudo sh -c 'echo auto > /sys/bus/pci/devices/0000:04:00.0/power/control'
sudo setpci -s 04:00.0 CAP_EXP+10.w=0003:0003
```

**If a global test is wanted instead** (simpler, but see the battery/heat caveat above):

```bash
# Apply
sudo sh -c 'echo performance > /sys/module/pcie_aspm/parameters/policy'

# Reverse
sudo sh -c 'echo default > /sys/module/pcie_aspm/parameters/policy'
```

Swap `0000:04:00.0` / `04:00.0` for your own WiFi device's PCI address
(`lspci | grep -i network`) if different.

Both the scoped commands above are **runtime-only** — they reset on reboot. Persisting
them (e.g. via a `systemd-tmpfiles` rule or a small `systemd` oneshot service run at
boot, since this isn't tied to a NetworkManager interface-up event the way `fq_codel`
is) is a reasonable next step once confirmed stable over real use, but hold off making
it permanent until it's proven itself across more than one session.

## Prevention / How to Recognize This Class of Bug Again

```mermaid
Expand Down Expand Up @@ -325,3 +233,51 @@ flowchart TD
bandwidth that isn't there. Persistent jitter even with fq_codel active is a sign to
also check channel congestion (`nmcli -f SSID,SIGNAL,CHAN dev wifi list`) and consider
moving the AP to a less congested channel.

## Beyond This Fix: Bufferbloat Further Upstream (Router / WAN Link)

Fixing the local WiFi interface's qdisc only manages queuing *at that hop*. The same
class of problem can also exist further out — the router's WAN uplink queue, the ISP's
modem, or within the ISP's own network — and a local `fq_codel` fix on the laptop won't
touch that.

A run of a bufferbloat test (via the tools linked from
[bufferbloat.net](https://www.bufferbloat.net/projects/)) came back **Grade C (~170ms
added latency under load)**, meaning load on the WAN link adds meaningfully to ping — the
same underlying signature as the WiFi bufferbloat above, just one or more hops further
out.

!!! note
This hasn't been root-caused yet — the directions below are where to look next, not
a confirmed fix. The rest of this article covers the local WiFi fix only; treat
WAN-side bufferbloat as a separate, still-open problem.

- **Isolate direction first.** Most bufferbloat test tools grade upload and download
separately. Upload-side bufferbloat is usually the router pushing packets to the modem
faster than the real uplink can drain them; download-side is usually a queue on the
ISP's own equipment (modem/CMTS/ONT) filling up before packets even reach the router.
- **Retest over a wired connection to the router** (bypass WiFi entirely) to confirm the
WAN-level grade isn't partly a leftover of the already-fixed WiFi issue, before
assuming the router/WAN link itself is the culprit.
- **Smart Queue Management (SQM) on the router** is the standard fix for this class:
shape traffic at the router to slightly under the real WAN bandwidth (both directions)
using a modern AQM (`cake` or `fq_codel`), so the router's own queue absorbs and
manages the backlog instead of an unmanaged one further upstream. On OpenWrt, that's
the `sqm-scripts` package (`cake` qdisc); on pfSense/OPNsense, the Traffic
Shaper/Limiters with CoDel; some consumer routers expose it as "Smart Queue
Management" or "Adaptive QoS" in their UI.
- **If SQM doesn't close the gap**, the queuing may be happening on the ISP's own
equipment, which a home router can't manage — that points toward the ISP itself rather
than anything fixable locally.

## Further Reading

- [Bufferbloat.net Projects](https://www.bufferbloat.net/projects/) — background on
bufferbloat, the tools/projects behind `fq_codel`/`cake`, and ongoing work on the
problem beyond this one interface-level fix.

## Related Articles

- [WiFi Card Drops Connection Under Load Due to PCIe Power Management](wifi-card-pcie-power-reassociation-drops.md) —
a separate, unrelated WiFi instability found on the same machine during this
investigation; don't assume this fix rules it out.
Loading
Loading