Skip to content

Add a CPython test suite and coverage reporting - #115

Merged
mikeysklar merged 1 commit into
adafruit:mainfrom
dhalbert:dhalbert-tests
Nov 20, 2026
Merged

mikeysklar merged 1 commit into
adafruit:mainfrom
dhalbert:dhalbert-tests

Conversation

@dhalbert

Copy link
Copy Markdown
Contributor

Tests and coverage setup written by Claude, reviewed by @dhalbert.

This library had no tests. This PR adds a pytest suite that runs on CPython, and reports test coverage in CI.

This PR does not change the library itself. The test generation found issues which will be fixed separately.

Tests

  • tests/ has tests for every module. Most Server tests use real sockets on 127.0.0.1, with routes based on the examples. Error paths that are hard to cause with real sockets (EAGAIN, ECONNRESET, timeouts, partial sends) use in-memory fake sockets.
  • The standard Build CI workflow already runs python -m pytest when a tests/ directory exists, so build.yml is unchanged.
  • tests/README.md explains how to run the tests and how they are organized.

Known bugs and open questions

The library has a bunch of behavior that looks wrong now. The problems are marked rather than asserted as correct, so that the tests are present but don't cause the CI to fail. We'll need to whittle this list down to zero through PRs.

  • 28 tests are marked xfail(strict=True). Each one asserts the correct behavior for a known bug. When the bug is fixed, the test passes, and strict=True makes CI fail until the marker is removed.
  • 6 tests are skipped. Each skip reason says whether it is a bug or a design question.
  • python -m pytest -rxs lists them all with their reasons.

The bugs most likely to affect users:

  • Form values sent as application/x-www-form-urlencoded are not decoded, so + and %XX escapes come back as sent.
  • A malformed request makes poll() raise instead of answering 400.
  • A plain-HTTP client connecting to an HTTPS server makes poll() raise SSLError.
  • On CPython, Websocket.receive() raises instead of closing when the client disconnects.
  • request.cookies raises ValueError on a cookie without =.

Coverage

  • .github/workflows/coverage.yml runs the tests under coverage and puts the report in the job summary. It is a separate workflow so that build.yml stays identical to the other libraries' copies.
  • .coveragerc turns on branch coverage and excludes code that cannot run on CPython, such as CircuitPython import fallbacks and CircuitPython-only SSL setup. A comment lists the lines still counted as missed on purpose: a path blocked by a known bug, and dead code.
  • Coverage is currently 99%.
  • .gitignore ignores .coverage, as requests, minimqtt and connection_manager already do.

- tests/: pytest suite for all modules, using real loopback sockets and
  in-memory fake sockets. Known bugs are strict xfails; open questions
  are skips with a reason. tests/README.md explains both.
- .coveragerc: branch coverage, exclusions for CircuitPython-only code,
  and a list of lines still counted as missed on purpose.
- .github/workflows/coverage.yml: runs the tests under coverage and puts
  the report in the job summary. Separate so build.yml stays standard.
- .gitignore: ignore .coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dhalbert
dhalbert requested a review from a team November 14, 2026 21:22

@mikeysklar mikeysklar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as advertised.

Check Result
5 runs, Python 3.14 334 passed, 6 skipped, 28 xfailed each time, under 1 s
5 random orders (pytest-randomly) same result each time
ResourceWarning (leaked sockets) none
Coverage 99%, branch coverage on
pre-commit ruff, ruff format, reuse pass
xfails with --runxfail all 28 fail for the bug their reason names

@mikeysklar
mikeysklar merged commit 5aba46f into adafruit:main Nov 20, 2026
2 checks passed
@dhalbert
dhalbert deleted the dhalbert-tests branch November 20, 2026 03:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants