Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/embeddedci-mcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ transparently, so an idle chat never blocks CI on that pod. `status` shows the p

| Flag | Environment | Default | |
| --- | --- | --- | --- |
| `--connection` | `BENCHPOD_CONNECTION` | the CLI's saved connection, else mDNS | host[:port], serial device, `usb`, `usb:<name>`, `discover`, `saved`, or `embeddedci:<device>`; unset = `benchpod set-connection`'s saved one, else find the one pod on the LAN |
| `--connection` | `BENCHPOD_CONNECTION` | the CLI's saved connection, else mDNS | host[:port], a pod name (`benchpod-a1b2c3`: mDNS, else USB), serial device, `usb`, `usb:<name>`, `discover`, `saved`, or `embeddedci:<device>`; unset = `benchpod set-connection`'s saved one, else find the one pod on the LAN |
| `--la-voltage` | `BENCHPOD_LA_VOLTAGE` | — | LA I/O voltage (1.8 or 3.3; 5.0 on a digital v3r4 pod) applied on connect |
| `--wiring-autosave` | `BENCHPOD_WIRING_AUTOSAVE` | off | `set_wiring` also stores the profile on the pod (not a change of the I/O voltage, power rail or SWD target: those need `save_wiring`) |
| — | `BENCHPOD_API_KEY` | — | cloud pods and the waveform library (without it, cloud tools use the `benchpod login` session) |
Expand Down
15 changes: 15 additions & 0 deletions packages/embeddedci/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
# Changelog — `embeddedci`

## 4.0.2, not released yet

- **Connecting by host name is fast again.** The SDK opens a new connection per command and
looked the host name up every time. On macOS a default lookup of a `.local` name waits about
5 s for an IPv6 answer an IPv4-only pod never sends, so `BenchPod("benchpod-c2f662.local")` took
about 50 s (10 lookups) against 0.8 s by IP address. Now a transport looks a name up once,
IPv4 first (any address family only when there is no IPv4 answer), and reuses the address. A
connect that fails looks the name up again, so a pod that gets a new DHCP lease is still found.
The TLS pin in `known_pods.json` stays keyed by the name you gave, and the SNI is unchanged.
- **A pod name works as a connection.** `BenchPod("benchpod-c2f662")` (or `"c2f662"`) finds the
pod on the LAN over mDNS (`[discovery]` extra), else on USB (`usb:benchpod-c2f662`), as the
`benchpod` CLI does. Without `[discovery]` it is the host name `benchpod-c2f662.local`.
`benchpod-c2f662.local` stays a host name. New: `discovery.find_pod(short)`, which stops
browsing as soon as that pod answers.

## 4.0.1

For BenchPod firmware 4.0.1. Works with firmware 4.0.0 too.
Expand Down
1 change: 1 addition & 0 deletions packages/embeddedci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,7 @@ explicitly. The pytest plugin does **not** fall back to it (see [pytest plugin](
| `192.168.1.213`, `benchpod-baea06.local`, `[fe80::1]` | network: JSON over TLS on port 8443 (see [TLS on the LAN](#tls-on-the-lan)); an address on the pod's USB network adapter (`10.237.x.y`) is plaintext 8080 |
| `host:8080`, `host:8443` | network on that port: `:8080` is plaintext (refused on Ethernet and Wi-Fi unless the pod's HTTPS-only is off), `:8443` TLS |
| `/dev/ttyACM0`, `COM3`, `\\.\COM10` | USB console, explicit device path |
| `benchpod-baea06`, `baea06` | the pod with that name (as on its label and in `discover`): its address found on the LAN over mDNS, else `usb:benchpod-baea06`. Without `[discovery]` it is the host name `benchpod-baea06.local` |
| `usb` | USB pod, auto-detected by probing the serial ports: its USB network adapter (full API), else its USB console (status only) |
| `usb:<name>` | the USB pod with that name when several are plugged into one computer: `usb:baea06`, `usb:benchpod-baea06` (any case, optional `.local`); otherwise as `usb` |
| `discover` (or `mdns`, `auto`) | find exactly one pod on the LAN via mDNS (needs `[discovery]`); errors on zero or several |
Expand Down
4 changes: 3 additions & 1 deletion packages/embeddedci/src/embeddedci/benchpod/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,9 @@ def __init__(
"""Open a BenchPod.

``connection`` is a host[:port], a serial device path, ``"usb"`` (auto-detect the USB
pod), ``"usb:<name>"`` (one of several USB pods, by name), ``"discover"`` (find one pod on the LAN via mDNS), or ``"embeddedci:<device>"``
pod), ``"usb:<name>"`` (one of several USB pods, by name), a pod name (``"benchpod-a1b2c3"``:
found on the LAN over mDNS, else on USB), ``"discover"`` (find one pod on the LAN via
mDNS), or ``"embeddedci:<device>"``
to drive a named device through embeddedci.com. When omitted ``BENCHPOD_CONNECTION`` is
used, else the default connection ``benchpod-cli`` saved (``benchpod set-connection``);
``"saved"`` names that one explicitly. Pass ``transport`` to inject a custom backend
Expand Down
48 changes: 39 additions & 9 deletions packages/embeddedci/src/embeddedci/benchpod/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
:func:`~embeddedci.benchpod.transport.open_transport`)
* ``usb:<name>`` -> the USB pod with that name, when several are plugged in
(``usb:baea06``, ``usb:benchpod-baea06``)
* ``benchpod-a1b2c3`` / ``a1b2c3`` -> the pod with that name: its address found on the LAN over
mDNS, else ``usb:<name>`` (as the ``benchpod`` CLI does).
``benchpod-a1b2c3.local`` stays a host name.
* ``discover`` / ``mdns`` / ``auto`` -> find a single pod on the LAN via mDNS
* ``embeddedci:<device-name>`` -> drive a named device through embeddedci.com (an API key
anywhere, or GitHub Actions OIDC)
Expand Down Expand Up @@ -43,6 +46,10 @@
USB_PREFIX = "usb:"

_COM_RE = re.compile(r"^COM[0-9]+$", re.IGNORECASE)
#: A pod's name as on its label and in ``discover``: ``benchpod-a1b2c3`` or ``a1b2c3`` (its identity
#: short id). Only this exact shape is a pod name; ``benchpod-a1b2c3.local`` and every other host
#: name go to the OS resolver (the pin of a ``.local`` name stays keyed by that name).
_POD_NAME_RE = re.compile(r"^(?:benchpod-)?[0-9a-f]{6}$", re.IGNORECASE)


@dataclass
Expand Down Expand Up @@ -140,9 +147,40 @@ def parse_connection(raw: str) -> ConnSpec:
return _discover_one()
if _is_device_path(s):
return ConnSpec(kind="serial", device=s)
if _POD_NAME_RE.match(s):
return _find_named_pod(normalize_pod_name(s))
return ConnSpec(kind="tcp", addr=_normalize_addr(s))


def _find_named_pod(short: str) -> ConnSpec:
"""The pod named ``benchpod-<short>``: its address when it answers on the LAN over mDNS, else
``usb:<short>``, which finds it on USB. Without the ``discovery`` extra (zeroconf) it is the
host name ``benchpod-<short>.local``, left to the OS resolver."""
from .discovery import find_pod

try:
pod = find_pod(short)
except RuntimeError: # no zeroconf
return ConnSpec(kind="tcp", addr=_normalize_addr(f"benchpod-{short}.local"))
if pod is None:
return ConnSpec(kind="serial", device="", pod=short)
_refuse_firmware_3(pod)
return ConnSpec(kind="tcp", addr=_normalize_addr(pod.addr))


def _refuse_firmware_3(pod) -> None: # noqa: ANN001 - a DiscoveredPod
if pod.supported:
return
from .errors import FirmwareTooOldError
from .feature_map import MIN_FIRMWARE

raise FirmwareTooOldError(
f"the BenchPod found on the LAN ({pod.name}, {pod.hostname}) announces no client API "
f"2: it runs firmware 3.x, and this SDK needs firmware {MIN_FIRMWARE} or later. Update "
"it with the BenchPod CLI (`benchpod flash-self`) or the firmware update in the web "
"app, or use embeddedci 3.x with it", api=pod.api or 1)


def _discover_one() -> ConnSpec:
"""Resolve a single LAN pod via mDNS, or raise a helpful error.

Expand All @@ -167,15 +205,7 @@ def _discover_one() -> ConnSpec:
f"hostname (e.g. {pods[0].hostname}):\n{listing}"
)
pod = pods[0]
if not pod.supported:
from .errors import FirmwareTooOldError
from .feature_map import MIN_FIRMWARE

raise FirmwareTooOldError(
f"the BenchPod found on the LAN ({pod.name}, {pod.hostname}) announces no client API "
f"2: it runs firmware 3.x, and this SDK needs firmware {MIN_FIRMWARE} or later. Update "
"it with the BenchPod CLI (`benchpod flash-self`) or the firmware update in the web "
"app, or use embeddedci 3.x with it", api=pod.api or 1)
_refuse_firmware_3(pod)
return ConnSpec(kind="tcp", addr=_normalize_addr(pod.addr))


Expand Down
35 changes: 31 additions & 4 deletions packages/embeddedci/src/embeddedci/benchpod/discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@
from __future__ import annotations

import ipaddress
import time
import threading
from dataclasses import dataclass, field
from typing import Dict, List
from typing import Dict, List, Optional

#: DNS-SD service type the firmware advertises.
SERVICE_TYPE = "_benchpod._tcp.local."
Expand Down Expand Up @@ -96,6 +96,30 @@ def discover(timeout: float = DEFAULT_TIMEOUT) -> List[DiscoveredPod]:
Raises :class:`RuntimeError` with an install hint if ``zeroconf`` is not
installed. Returns an empty list when nothing answers.
"""
return sorted(_browse(timeout).values(), key=lambda p: p.name)


def names_pod(pod: DiscoveredPod, short: str) -> bool:
"""Whether ``pod`` is the pod with the short id ``short`` (``"a1b2c3"``), by its DNS-SD
instance (``"BenchPod a1b2c3"``) or host name (``"benchpod-a1b2c3.local"``)."""
from .connection import normalize_pod_name

return normalize_pod_name(pod.name) == short or normalize_pod_name(pod.hostname) == short


def find_pod(short: str, timeout: float = DEFAULT_TIMEOUT) -> Optional[DiscoveredPod]:
"""Browse the LAN for the pod with the short id ``short`` (``"a1b2c3"``), stopping as soon as
it answers; None when it does not within ``timeout`` seconds. Raises :class:`RuntimeError`
without ``zeroconf``, like :func:`discover`."""
for pod in _browse(timeout, want=short).values():
if names_pod(pod, short) and pod.addresses:
return pod
return None


def _browse(timeout: float, want: str = "") -> Dict[str, DiscoveredPod]:
"""The pods heard within ``timeout`` seconds, by service name; with ``want`` (a short id) the
browse ends as soon as that pod answers with an address."""
try:
from zeroconf import ServiceBrowser, Zeroconf
except ImportError as exc: # pragma: no cover - exercised via install extra
Expand All @@ -105,6 +129,7 @@ def discover(timeout: float = DEFAULT_TIMEOUT) -> List[DiscoveredPod]:
) from exc

found: Dict[str, DiscoveredPod] = {}
done = threading.Event()
zc = Zeroconf()

def _record(zeroconf, type_: str, name: str) -> None:
Expand All @@ -129,6 +154,8 @@ def _record(zeroconf, type_: str, name: str) -> None:
api=_txt_int(txt, "api"),
https_only=txt.get("https_only", "") == "1",
)
if want and found[name].addresses and names_pod(found[name], want):
done.set()

class _Listener:
def add_service(self, zeroconf, type_, name):
Expand All @@ -142,8 +169,8 @@ def remove_service(self, zeroconf, type_, name):

ServiceBrowser(zc, SERVICE_TYPE, _Listener())
try:
time.sleep(timeout)
done.wait(timeout)
finally:
zc.close()

return sorted(found.values(), key=lambda p: p.name)
return dict(found)
71 changes: 69 additions & 2 deletions packages/embeddedci/src/embeddedci/benchpod/transport/tcp.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,16 @@
reads the reply. ``dap_start`` is special — after its ack the same socket
switches to a raw length-framed CMSIS-DAP stream, so the ack must be read one
byte at a time so no DAP bytes are swallowed.

A host name is resolved once per transport (IPv4 first, see :func:`resolve_host`) and the address
is reused for the following connections; a connect that fails resolves the name again, so a pod
that got a new DHCP lease is still found. TLS still uses the name as given: the certificate pin is
keyed by it, and it is the SNI for a name.
"""

from __future__ import annotations

import ipaddress
import json
import os
import socket
Expand Down Expand Up @@ -53,6 +59,35 @@ def _io_errors(what: str, timeout: Optional[float]) -> Iterator[None]:
except OSError as exc:
raise TransportError(f"{what}: {exc}") from exc


def _is_ip_literal(host: str) -> bool:
try:
ipaddress.ip_address(host)
return True
except ValueError:
return False


def resolve_host(host: str, port: int) -> List[str]:
"""The addresses of ``host``, IPv4 first: an ``AF_INET`` lookup, and an ``AF_UNSPEC`` one only
when that has no answer. A default lookup of a ``.local`` name waits about 5 s on macOS for an
AAAA answer an IPv4-only pod never sends; the ``AF_INET`` one takes milliseconds."""
try:
infos = socket.getaddrinfo(host, port, socket.AF_INET, socket.SOCK_STREAM)
except socket.gaierror:
infos = []
if not infos:
infos = socket.getaddrinfo(host, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
out: List[str] = []
for info in infos:
ip = str(info[4][0])
if ip not in out:
out.append(ip)
if not out:
raise socket.gaierror(f"{host}: no address")
return out


class _SocketRawLink:
"""Adapts a connected socket to :class:`RawLink` for the flash bridge."""

Expand Down Expand Up @@ -119,6 +154,8 @@ def __init__(self, addr: str, *, timeout: float = 30.0,
self.dial_timeout = dial_timeout
self.tls = _tls.tls_mode(tls)
self.tls_ca = tls_ca if tls_ca is not None else (os.environ.get("BENCHPOD_TLS_CA") or None)
#: Host name -> its resolved addresses, reused until a connect to them fails.
self._resolved: Dict[str, List[str]] = {}

# -- connection helpers -------------------------------------------------

Expand Down Expand Up @@ -161,12 +198,14 @@ def _dial(self, timeout: Optional[float] = None, kind: Optional[str] = None) ->
def _connect(self, addr: "tuple[str, int]", budget: float, io_timeout: Optional[float],
*, retry: bool = True,
attempt_timeout: float = _DIAL_ATTEMPT_TIMEOUT) -> socket.socket:
"""A TCP connection to ``addr``, re-probing within ``budget`` seconds."""
"""A TCP connection to ``addr``, re-probing within ``budget`` seconds. A host name is
resolved once (:meth:`_addresses`) and resolved again only after a failed connect."""
deadline = time.monotonic() + budget
last: Optional[Exception] = None
host, port = addr
while True:
try:
sock = socket.create_connection(addr, timeout=attempt_timeout)
sock = self._connect_resolved(host, port, attempt_timeout)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
sock.settimeout(io_timeout)
return sock
Expand All @@ -181,6 +220,34 @@ def _connect(self, addr: "tuple[str, int]", budget: float, io_timeout: Optional[
"(TLS on 8443), or over USB")
raise TransportError(f"could not connect to {addr[0]}:{addr[1]}: {last}{hint}") from last

def _addresses(self, host: str) -> List[str]:
"""The addresses to connect to for ``host``: the literal itself, else the cached
resolution, else a new one (IPv4 first, :func:`resolve_host`)."""
if _is_ip_literal(host):
return [host]
ips = self._resolved.get(host)
if ips is None:
ips = resolve_host(host, 0)
self._resolved[host] = ips
return ips

def _connect_resolved(self, host: str, port: int, attempt_timeout: float) -> socket.socket:
"""One connect attempt to each address of ``host`` in turn. When none answers the cached
resolution is dropped, so the next attempt resolves the name again (a new DHCP lease)."""
last: Optional[OSError] = None
try:
for ip in self._addresses(host):
try:
return socket.create_connection((ip, port), timeout=attempt_timeout)
except OSError as exc:
last = exc
except OSError: # the lookup itself failed
self._resolved.pop(host, None)
raise
self._resolved.pop(host, None)
assert last is not None
raise last

@staticmethod
def _recv_line(sock: socket.socket, buf: bytearray) -> bytes:
"""Read one newline-terminated line, buffering any overshoot in ``buf``."""
Expand Down
Loading