event_loop_blocked can emit a parentless span when the stall is sampled in job_task after session close #7646

Summary

An event_loop_blocked span can be emitted with no parent, becoming a stray root, when the stall is sampled in the outer job_task after the agent session has closed. The documented third-tier fallback in _emit_span — "else the report context (job_entrypoint)" — does not appear to be reached in this path.

Introduced by #7128 (1ac4183fc), so this is main-only; loop_monitor.py is not present in 1.8.4 or earlier.

Observed

In one 579 s voice session on main (8a677a62), 5 event_loop_blocked spans were emitted. 4 parented correctly. The 5th did not:

The orphan fired at t+561.18 s, in the same millisecond agent_session and session_close ended (t+561.17 s), and ~18 s before job_entrypoint itself ended (t+579.13 s). So the root span was still open and still a valid parent at the time.

Why the fallback chain doesn't cover it

_emit_span (telemetry/loop_monitor.py:446) selects a parent in three tiers, per its own comment — "under the span the blocked task was in, else the session root, else the report context (job_entrypoint)":

if report.parent_span_context is not None:
    parent = trace.set_span_in_context(trace.NonRecordingSpan(report.parent_span_context))
else:
    parent = session_context.session_root_context()
span = tracer.start_span(SPAN_NAME, context=parent, start_time=start_ns, attributes=attributes)

Tier 1 is unavailable for job_task. _run_job_task is created at ipc/job_proc_lazy_main.py:323, and only the inner job_user_entrypoint task (:359) has its body wrapped in tracer.use_span(job_span, ...). The outer task is what creates job_span, never runs inside it, so a stall sampled there has no task span to inherit.

Tier 2 is cleared at session close. voice/agent_session.py:1356 sets self._root_span_context = None, so session_root_context() returns None from that point on.

Tier 3 then appears not to fire. With parent = None, start_span(context=None) should resolve to the ambient context, and _emit runs inside self._report_context.run(_emit) whose snapshot was taken inside use_span(job_span) — so job_entrypoint should be current. The emitted span nonetheless has no parent. I have not pinned down why, and did not want to guess at a patch.

Note that event_loop_blocked is the only span in the system whose parentage is reconstructed rather than inherited — it originates on the watchdog thread, which has no ambient OTel context of its own. That is what set_report_context(contextvars.copy_context()) at job_proc_lazy_main.py:347 exists for, and it is why this path has no natural backstop.

Impact

Low. At most one diagnostic span per session, and it still carries the same trace id, so nothing else is detached. But a backend that groups by trace membership renders it as a separate root, which is noise in exactly the view #7533 set out to clean up.

Possible directions

Offered as suggestions rather than a diagnosis, since I could not confirm the tier-3 failure:

1. In _emit_span, fall through to the ambient/report context when session_root_context() yields no span with a valid context, instead of passing its result as context= unconditionally — this makes the documented tier 3 actually reachable.
2. Wrap _run_job_task's body in use_span(job_span) so tier 1 also covers stalls sampled in the outer job task.
3. Alternatively, skip emitting when no valid parent can be resolved — _emit_span already returns early when there is no job context (loop_monitor.py:442), on the same reasoning that "a root span here would be a stray trace".

Environment

• livekit-agents main @ 8a677a62, Python 3.13
• OTLP/HTTP export, voice (cascade) pipeline, single session per job process