feat(uprobe): implement uprobe breakpoint support (#2150 phase 1) - #2163
feat(uprobe): implement uprobe breakpoint support (#2150 phase 1)#2163sparkzky wants to merge 3 commits into
Conversation
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c8c8a99c5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
3a99e3d to
86f1f09
Compare
fslongjin
left a comment
There was a problem hiding this comment.
Thanks for the substantial work on this. Using a user-space XOL area and private breakpoint pages is the right general direction, and keeping uretprobe support out of this first phase is a reasonable scope choice.
However, I do not think this version is safe to merge yet. The following issues can change program behavior, break debugging, or cross process security boundaries.
-
pid == -1bypasses process access checks (kernel/src/perf/uprobe.rs, around line 234).For a non-negative pid, the code checks whether the caller may inspect the target process. For
pid == -1(and any value below-1), it skips that check and installs breakpoints in every process currently mapping the file. An unprivileged process could therefore modify code pages in processes owned by another user, attach BPF code, or crash those processes. Please reject invalid negative pids and apply an explicit privileged or per-process access check for any system-wide mode. Linux does not treatpid == -1as permission to modify every matching process. -
The active XOL operation is not kept alive until single-step completion (
kernel/src/mm/ucontext/uprobe.rs, around line 629;kernel/src/exception/uprobe.rs, around line 109).A thread can hit the breakpoint while another thread closes the perf fd. The close path can remove the probe and reuse its XOL slot before the first thread receives the debug exception. The first thread may then resume at the wrong address or remain in the XOL page. A per-thread active-probe object must hold the probe, original address, return address, XOL slot, and original trap flag until completion or abort. Probe removal must wait for active users before reusing the slot.
-
Signals and exceptions during XOL execution are not handled safely (
kernel/src/exception/uprobe.rs, around line 109).NEED_UPROBEis only a boolean. If a signal is delivered before the copied instruction runs, or if that instruction faults, a later unrelated debug exception can be mistaken for XOL completion and change the instruction pointer incorrectly. Linux keeps a per-thread uprobe state and has explicit complete and abort paths. DragonOS needs the same essential state, even if the implementation is simpler. -
Unrelated user debug exceptions are swallowed (
kernel/src/exception/uprobe.rs, around line 124).Every user-mode debug exception is sent to the uprobe handler. When
NEED_UPROBEis not set, the handler returns success without passing the exception to ptrace, hardware-breakpoint handling, or normalSIGTRAPdelivery. This breaks single-stepping and hardware breakpoints for all programs, including programs that do not use uprobes. The uprobe handler should report whether it consumed the exception; all other cases must continue through the normal debug path. -
The original trap flag is lost (
kernel/src/exception/uprobe.rs, around line 188).XOL completion always clears TF. If the program or debugger had already enabled single-step mode, hitting an uprobe silently disables it. Save the original TF value per thread, clear only a TF bit added by uprobe, and preserve the expected
SIGTRAPbehavior. -
Instructions crossing a page boundary are copied incorrectly (
kernel/src/mm/ucontext/uprobe.rs, around line 303).The read helper fetches bytes from the next page, but the copy into
old_instructionis limited to the bytes remaining in the first page. The rest of the XOL instruction becomes zero bytes and may execute as a different instruction. Copy the full decoded instruction, and fail registration if all bytes cannot be read safely. -
Removing one of several probes at the same address disables the others (
kernel/src/mm/ucontext/uprobe.rs, around line 692).Each consumer has a separate instance, but removing any one of them immediately restores the original byte. Other consumers remain in the list but can no longer receive hits. Consumers at one address should share one breakpoint, and the byte should be restored only when the last consumer is removed.
-
Removing a probe can discard valid changes to the rest of the page (
kernel/src/mm/ucontext/uprobe.rs, around line 735).The final removal maps the page saved at registration time back into the process. If the current private page was writable and the program changed other bytes while the probe was active, those changes are lost. Removal should preserve the current page contents and restore only the breakpoint byte.
-
Probe registration is only a snapshot of mappings that exist at open time (
kernel/src/perf/uprobe.rs, around line 252).A library mapped after
perf_event_open, a new image afterexec, or a child created byforkwill not receive the probe. There is also a race where a mapping can be removed and replaced after the file-mapping list is collected, causing the breakpoint to be installed in an unrelated executable mapping at the same address. The durable probe identity should be the file inode plus offset, with mapping, unmapping, fork, and exec paths applying or removing per-address-space instances. -
The x86 instruction safety check is incomplete (
kernel/crates/uprobe/src/arch/x86/mod.rs, around line 81).Rejecting control-flow instructions is not sufficient. Instructions such as
MOV SSandPOP SScan suppress the debug exception that the XOL path relies on. Linux uses an allow-list and special handling for instructions that change exception or trap behavior. Unsupported instructions should fail at registration rather than enter a broken XOL path. -
The perf interface is not compatible with the Linux PMU interface (
kernel/src/perf/mod.rs, around line 386).PERF_TYPE_MAXis used for both kprobe and uprobe, and the code guesses the kind from whether the name contains/. Standard tools obtain separate dynamic kprobe and uprobe types from sysfs. The current test hard-codes type 6, so it does not prove that agentsight or standard libbpf tooling can attach. Please expose a real uprobe event source/type and dispatch by that type. Also parse and honorperf_event_attr.disabled; probes currently start enabled even when the caller requested a disabled event. -
BPF callbacks and XOL lookup add too much work while interrupts are disabled (
kernel/src/exception/uprobe.rs, around lines 73 and 152).The handler holds the per-address-space spin lock while running all BPF callbacks, and the debug handler scans every probe to find the XOL slot. This can create long interrupt-off delays under frequent probes. Keep only a short protected lookup, hold a safe reference to the active probe, run callbacks after releasing the lock, and use the per-thread active state for constant-time XOL completion.
The current unit tests and successful kernel build are useful, but they cover the simple path only. Before merging, please add DragonOS runtime tests for:
- access from another user and invalid negative pids;
- two consumers on the same address;
- close/unregister racing with a hit;
- an instruction crossing a page boundary;
- signals or faults during XOL execution;
- ptrace, an existing TF bit, and hardware breakpoints;
- fork, exec, and mappings created after registration;
perf_event_attr.disabledand a real BPF callback.
These changes require a small architectural correction rather than isolated patches: keep the registered probe by file and offset, derive installed breakpoints for each address space, and keep the active XOL state on the executing thread. That structure will also remove several of the current races and performance costs.
…#2150 phase 1) Implement userspace breakpoint probes (uprobe), phase 1 of issue DragonOS-Community#2150, enabling agentsight to instrument SSL_read/SSL_write entry points. The design is XOL-based rather than reusing kprobe's kernel-buffer single-step (impossible at CPL=3). Key pieces: - per-mm uprobe table guarded by an irqsave SpinLock (not the global KPROBE_MANAGER lock nor the mm RwSem; the #BP/#DB hit path is IRQ-off) - breakpoint page install replicates do_wp_page private COW: copy_page_as_normal + single atomic set_entry + rmap attach/detach + flush_tlb_range. No transient empty PTE; each mm gets a private copy so writeback never persists 0xcc into the shared page-cache (.so) - XOL: a per-mm user slot page executes the saved instruction copy with RIP-relative relocation (yaxpeax-x86), validated at registration time - do_int3/do_debug gain is_from_user() dispatch. The #BP handler runs pre_handler + BPF (rip kept as the original probe address), then jumps rip to the pre-filled XOL slot, sets TF and NEED_UPROBE. The #DB handler recognizes XOL completion via NEED_UPROBE and restores rip; unconsumed user #BP is delivered as SIGTRAP(TRAP_BRKPT) - perf: PERF_TYPE_MAX dispatches to uprobe when the name contains '/'; UprobePerfEvent mirrors KprobePerfEvent and reuses BPF_PROG_TYPE_KPROBE Delivered in four batches: uprobe crate (x86 instruction analysis), mm integration (per-mm table / XOL / breakpoint page), exception dispatch, and perf attach. Verified: `make kernel` builds with 0 error / 0 warning; `cargo test -p uprobe` passes 7/7. An independent reviewer confirmed the F1-F10 review findings are satisfied with no kprobe/fork regression, and flagged two bugs that are fixed: re-registering the same probe_vaddr no longer reads 0xcc as the original instruction, and a RIP-relative displacement overflow now fails fast at registration instead of panicking at hit time. Out of scope: uretprobe (phase 2) and the QEMU runtime integration test. Refs: DragonOS-Community#2150 test(uprobe): add dunitest suite for uprobe breakpoint probes Add suites/normal/uprobe.cc covering the userspace perf_event_open uprobe path (issue DragonOS-Community#2150 phase 1): - RegisterAndTriggerSurvivesHit: perf_event_open(type=PERF_TYPE_MAX, config1=path, config2=offset) on the current process, then call the probed function and assert it survives the #BP -> XOL -> #DB -> resume hit path and returns the correct value - InvalidPathIsRejected / InvalidOffsetIsRejected: error inputs return negative errno Target offset is resolved from /proc/self/maps (executable segment + file pgoff), so the suite works regardless of PIE layout. Compiles cleanly via `make build-suites`; the gtest framework runs (the two negative cases pass on host Linux; the core trigger case is DragonOS-specific and is validated at runtime under QEMU). Refs: DragonOS-Community#2150 fix(uprobe): resolve CI failures - format check and cross-arch build - Apply rustfmt to uprobe integration code (reorder modules, imports, line width) to pass format-check on all arches - Add #[cfg(target_arch = "x86_64")] gates to uprobe integration points (exception/perf/mm-ucontext modules, AddressSpace fields, fork path, perf dispatch arm) so riscv64/loongarch64 build succeeds - Non-x86_64 perf dispatch returns ENOSYS for uprobe paths - Fix unused_mut on phys_addr in fork path for non-x86_64
- Thread 1: add ptrace access check (check_process_vm_access) before taking a target mm for cross-process uprobe, preventing unprivileged users from instrumenting arbitrary processes - Thread 2: reject control-flow instructions (call/jmp/ret/jcc/loop/int/ syscall) at registration time — XOL cannot safely single-step them - Thread 3: read_user_insn_bytes now continues into the next page when the probe is near a page boundary, returning real bytes instead of zero-padding that could decode to a different instruction - Thread 5: build_xol_slot fills trailing slot bytes with int3 (0xcc) so that a racy unregister during the XOL single-step window re-triggers #BP instead of executing zero-filled garbage
Security & correctness: - R1: pid==-1 (system-wide) now requires CAP_SYS_PTRACE; pid<-1 returns EINVAL. Per-pid path keeps check_process_vm_access. - R4: user #DB not consumed by uprobe now falls through to the normal DebugException path (restores pre-PR master behavior for ptrace/hardware breakpoints/single-step). - R5: original RFLAGS.TF is saved per-thread before XOL redirect and restored on completion (previously cleared unconditionally, silently disabling a program's own single-step mode). - R6: old_instruction copy now covers the full decoded instruction across page boundaries (was limited to first-page remainder). - R7: the breakpoint byte is restored only when the LAST consumer at that address unregisters (previously restored on every unregister, silently disabling remaining same-address consumers). - R8: unregister writes the original byte on the CURRENT mapped page instead of remapping the registration-time page, preserving the program's own writes to other bytes of that page. - R10: reject MOV SS (suppresses #DB) and POPF (overwrites RFLAGS/TF) at registration, in addition to control-flow instructions. - R11a: honor perf_event_attr.disabled (event starts disabled). Architecture (per-thread state, R2/R3/R12): - ActiveXol per-thread state on the PCB: probe_vaddr, return_addr, orig_tf, xol_page_base. Saved at #BP before rip redirect, consumed at #DB: O(1) completion independent of uprobe_list (racy unregister between #BP and #DB no longer corrupts resume), abort path when rip is outside the XOL page (signal/fault diversion), callbacks run outside the per-mm spinlock. Durable probe identity (R9): - Global registry keyed by inode+offset with consumer ids. New file mappings (dlopen/mmap via file_mapping_with_file_ext) and fork get late-applied probes; exec starts with an empty table. Consumer close drops registry entries plus late handles (per-mm unregister via the existing UprobeHandle::Drop with R7/R8 semantics). fork inherits instances with privatized child breakpoint pages. Verified: make kernel 0 error/0 warning; make fmt (clippy) clean; cargo test -p uprobe 9/9; QEMU dunitest uprobe 6/6, exec_abi 6/6, process_signal_fork 10/10.
86f1f09 to
0528893
Compare
|
感谢详尽的评审。12 项已全部处理完毕( 安全与正确性(R1/R4/R5/R6/R7/R8/R10/R11a)
架构修正(R2/R3/R12)
持久探针身份(R9)
R11(perf PMU 接口): 验证: |
关联 Issue
Refs #2150(阶段一:uprobe 断点探针)
概述
实现用户态断点探针(uprobe),使 agentsight 能在用户态函数(如
SSL_read/SSL_write)入口挂探针捕获参数。本 PR 完成阶段一(断点探针),命中路径#BP → XOL 单步 → #DB → 恢复端到端打通。设计决策
经盲区扫描 + 对抗评审纠正后的架构(关键点:不复用 kprobe 的内核缓冲区单步——CPL=3 时内核页不可执行):
KPROBE_MANAGER锁 / 非RwSem,命中路径关中断不可睡眠)do_wp_page私有 COW:copy_page_as_normal+ 单次set_entry原子帧替换 + rmap 账簿,每 mm 私有副本(writeback 不回写 0xcc 损坏 .so)do_int3/do_debug加is_from_user()二分;未消费用户态 #BP 投递SIGTRAP(TRAP_BRKPT)PERF_TYPE_MAX(6),按 config1 name 含/区分 uprobe/kprobe;复用BPF_PROG_TYPE_KPROBE改动文件
kernel/crates/uprobe/kernel/src/mm/ucontext/uprobe.rskernel/src/exception/uprobe.rskernel/src/perf/uprobe.rsinterrupt/{trap,mod}.rs、exception/mod.rs、mm/ucontext/{address_space,inner,mod}.rs、perf/mod.rs、process/state.rsuser/apps/tests/dunitest/suites/normal/uprobe.cc验证
make kernel:0 error / 0 warningcargo test -p uprobe:7/7 通过(指令长度、RIP-relative 检测/重定位、位移溢出)probe_vaddr读到 0xcc 当原指令 → 改为复用已有指令信息命中流程
flowchart TD A["用户态执行到 0xcc"] -->|"#BP"| B["do_int3: is_from_user?"] B -->|是| C["uprobe_breakpoint_handler\nlock uprobe_list(irqsave)"] C --> D["跑 pre_handler + BPF\nrip=原探针址"] D --> E["取 entries[0] XOL slot\n(slot 注册时已预填)"] E --> F["rip→XOL slot + TF + NEED_UPROBE"] F --> G["iretq: 用户态执行原指令"] G -->|"TF 触发 #DB"| H["uprobe_debug_handler\nNEED_UPROBE?"] H -->|是| I["rip 回原址+insn_len\n清 TF → post_handler"] I --> J["正常继续"]后续
sslsniff.bpf.c)