A lightweight, retrieval-augmented pipeline that recovers meaningful names for functions in stripped binaries — by feeding an off-the-shelf LLM the static-analysis context a reverse engineer would use (who the function calls, who calls it, its disassembly), instead of fine-tuning a model.
Function-name recovery is a core reverse-engineering task: a stripped binary has
sub_401250 where the source had read_whole_file. Recovering the intent speeds
up every downstream analysis.
The idea of augmenting an LLM with static-analysis context (call graphs, data flow) for binary understanding comes from:
Guoqiang Chen, Huiqi Sun, et al. ReCopilot: Reverse Engineering Copilot in Binary Analysis. QI-ANXIN Technology Research Institute, 2025. arXiv:2505.16366 · code
This repository is not ReCopilot and does not reuse its code, model, or data. ReCopilot builds an expert model by continued-pretraining + fine-tuning Qwen2.5-Coder-7B on 36B tokens. This project is an independent, deliberately lightweight alternative take on the same idea: no training at all — just retrieval of call context + prompting of a general model. The goal is to see how far context alone gets you, as a cheap, reproducible baseline. All code here is my own.
| ReCopilot (Chen et al., 2025) | This project | |
|---|---|---|
| Core idea | LLM + static-analysis context for binary understanding | same idea |
| How the model is built | continued-pretraining + SFT + DPO on 36B tokens, base Qwen2.5-Coder-7B | no training — retrieval of call context + prompting |
| Compute to reproduce | large (multi-GPU, weeks) | a laptop; the default backend needs no GPU at all |
| Tasks | 14 (names, types, structs, decompilation, summaries, …) | function-name recovery (extensible; see roadmap) |
| Context used | call graph + data flow | callees / callers / disassembly (call graph) |
| Runs offline, no key | — | yes (mock backend) |
| Best possible quality | higher (a purpose-built model) | bounded by the general LLM you plug in |
The honest positioning: ReCopilot is the stronger system; this is the cheap, fully-reproducible baseline that isolates how much of the job context+prompting alone can do — and its pipeline can serve ReCopilot's own base model (Qwen2.5-Coder via Ollama) to measure exactly that.
symbol-bearing binary
│ objdump / llvm-objdump
▼
per-function context ─── callees (what it calls) ── callers (who calls it) ── disassembly
│ ▲
│ (RETRIEVAL) └── reuses the call-graph extraction from my
▼ gpu-graph-study project
hide the real name → build a context-augmented prompt (AUGMENT)
▼
LLM backend proposes a name (GENERATE) ── mock | OpenAI-compatible | Ollama
▼
score vs. the real name (EVALUATE) ── exact match · token-F1 · semantic hit
Ground truth is free: a symbol-bearing binary already knows each function's real name. We hide it, ask the pipeline to recover it from context, and score the guess — no manual labeling.
Five methods scored on the same functions (experiments/compare_methods.py). All
are cheap alternatives to a fine-tuned model; the LLM column can host any backend,
including ReCopilot's base model via Ollama.
| Method | what it is | token-F1 | semantic-hit |
|---|---|---|---|
| majority-callee | name after the first callee | 0.00 | 0.00 |
| heuristic(calls) | signature-call → name map | 0.36 | 0.60 |
| nearest-neighbor (RAG) | retrieve most callee-similar labeled function | 0.00 | 0.00 |
| llm(mock) | offline heuristic stand-in | 0.36 | 0.60 |
| llm(qwen2.5-coder, Ollama) | ReCopilot-family base model — plug in to fill this row | — | — |
On the demo binaries the call-pattern heuristic already recovers 60% of names semantically, while the retrieval baseline fails — the corpus (one binary) does not cover the test binary's call patterns, so nearest-neighbor has nothing similar to retrieve. That is the expected, honest behaviour of retrieval under domain mismatch, and it motivates the LLM row: a model that reads the disassembly is not limited to what the corpus happens to contain.
⚠️ These are tiny demo binaries (≈5–7 functions each) — the numbers illustrate the pipeline and the method differences, not a benchmark verdict. A real evaluation needs a large labeled corpus of binaries; the harness is built to scale to that (--corpus a b c --test d).
To include a real base model:
ollama pull qwen2.5-coder:7b
PYTHONPATH=. python experiments/compare_methods.py \
--corpus examples/demo2 --test examples/demo --recopilot-model qwen2.5-coder:7b# 1) build the demo binary (needs gcc; or point --binary at any symbol-bearing binary)
bash examples/build.sh
# 2) run the pipeline — mock backend, no API key, no network:
PYTHONPATH=. python experiments/run_name_recovery.py --binary examples/demo
# offline smoke tests:
PYTHONPATH=. python tests/test_smoke.py# Local model via Ollama (free, private; ReCopilot's base family):
ollama pull qwen2.5-coder:7b
PYTHONPATH=. python experiments/run_name_recovery.py \
--binary examples/demo --backend ollama --model qwen2.5-coder:7b
# Or any OpenAI-compatible API:
pip install openai
export OPENAI_API_KEY=sk-... # OPENAI_BASE_URL for OpenRouter/Together/vLLM
PYTHONPATH=. python experiments/run_name_recovery.py \
--binary examples/demo --backend openai --model gpt-4o-miniThe pipeline is identical across backends — only the --backend flag changes — so
the mock lower bound and a real LLM are measured on exactly the same prompts.
src/rag/context.py # retrieval: disassemble -> per-function callees/callers/body
src/rag/prompt.py # augment: build the context-rich prompt (never leaks the name)
src/llm/{mock,openai_api,ollama}.py # pluggable generation backends
src/methods.py # the compared methods: heuristic, NN-retrieval, majority, LLM
src/eval/name_recovery.py # exact-match / token-F1 / semantic-hit metrics
experiments/run_name_recovery.py # single-method end-to-end run
experiments/compare_methods.py # head-to-head comparison + chart
examples/demo.c, demo2.c # symbol-rich demos (ground truth)
tests/test_smoke.py # offline checks
- Retrieval of call context + context-augmented prompting
- Pluggable backends (mock / OpenAI-compatible / Ollama) + name-recovery metrics
- Real-LLM run vs. the mock lower bound, across several binaries
- More tasks from the ReCopilot menu: type inference, one-line summarization
- Richer context: data-flow slice, string literals, and call-graph neighbors' names
MIT — see LICENSE. This project reuses none of ReCopilot's code, weights, or data;
it only builds on the published idea, with attribution above.
Emad Mahmodi · portfolio