Repository navigation
fix(docker): build from source and verify tool downloads - #126
Closed
stevenAthompson wants to merge 583 commits into
Closed
stevenAthompson wants to merge 583 commits into
stevenAthompson wants to merge 583 commits into
Conversation
task_scope 是任务的授权边界、资产查询的过滤基准，覆盖度开关只该决定要不要 拿它当分母算指标，不该决定要不要累积范围本身。此前关闭覆盖度会让 auto 与 agent 两条写入路径同时失效，task_scope 只剩界面手工添加的行。 - insert_assets 的自动入范围(source='auto')无条件执行，不再判 coverageDisabled - add_task_scope 移出 coverageOnlyTools，planner / 任务内主 Agent / 目标拆解 Agent 在覆盖度关闭时照样持有；list_untested_assets 仍随开关隐藏（纯覆盖度视角） - 移除 insert_assets 每条资产的 related 入参：唯一作用是决定要不要入范围，值又 不落库（同一资产再次登记即作废、界面上也看不出谁被判为无关），连带删除 StripCoverageParams 及其专用的 deepCopyJSONMap / nestedMap - add_task_scope 与 goalsScopeTail 的描述改为「授权边界」优先、「覆盖度分母」其次
HTTP/工具响应含裸 NUL 字节时，json.Marshal 将其编码为 \u0000 转义， PostgreSQL jsonb 列拒绝并抛 SQLSTATE 22P05，导致 SaveSideSnapshot 的 INSERT 每秒失败数十次刷屏。新增 jsonbClean 在 marshal 后剥除真正的 \u0000 转义（按前置反斜杠奇偶判断，避免误伤字面 \\u0000），并订正 exploration.go 中 "json.Marshal already sanitizes" 的错误注释。 Closes BaaRai#111, Closes BaaRai#112
list_assets 此前把任务 id 写死为 0、裸查整个共享资产库，模型又无从写出 范围过滤条件，于是把别的任务的资产（尤其 IP）当成本任务目标，测试方向 被带偏。 - 新增 QueryDSLInScope / GetByIDsInScope：只返回落在【本任务及直接关联 任务】task_scope 内的资产，按归属匹配而非字面值（根域范围含其全部子域/ 服务/接口，网段范围含段内主机及服务）；越权 id 静默丢弃 - 新增 scopeTargetCTE（纯 scope 归属判定，不含 anchor/tested）；whereBuilder 加占位符偏移，$1 留给 taskID - 修复 IP 直连主机(http://1.2.3.4/api)无法被网段范围命中的盲区：谓词加 OR ts.net >>= try_inet(a.domain) - 非任务上下文(Auto/pentest, taskID<=0)无范围可依，回退全库 - QueryDSL/CountDSL 原样未动，UI 的「测试资产」视图（按生产者过滤）不受影响
start.bat 之前是 LF 换行。cmd.exe 解析批处理按字节偏移边读边执行， 只认 \r\n，全 LF 时会读串行，表现为 goto loop 被截断成 'oop'、 "75"( 不是内部或外部命令、: 运算符不存在等一连串报错，脚本根本跑不起来。 同时文件是 UTF-8 无 BOM，中文控制台默认 GBK(936)，注释和 echo 全是乱码。 - 顶部加 chcp 65001，中文提示正常显示（artex 本身输出也是 UTF-8） - 新增 .gitattributes：*.bat 钉 eol=crlf、*.sh 钉 eol=lf， 避免在 Linux 上编辑或重新 checkout 后又被打回 LF
原措辞连用 4 个"绝不"，且把上报写成"最多写进 fact"的退让，易让 worker 为自保而不敢深想、漏报跨意图关键线索。改为：唯一红线收窄到 "别真去执行别的意图"，明确"发散是好事"，上报改为"务必写进 fact"。
不单独配置凭据，直接复用当前激活的 LLM 配置(base_url/api_key/model)。 这个来源与 ddgs/brave/tavily 性质不同：DeepSeek 没有可直接调用的搜索接口， 搜索只存在于其 Anthropic 兼容 messages 接口内部，以 web_search_20250305 服务端工具的形式由 DeepSeek 服务端执行。由此带来几个必须让用户知道的差异， 已在设置页写明： - 仅支持 DeepSeek 官方端点 + anthropic 协议。实测其 OpenAI 协议端点会直接以 "unknown variant, expected `function`" 拒绝一切服务端工具，web_search_options 之类的顶层字段则被静默忽略，没有可用的替代写法。 - 每次搜索额外消耗一次模型调用。 - 搜索请求由 DeepSeek 服务端发出，不经过搜索出口代理，也不计入流量留痕。 - 返回结果只有标题与链接，没有摘要(DeepSeek 只给 encrypted_content)，需要正文 时仍由 WebFetch 抓取。 上述条件不做程序校验，仅在界面说明，由用户自行确认——配置不满足时搜索会在实际 调用时报错，与其它来源填错 Key 的反馈路径一致，可用「测试搜索」按钮实跑验证。 依赖 norma v0.3.8。
on_tool_call 等事件触发在 merge 时，每条事件都重复整段任务描述+目标； 当任务目标很长（如续跑继承事实）且短时间多次触发时，合并消息体积爆炸 （实测 39 条 submit_flag 连击 → 十几万字）。 - triggeredRun 拆出 taskDesc/taskGoal 任务级字段，message 只留事件正文 - by_task 合并：同任务头只写一次 - all 合并：按任务分组（首次出现排序、任务内保序），每任务头只写一次， 交错触发也不重复 - 单条触发由 finalTriggerMessage 补一次任务头 - 任务描述截 200 字、目标截 500 字，单份也有界 - 新增 trigger_merge_test.go 覆盖去重/交错/截断/定时无头
…er_trace worker 也可复用其他 work 的观察、避免重复劳动：search_all 无需先知 intent_id 按关键字全局捞命中步骤，get_worker_trace 锁定某条 work 后列步骤/就地搜/取完整内容。list_worker_traces 仍归 planner/main。
seedWorkerReadToolsUnbind 的解绑清单里仍列着 search_all_worker_traces / get_worker_trace，而 d531ea9 已把这两个加进 WorkerTools()——新库启动时先 seed 进去又被这段迁移撸掉，导致 worker（含 benchmark 镜像）实际拿不到。 - 从解绑清单去掉这两个，只保留 list_facts/node_detail/list_companies/list_worker_traces - 新增一次性 seedWorkerCrossTraceRebind：把这两个补绑回 worker，修复已跑过旧 v1 迁移、被误删的现存库；新库为无害 no-op
worker 已能通过 search_all_worker_traces / get_worker_trace 拿到 intent_id/ 节点 id，配套把 node_detail 也下放给 worker，便于就地查节点完整详情。 - WorkerTools() 增加 node_detail - 从 seedWorkerReadToolsUnbind 解绑清单去掉 node_detail（保留 list_facts/ list_companies/list_worker_traces） - 原 seedWorkerCrossTraceRebind 泛化为 seedWorkerReadbackRebind，补绑清单追加 node_detail，flag 升到 v2 让已跑过旧迁移的库也补上
此前 add_hint 走裸 notify，触发的规划轮 triggers 为空，开场白说"无具体变动"， hint 只折在 graph_overview 里让 planner 自己发现。仿照 set_goals/NotifyGoal， 让 add_hint 记一条 hint 触发事件并带上提示文本。 - TriggerEvent 增加 Hints 字段；renderTriggers 增加 hint 分支，播报「人新增了 N 条战略提示：…」 - ToolSet 增加 notifyHint 回调 + SetNotifyHint；add_hint 整批写完统一唤醒一次 （优先 notifyHint，未接则退回裸 notify），notify 从 addOneHint 上移到 handler - 主 agent(server.go) 与跨任务编排(orchestration.go) 均接上 Task.NotifyHint
- generationFor 移除未使用的 genBySig 参数（连同只写不读的 map 构造） - 两处 WriteString(x+"\n") 改为 WriteString + WriteByte，避免多余拼接 - 一处 WriteString(fmt.Sprintf(...)) 改为 fmt.Fprintf
…、回看工具补绑修复、/btw NUL 修复、合并触发去重
mock 模式下 GET /api/tasks/<id>/side-questions 无路由,被读兜底当集合返回 [],data.items 为 undefined,旁路 hook 的 merge() 迭代抛 TypeError: t is not iterable,异常在 setState updater 中经 render 重抛、catch 接不住,整页被错误 边界接管。mock handler 显式返回空旁路历史,sideAPI.history 同时归一化返回值。
description UPDATE 失败时原会 return 整个 seedFindingWorkflowTools,连累 independent 的 v2_reporter 迁移一起被跳过。改为出错仅 log、不置 flag(下次 启动重试),继续执行后续迁移。
buildDomainReg 构造的 server 级 ToolSet 恒持 nil ExplorationStore(taskID=0)， tools 表可把其中任一工具绑给不在任务内运行的 agent(auto/pentest/reporter/ 自定义/旁路提问)。node_detail 等探索图工具直接解引用 t.ts，被这类 agent 调用即 nil deref；panic 发生在 harness 自己的 goroutine 上，server 侧任何 recover 都 够不到，整进程退出，重启后调度器又拉起同一 agent 再崩，形成崩溃循环。 - 新增 readExpTool/writeExpTool + needExploration：t.ts==nil 时返回可读的工具 错误(提示改用带 task_id 的跨任务读取工具)，把 17 个探索图工具切过去 - anchorOwner 补判空 - AugmentTools 出口给每个工具套 guardPanic：handler panic 降级成工具错误并打 stack，覆盖域工具/SDK/MCP/skill，进程不再陪葬 - 注明 buildDomainReg 的 nil-store 契约
在复测页签的漏洞选择列表项之间插入 shadcn Separator，改善视觉分组。保留选中状态、严重性与处置状态标签，滚动与选择行为不变。
审查模型改为接收版本化 JSON：任务描述/目标/操作约束、工作目录、Worker 意图、当前轮输入，以及按 tool_use_id 配对的近期工具执行记录；末尾是完整的 tool_name 与 arguments。 - 任务约束每次审查从数据库重新读取；读取失败或约束超预算转人工。 - 历史只保留唯一配对的执行事实，排除 Agent 自述、未完成请求与平台拒绝反馈。 - 裁决改为 JSON decision(allow/ask/deny) + comment，三种结果均须说明实际操作、成功后果与命中规则。 - 所有模型裁决（含自动放行）保存实际模型输入与 SHA-256 指纹，详情页可展开。 - 对话审批卡片支持展开完整请求、模型输入、裁决与执行结果。 - 修复父目录 lockfile 导致的 Turbopack 根目录误判。 官方默认拦截规则、工具范围与优先级不变。
ParseVerdict 要求裸 JSON，而模型把 JSON 套进 markdown 围栏是常见偏差。 围栏导致解析失败后，reviewer 返回错误、Judge 走 FailAction，其默认值是 allow —— 模型判定的 DENY 会被静默降级为放行，审批记录只显示“模型输出 无法解析”，看不到模型其实给了拒绝。 解析入口前置一次围栏解包。这是确定性解包而非修复：载荷仍交由原 ParseVerdict 严格校验，重复字段、多余字段、多个对象、尾部残留的判定 全部不变。 要求开围栏与闭围栏同时存在。回复在 MaxTokens 处被截断时的形态正是 “有开围栏、无闭围栏”，对其补全会造出模型从未给出的裁决，因此这种输入 继续视为无法解析。 测试：原先断言“围栏必须拒绝”的用例改为接受，另补未闭合围栏、围栏后 带尾随文本、散文夹 JSON 三条必拒用例。
系统设置新增「实验功能」区块，内含 noa 模型驱动上下文压缩开关（默认关）。 开启后平台接入的四类 agent（planner / worker / 主 agent / 对话）由 noa 接管 上下文压缩，取代内置 compaction；关闭即恢复内置压缩。 - agent: 新增 enableNoa 共享接入器（noaadapter.Enable 唯一入口，all-or-nothing）， 四类 agent 各加 noaEnabledFn 解析器 + SetNoaEnabled，每 run 读一次，切换只影响 之后启动的 run，无需重建 agent。归档落在各 agent 本 run 工作目录下（任务级持久）， session id 与 transcript 同规则。启用失败经 onWarn 上报并回退内置压缩，不中断任务。 - server: settings key noa_compaction + Manager 读写；8 处 agent 装配点注入平台级 解析器（server 5 + task_llm 3）；GET/PUT /api/settings 增加字段。 - web: 系统设置页新增实验功能卡片与开关（乐观更新、失败回滚），类型与 mock 补字段。 官方默认行为不变：开关默认关闭时 noaadapter.Enable 不被调用，内置 compaction 原样工作。
* fix: 精简审查背景并区分人工消息与调度输入 * fix: 模型审查仅提交当前调用与选定背景 * feat: 审批来源定位原会话并移除 Worker 审查背景
各 agent 的 noa 归档基目录由本 run 的工作目录改为全局 workDir， 统一落在 <workDir>/noa/<SessionID>/。SessionID 全局唯一，同一基目录 内互不冲突。
feat(server): capability-gap report from skill misses
docs(rsi): Loop C adoption contract and credits
Extracts candidate lessons from terminal tasks (confirmed, negative, retest caution), gates them on cue + action + evidence, and makes extraction idempotent through a per-source evidence key. ARTEX_RSI_LESSONS selects off / shadow / inject. Shadow logs what would have been shown; inject appends a code-owned, data-not-instructions block after the planner and worker system prompts. No prompt mutation and no adoption: status stays candidate. Credit: the Library admission rule comes from harness2 (Google Research, Apache-2.0, arXiv 2610.14224); the gain taxonomy from awesome-rsi (lobehub).
feat(rsi): evidence-gated lesson library in shadow mode
Sessions keep state=running in the store during a task-level pause, so the sessions tab treated them as live and the run duration kept ticking with Date.now(), visibly advancing time while the task (and its deadline budget) was frozen. Pass taskPaused down from the detail page and pin the duration at the last recorded step while paused. Closes BaaRai#84
…ion, stagnation surfacing Three orchestration mechanisms from the Q00/ouroboros evaluation (BaaRai#99): - Goal ambiguity gate at task creation: a goal must name a target (URL, domain, IPv4, localhost:port or absolute path) and reach a length floor, else the API returns 400 before any token is spent. Deliberately target-less goals opt out with the literal marker [targetless]. Guards against the live failure where a goal naming no target sent every worker to an unrelated asset. - Consecutive-failure escalation on the task LLM chain: quota exhaustion already advances the chain; now 3 consecutive non-quota call failures on the same chain profile advance it too, with the true reason recorded. Escalation-only by design (no auto-downgrade: mid-task model swaps risk behavior drift; the operator keeps the manual chain reset). - Stagnation surfacing in the planner overview: finished intents are fingerprinted (case-folded, digits collapsed); a direction dispatched 3+ times with zero findings since its first attempt is flagged in a self-describing stagnation_warning payload that instructs the planner to diversify instead of grinding. Absent when nothing stagnates. Closes BaaRai#99
…uf, dalfox, sqlmap Installs the drost-community take-1 subset (BaaRai#100) directly into the runtime image as pinned release binaries (plus a sqlmap checkout symlinked into PATH), so recon-phase workers get modern tooling through their existing Bash access with no new trust surface (no docker, no MCP wrapper). Versions are ARGs for deliberate upgrades; each tool is executed once at build time so a broken download fails the build instead of a hunt. Closes BaaRai#100 (takes 2 and 3 stay design notes: docker-exec integration remains blocked per BaaRai#90; catalog/execute MCP gating is future work).
…ll with vendored PoCs Implements BaaRai#109. Skills ship the chain-1 workflow (session file injection through upload_progress filename + app LFI -> RCE, CVE-2026-109618/109619) with hard gates: authorized targets only, program zero-day policy checked before any live attempt (patches landed 2026-10-21 for 8.2-8.5), passive detection before exploitation, lab-first practice via the vendored docker lab, chains 2/3 (UAF, serializer mismatch) marked lab-only, one-shot verification with a marker. Upstream PoCs vendored under scripts/poc/php2shell with attribution and the MIT notice (upstream declares MIT in its README).
Implements the per-intent rotation shape from BaaRai#85: a task created with llm_rotate=true distributes worker intents round-robin over its LLM chain heads, so lane A can hunt on one model while lane B hunts on another (different lenses per intent, no extra tokens). Default stays the classic ordered failover from the active head; planner/goals/mainagent keep the active head always. Mechanics: the engine pins the next chain head into the run context when a worker claims an intent; the pin travels through the provider resolver (hooks.current now takes ctx). A pin is only a dispatch preference - it must belong to the task chain and never bypasses failover state (an exhausted chain stays exhausted). Schema: tasks.llm_rotate boolean, default false, idempotent ALTER; exposed on the create payload and the task DTO. Closes BaaRai#85 (the ensemble/P4 shape stays parked as before)
- resetLLMFailStreaksForTest helper: the package-global streak is in-memory state, so the no-switch-on-ordinary-error subtests now reset it before each case (accumulation across subtests sharing task/profile keys made the third case escalate, breaking the contract the test guards) - core lifecycle and credentials fixtures: goals now name the lab target, per the ambiguity gate (these tests exercise lifecycle/credentials, not the gate) - canonical reset helper moved into task_llm.go next to the counter it resets
fix(web): freeze session run duration while the task is paused
feat(engine): ouroboros takes - goal ambiguity gate, failover escalation, stagnation surfacing
feat(image): web-recon toolbox - nuclei, httpx, subfinder, katana, ffuf, dalfox, sqlmap
feat(skill): php2shell - gated session.upload_progress LFI-to-RCE skill with vendored PoCs
feat(engine): per-intent LLM chain-head rotation (opt-in llm_rotate)
Seam fix after merging BaaRai#111 (escalation) with BaaRai#114 (ctx-threaded hooks.current): the escalation test predates the signature change and kept compiling on its own branch, so the merge left main red.
fix(test): escalation test to the ctx-aware hooks.current signature
The dalfox musl tarball does not carry the binary at archive root, so a rooted tar extract failed the image build; extract to a temp dir and install by name (same shape as the projectdiscovery zips). Also dalfox has no version subcommand - it treated it as a scan target; use --version. Locally verified: image builds clean, nuclei v3.11.1 and dalfox 3.2.4 execute inside the built image.
fix(image): toolbox tar extraction paths + dalfox version flag
Toolbox layer pins bbot v3.0.2 (AGPL-3.0, same license as ARTEX; standalone CLI dependency, not linked code) in its own venv with module dependencies preinstalled via --install-all-deps, so scans never need pip under a program egress allowlist. Build-time gate dry-runs a dep-heavy module sample twice: the first primes the dependency check, the second must be free of Installing and ERRR lines or the build fails. The bbot-recon skill gates usage on program policy: passive zero-touch mode via seeds-only scans (-s, no -t) when scanners are banned, active-safe and loud module sets when the program allows them, scope discipline with blacklist and strict-scope, --no-deps mandatory on every run, JSONL ingestion into facts and finding candidates with bbot findings treated as leads that still need independent verification. Closes BaaRai#117
feat: bbot recursive recon orchestrator (toolbox layer + policy-gated skill)
Syncs BaaRai/Norma to upstream Autumn-27/Norma v0.4.4 (fast-forward, our earlier changes were already merged upstream) and repins the replace from the pre-merge pseudo-version to the tag. What v0.4.4 brings for us: per-task process-tree RSS watchdog (kills the process group over a memory cap, matters now that workers spawn nuclei, ffuf, sqlmap and bbot), UTF-8-aligned output truncation, non-interactive default env for subprocesses (no more tools hanging on hidden stdin reads), input usage recovery from message_delta for accurate token accounting, and streaming / thought_signature fixes. Full test suite green against a scratch postgres.
chore: bump norma to v0.4.4 (fork synced to upstream tag)
docs: AI PR review REA analysis and ARTEX integration plan
The published autumn27/artex image no longer exists, and the build downloaded several tool archives without verification. The image is now produced by a multi-stage build: frontend export, Go binary with the embedded UI, pinned and checksum-verified tools, then the runtime layer. docker compose builds locally; install.sh and update.sh build instead of pulling. The release workflow publishes to GHCR, and sqlmap, Node, and Playwright are pinned.
Contributor
Author
|
Superseded by #127, which is rebased on v0.4.0. |
murilosoave
pushed a commit
to murilosoave/ARTEX
that referenced
this pull request
Oct 25, 2026
ParseVerdict 要求裸 JSON，而模型把 JSON 套进 markdown 围栏是常见偏差。 围栏导致解析失败后，reviewer 返回错误、Judge 走 FailAction，其默认值是 allow —— 模型判定的 DENY 会被静默降级为放行，审批记录只显示“模型输出 无法解析”，看不到模型其实给了拒绝。 解析入口前置一次围栏解包。这是确定性解包而非修复：载荷仍交由原 ParseVerdict 严格校验，重复字段、多余字段、多个对象、尾部残留的判定 全部不变。 要求开围栏与闭围栏同时存在。回复在 MaxTokens 处被截断时的形态正是 “有开围栏、无闭围栏”，对其补全会造出模型从未给出的裁决，因此这种输入 继续视为无法解析。 测试：原先断言“围栏必须拒绝”的用例改为接受，另补未闭合围栏、围栏后 带尾随文本、散文夹 JSON 三条必拒用例。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The published
autumn27/arteximage no longer exists (Docker Hub returns 404), whiledocker-compose.ymlstill pulled it from a namespace that could be reclaimed. This builds the image from source instead.Changes:
docker-compose.ymlbuildsartex:local;install.shandupdate.shbuild instead of pull.env.exampleupdatedVerification on the build machine (no Docker daemon available there):
npm run build:staticwith Node 22: passGOOS=linux GOARCH=amd64 go build -tags embedui -trimpath -ldflags ... ./cmd/artex: pass; the resulting binary starts and prints usagego vet ./...: passhadolint Dockerfile: no errors (DL3008 and DL4006 warnings only)docker builditself was not executed because the machine has no container runtime. Keeping this as a draft untildocker compose buildhas been run once on a Docker host or in CI.Note: #125 also edits README.md and Dockerfile, so if GitHub reports a conflict, merge #125 first and rebase this branch.