Skip to content

feat(loop): explicit mitigation for retrieval failure modes - #222

Open
raymondginger2018-sudo wants to merge 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:feat/memory-retrieval
Open

feat(loop): explicit mitigation for retrieval failure modes#222
raymondginger2018-sudo wants to merge 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:feat/memory-retrieval

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor

Summary

Adds explicit handling for three failure modes an agent memory loop must face:

  1. Retrieved nothing — no similar entry in store → similarity threshold + explicit no-memory fallback (never best-effort guessing)
  2. Retrieved the wrong thing — pure semantic recall misses identifiers/API names → hybrid keyword + vector recall, threshold enforcement
  3. Retrieved but not used — chunks in history but never injected → assert_all_injected() self-check

Design

File

  • core/loop/memory_retrieval.py (new, 135 lines)

Part of GenAI lesson 15 memory-loop quality family.

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor Author

设计说明

问题:RAG/记忆检索存在三类典型失败:相似度阈值过严(漏召回)、过松(噪声召回)、排序不稳定(同一查询多次结果不同)。失败时 agent 通常静默降级或直接报错,没有显式处理。

解法memory_retrieval.py——定义默认相似度阈值 0.45,提供显式的失败分类和降级策略。

关键设计决策

  • 显式失败模式:把"静默错误"变成"可观察的失败类型",调用方可以根据类型决定重试/放宽/放弃
  • 复用 PR feat(loop): prompt-injection regression suite (pure mechanism) #216render_data_block() 来编码检索结果——检索内容是不可信数据,必须走同一套注入防护编码
  • 默认阈值 0.45 来自经验值,可通过参数覆盖

依赖说明:本文件 import 了 core.loop.injection_regression.render_data_block(即 PR #216)。建议 #216 先合入,本 PR 随后。如果维护者希望独立合入,我可以移除该依赖并内联一个最小版本。

测试建议:对三类失败模式各构造一组检索结果,验证分类正确

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant