Skip to content

feat(loop): SLM/LLM subtask complexity routing - #218

Open
raymondginger2018-sudo wants to merge 2 commits into
HKUDS:mainfrom
raymondginger2018-sudo:feat/slm-routing
Open

feat(loop): SLM/LLM subtask complexity routing#218
raymondginger2018-sudo wants to merge 2 commits into
HKUDS:mainfrom
raymondginger2018-sudo:feat/slm-routing

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor

Summary

Small language models (SLM) fit local/edge/low-cost niches. This PR adds an explicit subtask-class router: high-frequency, low-complexity subtasks (tool-result cleanup, summarization, classification) should ride the SLM path, while deep reasoning stays on the LLM path.

Components

core/loop/slm_routing.py (new, 152 lines)

Pure decision mechanism:

  • route_subtask(task_class, ...) -> RoutingDecision
  • Subtask classes: simple / medium / complex
  • Env-tunable model overrides: DEEPCODE_SLM_MODEL, DEEPCODE_LLM_MODEL
  • DEEPCODE_SLM_ROUTING=0 disables SLM routing (falls back to LLM tier)

core/loop/slm_tasks.py (new, 102 lines)

First consumer: turns the routing decision into a preview-shaping policy for oversized tool results — when cleanup is SLM-grade, the persisted preview is shaped as a clean dense digest; when LLM-grade, raw truncation stays.

Design

  • Zero network, zero async — decision-only, deployment-adaptable
  • Route by task complexity, not by caller identity
  • Clean separation: routing decision vs. execution channel

Files

  • core/loop/slm_routing.py
  • core/loop/slm_tasks.py

Part of GenAI lesson 19 SLM/LLM cost-tier family.

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor Author

设计说明

问题:目前所有 subtask 都走同一个 LLM,简单任务(字符串处理、文件重命名)和复杂任务(多步推理、代码生成)消耗同样成本与延迟。SLM(小模型)在简单任务上已经足够,但缺乏路由机制。

解法:两个文件——

  • slm_routing.py:核心路由器 route_subtask(),按任务复杂度分 SUBTASK_SIMPLE / MEDIUM / COMPLEX 三档
  • slm_tasks.py:第一个 consumer 示例,展示如何将简单 subtask 交给 SLM

关键设计决策

  • 环境变量覆盖:DEEPCODE_SLM_MODELDEEPCODE_LLM_MODELDEEPCODE_SLM_ROUTING=0(可整体关闭)——不改变默认行为,opt-in
  • 复杂度判定是静态规则(token 数 + 关键词 + 结构特征),不走 LLM 判断,避免"用 LLM 判断要不要用 LLM"的递归成本
  • 与现有 LLM provider 抽象解耦,路由结果只是返回模型名,不碰调用链

测试建议:对三档分别准备典型任务,验证路由结果;再跑 DEEPCODE_SLM_ROUTING=0 验证回退路径

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant