feat(loop): SLM/LLM subtask complexity routing - #218
Open
raymondginger2018-sudo wants to merge 2 commits into
Open
feat(loop): SLM/LLM subtask complexity routing#218raymondginger2018-sudo wants to merge 2 commits into
raymondginger2018-sudo wants to merge 2 commits into
Conversation
Contributor
Author
设计说明问题:目前所有 subtask 都走同一个 LLM,简单任务(字符串处理、文件重命名)和复杂任务(多步推理、代码生成)消耗同样成本与延迟。SLM(小模型)在简单任务上已经足够,但缺乏路由机制。 解法:两个文件——
关键设计决策:
测试建议:对三档分别准备典型任务,验证路由结果;再跑 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Small language models (SLM) fit local/edge/low-cost niches. This PR adds an explicit subtask-class router: high-frequency, low-complexity subtasks (tool-result cleanup, summarization, classification) should ride the SLM path, while deep reasoning stays on the LLM path.
Components
core/loop/slm_routing.py(new, 152 lines)Pure decision mechanism:
route_subtask(task_class, ...) -> RoutingDecisionsimple/medium/complexDEEPCODE_SLM_MODEL,DEEPCODE_LLM_MODELDEEPCODE_SLM_ROUTING=0disables SLM routing (falls back to LLM tier)core/loop/slm_tasks.py(new, 102 lines)First consumer: turns the routing decision into a preview-shaping policy for oversized tool results — when cleanup is SLM-grade, the persisted preview is shaped as a clean dense digest; when LLM-grade, raw truncation stays.
Design
Files
core/loop/slm_routing.pycore/loop/slm_tasks.pyPart of GenAI lesson 19 SLM/LLM cost-tier family.