Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.i18n.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Bilingual consistency record for the repository README files.
# README.md is the default English page; README.zh.md is its Chinese counterpart.
# Values are git blob hashes from the last confirmed-consistent review.
README.md: 1163cc9c9a7332a95f8b2448461243f0813ffdf2
README.zh.md: 6ce7a8314f10dcd45316906f79991e0604b0814c
README.md: 767a4e02b360e6c2b28ee54d79067b5f64e4520d
README.zh.md: 04405f75c75884a3682defb784c26d0e0c541129
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Connect [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) to t

`dsh` is DeepSeek AI's open-source, plugin-based agent harness. This repository provides a companion browser bridge plugin and Chrome/Firefox MV3 extension as one standalone pnpm workspace.

Browser operation remains text-only: pages become structured text with a numbered inventory of interactive elements, and the model addresses those elements by number. dsh 0.2.0 multimodal chat is separate from that page channel—the side panel accepts PNG, JPEG, WebP, and GIF attachments when the host advertises image support, while browser tools still never capture screenshots.
Browser operation is text-first: pages become structured text with a numbered inventory of interactive elements, and the model addresses those elements by number. `browser_screenshot` adds the one image path — the extension captures the visible area, the bridge writes those bytes to a local file, and the model receives that path instead of pixels. dsh 0.2.0 multimodal chat is separate from that page channel: the side panel accepts PNG, JPEG, WebP, and GIF attachments when the host advertises image support.

> [!IMPORTANT]
> The workspace pins dsh 0.2.0-rc.2, the minimum supported runtime. Older DSH releases are not supported.
Expand Down Expand Up @@ -59,6 +59,7 @@ The paired Playwright / extension duration ratio was **1.24** (95% CI **1.16–1
| Follow tab | `browser_follow_tab` | Bind later browser tools to a tab returned by `browser_list_tabs` without activating it |
| Close tab | `browser_close_tab` | Close a tab returned by `browser_list_tabs` |
| Read region | `browser_get_text` | Lazy-loaded or partial page text |
| Capture the view | `browser_screenshot` | Save the visible area as a local image and return its path for `read_image`; use it for visual content (captcha, chart, layout) |
| Wait for stability | `browser_wait` | Page-load and render-settle detection |
| Send images | `session.prompt` / `session.attachment` | Host-capability-gated image drafts, image-only prompts, and durable history previews |
| Quote a selection | side panel composer | Text you highlight in the page appears in the composer and is sent with your next message as fenced, attributed page content |
Expand All @@ -76,7 +77,7 @@ scripts/install.ps1
## Why this design

- **Your real browser, not a headless copy**: the model works in the page you already have open, retaining logins, sessions, and cookies.
- **A text-first page interface**: numbered controls, stable IDs across snapshots, delta updates, and masked sensitive values make pages operable without screenshots; user-attached chat images use dsh's separate multimodal message path.
- **A text-first page interface**: numbered controls, stable IDs across snapshots, delta updates, and masked sensitive values make pages operable as text; `browser_screenshot` is the deliberate exception, and its pixels are saved to a local file rather than entering the conversation. User-attached chat images use dsh's separate multimodal message path.
- **Pointing instead of describing**: highlight the passage you mean and the side panel quotes it, so "explain this" needs no page tour. The quote is captured only while a panel is open, and nothing is sent until you send the message.
- **A narrow privacy boundary**: passwords and payment-card values are always rendered as `••••` and never leave the page.
- **A guarded bridge**: authenticated handshakes protect remote connections, privileged gateway methods reject non-loopback callers, and the extension binds tools to one user-controlled tab.
Expand Down Expand Up @@ -185,7 +186,7 @@ If you encounter `cache.hydratePrepared is not a function`, update the repositor
- The bridge path sits outside the `/api` trust boundary and performs its own bearer-token authentication.
- Local Chrome extension origins retain zero-configuration loopback access; Firefox origins are per-install UUIDs and must present the bearer token.
- Privileged gateway methods such as `settings.*`, `credentials.*`, and `host.open*` reject non-loopback sources.
- The browser-page pipeline is text-only and never captures screenshots; explicitly attached chat images use dsh's durable attachment service. Password and payment-card values never leave the page.
- The browser-page pipeline returns text, and its single image route (`browser_screenshot`) writes each capture to a local temp file so the model is handed a path rather than pixels; explicitly attached chat images use dsh's durable attachment service. Password and payment-card values never leave the page.
- When work begins, the assistant binds to the active tab (at prompt submission, or at the first direct browser-tool call). If you switch tabs manually, later browser actions pause and the side panel asks whether the assistant should continue on the original tab or follow the new one. Choosing the original tab permits background operation; the extension never silently retargets or changes your visible tab. Closing the controlled tab also pauses tools until you explicitly select the current page.
- Text you highlight is captured only while a side panel is open and page sharing is not `off`, and never from password or payment-card fields. It stays inside the extension until you send the message, is dropped when you dismiss it or its page navigates or closes, and reaches the model inside the same untrusted-content boundary as page snapshots — including its source title and URL, which the page also controls.
- Page-authored text is wrapped as untrusted input. The default `auto` mode reads only the controlled tab without an extra prompt; privacy-sensitive users can select `ask` for per-read confirmation or `off` to block reads entirely. In `ask` mode, the read dialog can allow one read or persistently switch back to `auto`; this can be reversed in Settings. Read page text is sent to the selected model.
Expand Down
7 changes: 4 additions & 3 deletions README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

`dsh` 是由 DeepSeek AI 开发的开源、插件化 agent harness(智能体框架)。本仓库将配套的浏览器桥插件与 Chrome/Firefox MV3 扩展组成一个独立的 pnpm workspace。

浏览器操作仍采用纯文本设计:页面会转换为结构化文本和带编号的交互元素清单,模型通过编号定位元素。dsh 0.2.0 的多模态对话走独立通道——宿主声明图片能力时,侧栏可发送 PNG、JPEG、WebP 和 GIF;浏览器工具本身仍不会截取页面截图。
浏览器操作以文本为主:页面会转换为结构化文本和带编号的交互元素清单,模型通过编号定位元素。`browser_screenshot` 是唯一的图片通道——扩展截取可见区域,桥接层把这些字节写成本地文件,模型拿到的是文件路径而不是像素。dsh 0.2.0 的多模态对话走独立通道:宿主声明图片能力时,侧栏可发送 PNG、JPEG、WebP 和 GIF。

> [!IMPORTANT]
> 当前工作区固定使用 dsh 0.2.0-rc.2,也是最低支持版本;不再支持旧版 DSH。
Expand Down Expand Up @@ -59,6 +59,7 @@ Playwright / 扩展的配对耗时比为 **1.24**(95% CI **1.16–1.34**):
| 跟随标签页 | `browser_follow_tab` | 将后续浏览器工具绑定到 `browser_list_tabs` 返回的标签页,而不激活该标签页 |
| 关闭标签页 | `browser_close_tab` | 关闭 `browser_list_tabs` 返回的标签页 |
| 读取区域 | `browser_get_text` | 懒加载内容 / 局部文本 |
| 截取画面 | `browser_screenshot` | 把可见区域存成本地图片并返回路径供 `read_image` 查看;页面含义靠视觉(验证码、图表、排版)时用它 |
| 等待稳定 | `browser_wait` | 页面加载与渲染稳定检测 |
| 发送图片 | `session.prompt` / `session.attachment` | 按宿主能力启用图片草稿、纯图片消息和持久历史预览 |
| 引用选中内容 | 侧栏输入框 | 在页面里划选的文字会出现在输入框,随下一条消息一起发送,并带上来源与不可信内容边界 |
Expand All @@ -76,7 +77,7 @@ scripts/install.ps1
## 为什么这样设计

- **使用你的真实浏览器,而不是无头副本**:模型操作你已经打开的页面,登录态、会话和 Cookie 均会保留。
- **纯文本页面接口**:编号控件、跨快照稳定 ID、delta 更新和敏感值掩码,使模型无需截图也能操作页面;用户主动添加的对话图片走 dsh 独立的多模态消息通道。
- **文本优先的页面接口**:编号控件、跨快照稳定 ID、delta 更新和敏感值掩码,使模型以文本方式操作页面;`browser_screenshot` 是刻意的例外,其像素会写成本地文件而不会进入对话。用户主动添加的对话图片走 dsh 独立的多模态消息通道。
- **用「指」代替「描述」**:直接划选你要问的那段文字,侧栏会把它引用下来,说「解释这个」不必再描述整页内容。只有侧栏打开时才会捕获,并且在你发送消息之前不会离开浏览器。
- **收窄隐私边界**:密码和支付卡字段始终显示为 `••••`,字段值不会离开页面。
- **受保护的桥连接**:远程连接使用认证握手,特权网关方法拒绝非回环调用方,扩展把工具绑定到一个由用户控制的标签页。
Expand Down Expand Up @@ -185,7 +186,7 @@ pnpm --filter dsh-browser-extension run test
- 桥路径在 `/api` 信任栅栏之外,自带 bearer token 认证。
- Chrome 扩展的本地 Origin 保留零配置回环访问;Firefox Origin 是每次安装生成的 UUID,必须携带 bearer token。
- 特权网关方法(`settings.*`/`credentials.*`/`host.open*`)对非回环来源一律拒绝。
- 单活动连接;浏览器页面管线为纯文本且不截图;用户主动添加的对话图片交给 dsh 持久附件服务,密码和卡号值永不回传。
- 单活动连接;浏览器页面管线以文本为主,其唯一的图片通道(`browser_screenshot`)把每次截图写入本地临时文件,模型拿到的是路径而不是像素;用户主动添加的对话图片交给 dsh 持久附件服务,密码和卡号值永不回传。
- 助手开始工作时会绑定当时的活动标签页(提交提示时绑定;直接调用浏览器工具时则在首次调用绑定)。用户手动切页后,后续浏览器操作会暂停,侧栏会询问让助手继续原页面还是跟随新页面;选择原页面后允许在后台继续,但扩展绝不静默改绑或切换用户正在看的页面。受控标签页关闭后也会暂停,直到用户显式选择当前页。
- 只有在侧栏打开、且页面共享不是「关闭」时才会捕获划选内容,密码和卡号字段永不读取。内容在发送之前始终留在扩展内部;移除、页面跳转或标签页关闭都会丢弃它;发送时与页面快照一样包在不可信内容边界内,来源标题和 URL 同样由页面提供,因此也放在边界之内。
- 网页文字会标记为不可信输入。默认「自动共享」只按需读取受控标签页且不额外弹窗;对隐私敏感时可选择「每次询问」,或用「关闭」完全阻断读取。在「每次询问」模式下,读取弹窗可以仅允许一次,也可以持久切回自动读取;之后仍可在设置中关闭。读取的页面文字会发送给当前选择的模型。
Expand Down
4 changes: 2 additions & 2 deletions extensions/dsh-browser/README.i18n.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write dsh-browser/extensions/dsh-browser/README.md
README.md: 6b6ea0b302a340fca44d6f461ebf84a7b5419c89
README.zh.md: 36f25412f7e1431aa89a8ac41ef045c9b7676790
README.md: 30d41256b315c6e93490e1fca400a04678075f3c
README.zh.md: 3bd1765041e2d85f8bd1cf367071cd0eba0e621f
Loading