Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ uv run pytest -q

真实 UltiCode HTTP / 模型 e2e 均为显式 opt-in。`e2e_sourced_analysis.py` uses agent-authored synthetic Markdown—not submissions, DTOs, or licensed user material—and validates a read-only submission projection without a real model. U03 workflow model analysis additionally requires a valid U02 gate and active budget authorization; other evaluation scripts follow their own gates. Supply credentials through a secure environment/secret store, never command text or logs. Runner contracts live in source and Linear; keep per-run results out of core docs.

`DeepseekModel` requests provider JSON Output with `response_format={"type":"json_object"}` and an explicit JSON prompt. The guard accepts only that exact format when supplied; legacy requests may omit the field. Empty, truncated, malformed or schema-invalid decisions still fail strict validation after known usage is settled. JSON Output does not replace authorization, tool argument validation or citation verification. See [DeepSeek JSON Output](https://api-docs.deepseek.com/guides/json_mode/).

只读工具模型遇到混合请求时拒绝越权部分,继续执行独立且已授权的部分;用户已明确要求的合法只读操作应直接调用工具,不再次征求确认或只提出执行建议。提交分析缺少明确 ID 或可靠会话选择时,不调用提交选择工具或用最近提交列表代替澄清;混合请求先执行独立授权的只读查询,再在最终答案中询问缺失的提交 ID。工具仍绑定当前服务端会话,不能因请求要求切换身份。隔离验收同时要求没有泄露和本人数据的正向工具对照,不能以整段拒绝冒充完整通过。

授权周期的 `authorized_budget_period` 仍只保存生命周期元数据;其快照始终明确
Expand Down Expand Up @@ -494,6 +496,7 @@ The same explicitly authorized rollover contract applies to the later supported
| `acceptance-revalidation-v12` | V11 |
| `acceptance-revalidation-v13` | V12, including its retained unknown request |
| `acceptance-revalidation-v14` | Settled V13 on the integrated U03 candidate |
| `acceptance-revalidation-v15` | Settled V14, including its invalid-JSON failure |

Before preparation, retain the predecessor ledger, binding and guard at their pinned
fingerprints and preserve the entire earlier history chain. The selected policy's
Expand Down Expand Up @@ -526,6 +529,12 @@ commitments and inherited unknown liabilities remain cumulative. Failed protocol
evidence stays in the predecessor, while the integrated candidate receives a fresh
complete acceptance prefix without changing the existing purpose or token limits.

V15 carries the permanently halted V14 ledger, binding and guard into the JSON-output
candidate under the same cumulative USD200 authorization. Keep the invalid-JSON
failure and all known costs in V14; do not resume it or reinterpret it as passing
acceptance. The earlier three unknown liabilities and every purpose/token cap remain
unchanged. A new complete prefix must establish acceptance for the changed adapter.

Issue the evidence-bound U02 gate only after those inputs validate. Gate artifact references are
relative to the private directory containing the gate; place the referenced artifacts there:

Expand Down
4 changes: 2 additions & 2 deletions services/agent/data/repository_corpus_manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
},
{
"doc_id": "repository-development",
"version": "sha256-24c1cea137aab6834ef530d75fdf95984ebbe55bafc5322d6173a6c65e8a5a89",
"chunk_id": "repository-development:sha256-24c1cea137aab6834ef530d75fdf95984ebbe55bafc5322d6173a6c65e8a5a89:1",
"version": "sha256-f26b77d34a904e1fae301357a5569b657309e95ac1a84ec1e6b6a6d735f130c3",
"chunk_id": "repository-development:sha256-f26b77d34a904e1fae301357a5569b657309e95ac1a84ec1e6b6a6d735f130c3:1",
"source_path": "docs/DEVELOPMENT.md",
"access_scope": "repository-public",
"sample_kind": "real",
Expand Down
2 changes: 2 additions & 0 deletions services/agent/src/agent_service/gate.py
Original file line number Diff line number Diff line change
Expand Up @@ -988,6 +988,8 @@ def _check_dav58_raw_judge(
expected_body = {
"model": "deepseek-flash",
"messages": [{"role": "system", "content": system}, {"role": "user", "content": user_prompt}],
**({"response_format": {"type": "json_object"}}
if isinstance(request_body, dict) and "response_format" in request_body else {}),
"temperature": 0, "max_tokens": 2000, "thinking": {"type": "disabled"},
}
body_sha = hashlib.sha256(json.dumps(
Expand Down
14 changes: 14 additions & 0 deletions services/agent/src/authorized_budget_period.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
REVALIDATION_V12_POLICY_ID = "acceptance-revalidation-v12"
REVALIDATION_V13_POLICY_ID = "acceptance-revalidation-v13"
REVALIDATION_V14_POLICY_ID = "acceptance-revalidation-v14"
REVALIDATION_V15_POLICY_ID = "acceptance-revalidation-v15"
REVALIDATION_POLICY_IDS = frozenset({REVALIDATION_POLICY_ID, REVALIDATION_V2_POLICY_ID, REVALIDATION_V3_POLICY_ID, REVALIDATION_V4_POLICY_ID, REVALIDATION_V5_POLICY_ID, REVALIDATION_V6_POLICY_ID, REVALIDATION_V7_POLICY_ID, REVALIDATION_V8_POLICY_ID, REVALIDATION_V9_POLICY_ID, REVALIDATION_V10_POLICY_ID, REVALIDATION_V11_POLICY_ID, REVALIDATION_V12_POLICY_ID})
REVALIDATION_V2_HISTORY = MappingProxyType({
"attempts": 164, "known_actual_micro_usd": 42_307,
Expand Down Expand Up @@ -227,6 +228,17 @@
})
REVALIDATION_POLICY_IDS = REVALIDATION_POLICY_IDS | {REVALIDATION_V14_POLICY_ID}

REVALIDATION_V15_HISTORY = MappingProxyType({
"attempts": 1183, "known_actual_micro_usd": 330_069,
"known_committed_micro_usd": 10_440_914,
"unknown_attempts": 3, "unknown_encumbrance_micro_usd": 2_359_296,
"cumulative_attempt_limit": 1424, "cumulative_limit_micro_usd": 200_000_000,
})
REVALIDATION_V15_POLICY = MappingProxyType({
**REVALIDATION_V14_POLICY, "history": REVALIDATION_V15_HISTORY,
})
REVALIDATION_POLICY_IDS = REVALIDATION_POLICY_IDS | {REVALIDATION_V15_POLICY_ID}


def policy_for(policy_id: str):
if policy_id == POLICY_ID:
Expand Down Expand Up @@ -259,6 +271,8 @@ def policy_for(policy_id: str):
return REVALIDATION_V13_POLICY
if policy_id == REVALIDATION_V14_POLICY_ID:
return REVALIDATION_V14_POLICY
if policy_id == REVALIDATION_V15_POLICY_ID:
return REVALIDATION_V15_POLICY
raise PeriodError("unsupported policy")


Expand Down
1 change: 1 addition & 0 deletions services/agent/src/boundary_evaluation.py
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,7 @@ def _request_body(
body: dict[str, object] = {
"model": getattr(model, "_model", None),
"messages": api_messages(messages),
"response_format": {"type": "json_object"},
"temperature": 0,
"max_tokens": getattr(model, "_max_tokens", None),
}
Expand Down
3 changes: 2 additions & 1 deletion services/agent/src/dav58_live_guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,8 @@ def begin(self, request: httpx.Request, lane: str, reservation=None):
raise ModelBudgetExceeded("guard and budget history disagree")
body = json.loads(request.content)
if (not isinstance(body, dict)
or set(body) != {"model", "messages", "temperature", "max_tokens", "thinking"}
or set(body) - {"response_format"} != {"model", "messages", "temperature", "max_tokens", "thinking"}
or ("response_format" in body and body["response_format"] != {"type": "json_object"})
or body["model"] != "deepseek-flash"
or body["thinking"] != {"type": "disabled"}
or type(body["max_tokens"]) is not int
Expand Down
1 change: 1 addition & 0 deletions services/agent/src/deepseek_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ async def decide(self, messages: list[dict[str, object]]) -> ModelDecision:
body: dict[str, object] = {
"model": self._model,
"messages": api_messages,
"response_format": {"type": "json_object"},
"temperature": 0,
"max_tokens": self._max_tokens,
}
Expand Down
26 changes: 22 additions & 4 deletions services/agent/src/revalidation_history.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,14 +268,30 @@ def _check_unknown_rollover(snapshot, guard, *, predecessor="v2"):
"dav58-increment-10a5b715a8dc4a6b96056116a789abea.json": "a4f08105519da4c96c258625676b92d428028066da3e230c8a1e7bad2a8607a8",
}

V14_ROLLOVER_IDENTITY = {
"period_id": "revalidation-v14-20261011",
"identity": "38633ca751c44e94b0ae63d17e672bde",
"config_sha256": "a650011b552cde66b265fb030daaba824ee1f660d05f48326693ff4e8210d7c1",
"policy_id": "acceptance-revalidation-v14",
}
V14_ROLLOVER_SHA256 = {
"budget.sqlite3": "46bc407b736cae0ad7d9c271b028f6556c363eb7a5de5906699b7636c56a58c6",
"binding.json": "d8f7384635e3a7d2f103bda75cf39dd1027966fef34b6a6960cc93b7ee355bb2",
"dav58-increment-38633ca751c44e94b0ae63d17e672bde.json": "605aababbad617e0d25f2c26103e5ece9960f68536dd4374fc526d353fbada85",
}


def _validate_rollover_history(sources, *, settled_policy="v1"):
from authorized_budget_period import REVALIDATION_V14_HISTORY
from authorized_budget_period import REVALIDATION_V14_HISTORY, REVALIDATION_V15_HISTORY
from authorized_budget_period import PeriodError, PeriodIdentity, REVALIDATION_V2_HISTORY, REVALIDATION_V4_HISTORY, REVALIDATION_V5_HISTORY, REVALIDATION_V6_HISTORY, REVALIDATION_V7_HISTORY, REVALIDATION_V8_HISTORY, REVALIDATION_V9_HISTORY, REVALIDATION_V10_HISTORY, REVALIDATION_V11_HISTORY, REVALIDATION_V12_HISTORY, _parent, _file
from model_budget import ModelBudget, authorization_slot
from dav58_live_guard import IncrementalGuard

if settled_policy == "v13":
if settled_policy == "v14":
original = _validate_rollover_history(sources, settled_policy="v13")
identity, fingerprints = V14_ROLLOVER_IDENTITY, V14_ROLLOVER_SHA256
totals, history = (97, 33_416, 872_400), REVALIDATION_V15_HISTORY
elif settled_policy == "v13":
original = _validate_unknown_rollover_history(sources, predecessor="v12")
identity, fingerprints = V13_ROLLOVER_IDENTITY, V13_ROLLOVER_SHA256
totals, history = (98, 33_738, 877_200), REVALIDATION_V14_HISTORY
Expand Down Expand Up @@ -351,7 +367,7 @@ def _validate_rollover_history(sources, *, settled_policy="v1"):
try:
# Validate sealed receipts without making the permanently halted journal resumable.
receipt_view = guard
if settled_policy == "v13":
if settled_policy in {"v13", "v14"}:
if guard.get("halted") is not True:
raise PeriodError("previous guard must be permanently halted")
receipt_view = {**guard, "halted": False}
Expand All @@ -364,12 +380,14 @@ def _validate_rollover_history(sources, *, settled_policy="v1"):


def validate_history(sources, *, policy_id="acceptance-revalidation-v1"):
from authorized_budget_period import REVALIDATION_V13_POLICY_ID, REVALIDATION_V14_POLICY_ID
from authorized_budget_period import REVALIDATION_V13_POLICY_ID, REVALIDATION_V14_POLICY_ID, REVALIDATION_V15_POLICY_ID
from authorized_budget_period import PeriodError, REVALIDATION_HISTORY, _parent, _file
from authorized_budget_period import REVALIDATION_POLICY_ID, REVALIDATION_V2_POLICY_ID, REVALIDATION_V3_POLICY_ID, REVALIDATION_V4_POLICY_ID, REVALIDATION_V5_POLICY_ID, REVALIDATION_V6_POLICY_ID, REVALIDATION_V7_POLICY_ID, REVALIDATION_V8_POLICY_ID, REVALIDATION_V9_POLICY_ID, REVALIDATION_V10_POLICY_ID, REVALIDATION_V11_POLICY_ID, REVALIDATION_V12_POLICY_ID

if policy_id == REVALIDATION_V14_POLICY_ID:
return _validate_rollover_history(sources, settled_policy="v13")
if policy_id == REVALIDATION_V15_POLICY_ID:
return _validate_rollover_history(sources, settled_policy="v14")
if policy_id == REVALIDATION_V13_POLICY_ID:
return _validate_unknown_rollover_history(sources, predecessor="v12")
if policy_id == REVALIDATION_V2_POLICY_ID:
Expand Down
17 changes: 16 additions & 1 deletion services/agent/tests/test_dav58_live_guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,9 @@ async def run():


@pytest.mark.parametrize("changes", [{"model": "deepseek-v4-pro"}, {"thinking": {"type": "enabled"}},
{"max_tokens": 2001}, {"stream": True}, {"tools": []}])
{"max_tokens": 2001}, {"stream": True}, {"tools": []},
{"response_format": {"type": "text"}}, {"response_format": None},
{"response_format": {"type": "json_object", "extra": True}}])
def test_unexpected_paid_payload_rejected_before_send(tmp_path, changes):
guard = IncrementalGuard(tmp_path / "increment.json")
body = {"model": "deepseek-flash", "messages": [{"role": "user", "content": "x"}], "max_tokens": 2000,
Expand All @@ -182,6 +184,19 @@ def test_unexpected_paid_payload_rejected_before_send(tmp_path, changes):
assert guard.state["receipts"] == []


def test_guard_accepts_provider_json_output(tmp_path):
guard = IncrementalGuard(tmp_path / "increment.json")
body = {"model": "deepseek-flash", "messages": [{"role": "system", "content": "Reply in JSON."}],
"max_tokens": 2000, "temperature": 0, "thinking": {"type": "disabled"},
"response_format": {"type": "json_object"}}
try:
entry = guard.begin(httpx.Request("POST", "https://api.deepseek.com/chat/completions", json=body),
"dav58_loop")
assert entry["lane"] == "dav58_loop"
finally:
guard.close()


def test_existing_journal_cannot_reset(tmp_path):
path = tmp_path / "increment.json"
IncrementalGuard(path)
Expand Down
20 changes: 20 additions & 0 deletions services/agent/tests/test_deepseek_model.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import asyncio
import json

import deepseek_model
import httpx
Expand All @@ -7,6 +8,25 @@
from deepseek_model import DeepseekModel, ModelProtocolError, _parse_decision


@pytest.mark.parametrize("tools,content", [
({}, '{"answer":"ok"}'),
({"get_problem": "args"}, '{"tool":"get_problem","args":{}}'),
])
def test_decision_requests_provider_json_output(tools, content):
async def handler(request):
payload = json.loads(request.content)
assert payload["response_format"] == {"type": "json_object"}
assert "json" in payload["messages"][0]["content"].lower()
return httpx.Response(200, json={"choices": [{"message": {"content": content}}]})

async def scenario():
async with DeepseekModel("test-key", tool_specs=tools,
transport=httpx.MockTransport(handler)) as model:
await model.decide([{"role": "user", "content": "question"}])

asyncio.run(scenario())


@pytest.mark.parametrize(
"response",
[
Expand Down
18 changes: 18 additions & 0 deletions services/agent/tests/test_v15_json_history.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
from authorized_budget_period import policy_for


def test_json_acceptance_retains_all_known_costs_and_unknown_liabilities():
old = policy_for("acceptance-revalidation-v14")
policy = policy_for("acceptance-revalidation-v15")
history = policy["history"]
assert history["attempts"] == old["history"]["attempts"] + 97 == 1183
assert history["known_actual_micro_usd"] == old["history"]["known_actual_micro_usd"] + 33416 == 330069
assert history["known_committed_micro_usd"] == old["history"]["known_committed_micro_usd"] + 872400 == 10440914
assert history["unknown_attempts"] == old["history"]["unknown_attempts"] == 3
assert history["unknown_encumbrance_micro_usd"] == old["history"]["unknown_encumbrance_micro_usd"] == 2359296
assert history["cumulative_limit_micro_usd"] == old["history"]["cumulative_limit_micro_usd"] == 200000000
assert history["cumulative_attempt_limit"] == history["attempts"] + policy["attempts"] == 1424
assert policy["lanes"] == old["lanes"]
assert policy["limit_micro_usd"] == old["limit_micro_usd"]
assert (history["known_committed_micro_usd"] + history["unknown_encumbrance_micro_usd"]
+ policy["limit_micro_usd"] <= history["cumulative_limit_micro_usd"])
Loading