Repository navigation
[89318757] Backend: board-program cost-per-cycle vs approval-rate cadence review - #1455
roboco-app[bot] wants to merge 5 commits into
Conversation
…l rate (#1452) * [458d14ca] feat(board-programs): per-program cost-per-cycle vs approval-rate metric service * [458d14ca] docs(board-programs): document cadence metric service and fields --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech>
…t, plus CEO review (#1454) * [e37cf7ef] feat(board-programs): slow cadence when approval rate does not justify cycle cost * [e37cf7ef] fix(board-programs): share base interval, savepoint cadence lookups, tighten tests Entire-Checkpoint: 01M6J2WDMHC6JVJAS1NXMTE5PY * docs: Board Program cadence-slowing policy and CEO cadence review --------- Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech>
|
Thanks for opening your first pull request on RoboCo! Quick checklist before review (most of these are enforced by CI, but worth a glance):
See CONTRIBUTING.md for the full workflow and the Code of Conduct for the community standards we follow. Welcome aboard — a maintainer will review shortly. |
There was a problem hiding this comment.
In-path PR-review gate — CHANGES REQUESTED 🔴
Reviewed by pr_reviewer (RoboCo PR reviewer). Posted by the project bot account; the gate verdict is authoritative in RoboCo.
[F-b202dcfd] docs/rag/architecture/board-programs.md:85 (major) — The KB-indexed RAG doc describes the shipped state: the metric's consumer (cadence policy), the CEO routes and the three board_cadence_* settings exist in this same PR. → Says the metric 'has no route, no policy and no setting yet: the cadence-policy task is its intended consumer' — false once this assembly merges. → Rewrite that sentence to point at BoardProgramCadencePolicyService, GET /board-programs/cadence-review, POST /{key}/cadence-slowdown and the board_cadence_* settings (or fold the policy section in here, see next finding).
[F-053d2ca5] docs/architecture/board-program-cadence-policy.md:1 (minor) — Docs placed per CLAUDE.md: agent-facing corpus in docs/rag/ (auto-indexed), specs/working notes in docs/internal/. → Creates a new top-level docs/architecture/ directory (this is its only file), outside the KB-indexed corpus, so the policy/settings/reversal doc is invisible to agents. → Move the content into docs/rag/architecture/board-programs.md as a 'Cadence policy and CEO review' section (or docs/rag/architecture/board-program-cadence-policy.md) and delete docs/architecture/.
[F-7a807cba] roboco/foundation/policy/board_program_cadence.py:50 (minor) — approved_per_cycle uses the same cycle population as the numerator: items_approved and cycles_counted both from cost-attributable cycles (or both from all cycles). → items_approved sums ALL cycles (incl. cycles_missing_cost) but is divided by cycles_counted (cost-known cycles only), inflating approvals per cycle and biasing against slowing. → Have the metric service also return approvals over counted cycles only (e.g. items_approved_counted via a filtered sum in _cycle_aggregates) and feed that to decide_slowdown; add one test with a missing-cost cycle that carries approvals.
…ove cadence docs to rag (#1479) Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
There was a problem hiding this comment.
In-path PR-review gate — CHANGES REQUESTED 🔴
Reviewed by pr_reviewer (RoboCo PR reviewer). Posted by the project bot account; the gate verdict is authoritative in RoboCo.
[F-af7e364f] — (blocker) — CI check 'ci' green on the assembled PR #1455 head commit before the gate can pass. → CI failing: ci. The pr_pass CI guard refused the transition. The diff review itself found no blocking defects, and make gate passes locally. → Open the failing 'ci' run on PR #1455, fix the failing step (most likely the pytest stage: the new real-Postgres tests in tests/unit/services/test_board_program_cadence.py and test_board_program_cadence_policy.py, or the coverage floor), push, and re-submit.
[F-23e0a4b9] roboco/services/board_program_cadence_policy.py:110 (minor) — The CEO review shows the exact numbers the decision used, so that every slowdown can be reproduced from the row (AC3). → The row shows the all-cycle items_approved but not items_approved_counted or cycles_counted, which decide_slowdown actually uses. With deleted-task cycles, the row can show approvals while cost_per_approved is null (infinite). → Add cycles_counted and items_approved_counted to ProgramCadenceReview and fill them from the metric in review().
#1508) * [188c8d18] fix(cadence): expose counted fields in review and map cadence modules * [188c8d18] docs(board-programs): document counted fields on the cadence review row --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech>
There was a problem hiding this comment.
In-path PR-review gate — CHANGES REQUESTED 🔴
Reviewed by pr_reviewer (RoboCo PR reviewer). Posted by the project bot account; the gate verdict is authoritative in RoboCo.
[F-bb04c081] — (blocker) — CI check 'ci' is green on the head commit of assembled PR #1455 before the gate can pass. → CI is still failing (ci) on the head commit, and the pr_pass CI guard refused the transition again in round 3. → Open the failing 'ci' run on PR #1455 and read the failing step's log. The new real-Postgres tests (tests/unit/services/test_board_program_cadence.py, test_board_program_cadence_policy.py) and the coverage floor are the likely suspects. Fix it, push, and confirm CI is green before re-submitting.
[F-8c28c13a] roboco/api/routes/board_programs.py:55 (nit) — The new state-changing POST /{key}/cadence-slowdown carries the same guard_deco.rate_limit / block_clouds decorators as the sibling POST /{key}/run-now. → The route has no guard_deco decorators. → Optional: add @guard_deco.rate_limit(requests=30, window=60) and @guard_deco.block_clouds() to match run-now.
Root cause: c90b249 made BoardProgramEngine._run_due_one call get_board_program_cadence_policy_service(session).effective_interval, which reads settings and metrics through the AsyncMock session stub in test_decisions_b_infra.py. The leaked never-awaited AsyncMock coroutines were finalized during the next test, test_b13_rotation_error_keeps_ deterministic, and filterwarnings=error turned them into a failure. Stub the policy in the two _run_due_one tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Goal: the CEO gets a cadence review of every board program showing cost per cycle against that program's proposal approval rate, and programs whose approval rate does not justify a $50 cycle are slowed automatically.
Intake observations, forwarded verbatim: Board program cycles now cost as much as top delivery work. The Scales rebalance cycle is the second most expensive task all-time at $54.86, just behind the top delivery task at $55.51 ('Add IP/provider/overlap validators to RuleUpdate schema'), and it produces no shipped artifact, only proposals. With a dozen board programs on periodic cadence that is a recurring line item. roboco-api at $808.11 is the largest project spend (guard-core-saas-backend $607.17, frontend $256.27). The intake gave no file:line targets; locate the board program scheduler, cycle cost accounting and proposal approval data yourselves.
Constraints: logic in roboco/services, data models in roboco/models, routes thin. The $50 threshold must be configurable. Require a minimum sample of proposals before slowing a program, and surface every slowdown in the review so the CEO can reverse it.
Suggested units, refine each into its own developer leaf: (1) metric: per-program cost per cycle joined to proposal approval rate, exposed from a service; independent. (2) policy: slow cadence when approval rate does not justify the cycle cost, plus the CEO review output; depends on the interface from unit 1, but its policy logic and tests can proceed in parallel against that interface. Use both developers in parallel where it genuinely splits.