Keep fallback retrieval failures distinct from empty results

[Maya-Kid]

Latest dev synchronization (2026-09-21)

Based on dev@474b904dc3e8014d42404a3c067af9906036a953; current head 5071b883b7fcfcfe6a745e49dd788817a5ed42ac.

Revalidated the combined integration tree 6cd976351b4d7cc7389730fce530526f4d197992 on isolated Linux PostgreSQL fixtures (all 675 tracked source files hash-matched):

• Rust fmt, strict workspace/all-target Clippy, workspace build: pass. Workspace tests: 1,015 passed, 0 failed, 5 ignored. The new upstream delivery tests were additionally run on an exclusive database as CONTRIBUTING requires: both parent delivery/process-kill tests and the busy-deferral unit test passed. The crash-child entry point ran through its parent, not directly; the existing public-network RSS test remains ignored.
• Combined frontend: 129 module tests, 27 browser subtests (29 including parent suites); build/typecheck/guard passed. Browser fixtures ran serially.
• Five browser scenarios against the actual Linux server and PostgreSQL passed: 65-conversation pagination, stale view ownership, citation preview/reattachment/reload, history-to-idle handoff, and actual assistant INSERT failure. Model replies were locally scripted; this is not a new paid-model accuracy evaluation.
• The single PostgreSQL Chat CI command, cargo test -p utopia-server api::chat::, listed and executed 47 tests. All 37 introduced tests were observed passing: Hand off exhausted tool runs to an evidence-only final answer #845 22, Keep older chat handles from unregistering newer generations #848 1, Publish citation sources added by document reads #849 2, Keep fallback retrieval failures distinct from empty results #850 4, Preserve exact chat outcomes across stream closure and reattachment #851 3, Do not report fallback chat completion when saving fails #852 3, and upstream Assert a chat turn ends in exactly one earned terminal #858 2. No test module was deleted during CI consolidation.
• Seven real filtered runs each omitted one group. Cargo returned green, but the independent execution-coverage checker returned exit 1 for every omission; the complete run returned exit 0. All four shared terminal-contract table rows executed without pending skips.

The evidence-only handoff core remains frozen. Earlier validation below belongs to its stated historical heads. Automated browser checks do not replace the maintainer's requested visual review.

---

When a provider rejects tool calling, legacy_rag currently converts any retrieval error into an empty result and requests an ordinary answer. A database failure can therefore look like missing evidence and end with done.

Handle retrieval errors explicitly: log the diagnostic server-side, send the existing safe error event, and stop before requesting or saving a fallback answer. Successful empty retrieval and embedding-to-BM25 degradation keep their existing behavior.

Linux validation against an isolated migrated PostgreSQL database, using real Chat handlers and a local HTTP model substitute:

• Reproduced the original false-success path by closing the fixture's database pool only after the initial tool request reached the upstream. Authentication/settings and user persistence had already succeeded; fallback retrieval then returned a real database access error. The original code still requested and streamed a plain answer with done; the fix emits error, sends no plain-model request, and stores no assistant answer.
• Four new regressions pass: retrieval failure, genuinely empty search, failed embedding with a real BM25 hit, and provider 401/402/429/500 errors. Assertions include the existing two-request compatibility negotiation for 400/422.
• Formatting, strict workspace clippy, all 14 Chat tests and all 6 retrieval tests pass with UTOPIA_TEST_REQUIRE_DB=1. CI adds the guarded fallback regressions to the PostgreSQL job.

No paid endpoint or business data was used. The change does not alter fallback eligibility, retrieval ranking, the empty-result prompt, persistence policy, or #845's finalization boundary. Client EOF semantics are a separate workstream; this PR guarantees the server emits an error without a success terminal.