feat: add knowledge param and knowledge result models to search (DX-835)

[tyler5673]

Summary

Supersedes #62, #63 and #64 — each closed to keep the review readable. The work
is unchanged and carries here as a single squashed commit, so per-commit SHA
references from those PRs no longer apply.

Adds the knowledge request parameter on POST /v1/search and the knowledge
result models that come back with it.

Request — knowledge="core" on search() / search_async() and the
deprecated you.search.unified* shims. Normalized to lowercase like the other
enum-typed params. An invalid value raises ValidationError locally before any
network call, mirroring the server's 422 (same fail-fast contract as
extraction). Every parameter on these methods is keyword-only, so inserting
knowledge mid-signature is non-breaking — hence a minor bump to 3.5.0.

Response — Results.knowledge is an optional List[KnowledgeResult]. Each
result carries type, title, and attribution, plus description and an
optional as_of date for type: answer results. KnowledgeAttribution carries
name and an optional source_description.

Two contract details drove the model design:

• type is a plain str, not an enum. The spec says to ignore an unrecognized
  value rather than fail on it, since a new kind may populate a different set of
  fields. An enum here would turn a future server-side addition into a client
  crash.
• The API omits results.knowledge when nothing relevant is found rather
  than returning [], so the parsed field is None. Docs and examples iterate
  with or [].

Contract grounding

Grounded only against official public surfaces:

• https://you.com/docs/openapi/web-search.json — the only spec of the seven
  published specs that mentions knowledge at all
• https://docs.you.com/api-reference/search/v1-search.md
• https://docs.you.com/administration/billing.md

Also verified against the live prod API: enum shape, the omitted-key behavior,
as_of genuinely optional, invalid value → 422, and that count does not
cap knowledge (a count=1 call still returned 2 knowledge results — knowledge
has its own documented limit of 25).

The implementation is scoped to exactly what the public spec defines: no value
beyond the one published there is accepted or modeled.

Also in this PR: drift-checker blind spot

scripts/check_drift.py compared only top-level response properties, so a
new nested field like results.knowledge was invisible — the checker reported
the request-param drift but not the response drift. This is why the original
warning named only the parameter.

The response check now recurses through nested object schemas and arrays. It
falls back to the previous flat comparison when a response resolves to a oneOf
union rather than a single object schema, so union responses are not mistaken
for empty schemas.

Recursion immediately surfaced two pre-existing gaps unrelated to Knowledge.
Both are now closed by adding the fields rather than suppressed:

Spec	Field	Was missing in SDK	Now
answer	results.web	description, thumbnail_url	added to AnswerSearchResult
finance-research	output.sources	snippets	added to FinanceResearchSource

The answer one was live data loss. Prod returns description on 20/20 web
results and thumbnail_url on 16/20, and both were discarded at parse time.
WebResult on the search endpoint already declared them, so AnswerSearchResult
was simply the narrower of two siblings describing the same wire shape.

The finance one was not losing data yet — two live calls at deep and
exhaustive returned 10 sources carrying only title and url — but the spec
defines snippets and the sibling Source model on the Research API already
declared it with an identical description. It was added anyway, because the
suppression holding it back can only notice the SDK catching up, never the
API starting to honour its own spec, so that gap would have stayed silent from
the API side indefinitely.

KNOWN_RESPONSE_GAPS is therefore empty, and drift --verbose reports no
drift with an empty table — which is the real proof the gaps closed rather than
got hidden. Both suppression tables stay self-invalidating: an entry is reported
stale once the side it excuses catches up, and that path is negative-tested so it
is still live for the next gap.

Negative-tested the fix three ways: dropping knowledge from Results is
caught; a stale suppression entry is reported; and a simulated spec enum gain is
caught by the enum check.

Two follow-ups hardened that recursion:

• visited was a global cache, not a cycle breaker. It returned early
  whenever the same pydantic class reappeared, so a model used at two response
  paths backed by different schemas was compared only at the first and real
  drift on later branches went unreported. Reproduced before fixing, with a root
  model whose two fields share a type but whose spec schemas differ by one field:
  only the first path was reported. visited is now discarded on exit
  (try/finally), so it breaks cycles without suppressing sibling branches.
  Checked against self-referential (Node.child -> Node) and mutual
  (A.b -> B.a -> A) schemas: both still terminate, and drift inside the cycle
  is still reported.
• Fixing that unmasked a third gap, in the opposite direction — a field the
  SDK declares that the spec schema at that path does not:
  results.news.contents carries highlights in the SDK but not in the spec.
  results.web[].contents resolves to WebContentsPost, which defines
  highlights, while results.news[].contents resolves to the narrower
  Contents schema (html, markdown), and the SDK shares one Contents model
  across both. Settled against prod rather than by inspection: with
  extraction_mode="highlights" news items carry no contents at all, and with
  the deprecated livecrawl="all" they carry html only, so highlights never
  arrives on the news path and the spec is the accurate side. Narrowing the model
  would be a breaking change for no behavioral gain, so
  KNOWN_SHARED_MODEL_EXTRAS records the gap — mirroring KNOWN_RESPONSE_GAPS
  and self-invalidating the same way.

Negative-tested eight ways in total:

1. dropping knowledge from Results is caught;
2. a stale KNOWN_RESPONSE_GAPS entry is reported (re-checked after the table
   went empty, by injecting an entry for a field the SDK now defines);
3. a stale KNOWN_SHARED_MODEL_EXTRAS entry is reported (injected html, which
   the spec does define at that path);
4. a field the spec dropped is not reported stale — the case the original
   known & model_fields intersection got wrong;
5. a simulated spec enum gain is caught by the enum check;
6. drift on a second branch that reuses the same model is caught instead of
   skipped;
7. a self-referential schema (Node.child -> Node) terminates and still reports
   drift inside the cycle;
8. a mutual cycle (A.b -> B.a -> A) terminates and still reports drift inside
   the cycle.

scripts/check_drift.py had no committed test coverage at all, which is why two
bugs in one function — the visited cache and the staleness intersection —
surfaced through review rather than through a failing test, and why every
reproduction above was hand-written and then thrown away.
tests/test_check_drift.py (22 tests) now pins those rules permanently:
response recursion including a sibling branch that reuses a model, cycle safety
for self-referential and mutual schemas, all three staleness cases for both
suppression tables, and the _resolve_schema / _nested_model helpers. Both
regressions were confirmed to actually fail when their fix is reverted and pass
when it is restored, so they are regression tests rather than mere coverage.

Also in this PR: a live wire audit

check_drift.py compares the published specs against the models. That is blind
to a field the API returns but no spec declares — the model drops it at parse
time and every static check still agrees with itself, reporting no drift. That
is precisely how AnswerSearchResult came to lose description and
thumbnail_url: the spec does declare both, but the fact that settled it (prod
actually sending them, on 20/20 and 16/20 results) only came from reading the
wire.

scripts/audit_wire.py closes the gap. It wraps the transport, keeps the raw
JSON, and walks it next to the parsed model, reporting any key no field
consumed. Across all seven endpoints that is 228 wire keys, none unexplained.
It needs YDC_API_KEY, so it is a pre-release check rather than a CI gate;
--strict exits 1 on an unexplained drop, negative-tested by emptying the known
list and confirming it fails.

Two keys are recorded in KNOWN_WIRE_EXTRAS as observed-but-undeclared rather
than modeled — Documented skips below explains why each is a decision rather than
an oversight.

Verification

Gate	Result
pytest tests/ (CI gate, live + perf excluded)	450 passed
pytest tests/test_live.py (full suite against prod)	46 passed — incl. all 15 research/finance cases
Recovered response fields, live against prod	description 20/20 and thumbnail_url 16/20 answer web results now parse; previously all dropped
check_drift.py with an empty KNOWN_RESPONSE_GAPS	no drift — proof the two gaps closed rather than got hidden
mypy src/youdotcom/ scripts/	clean, 87 files
pylint src/youdotcom/ --disable=all --enable=E (CI gate)	10.00/10
python scripts/check_drift.py --verbose / --strict	no drift, exit 0
Go mock server	go build ./... clean; gofmt clean on the edited handler
uv build --sdist --wheel	all three new modules present in the wheel
New examples/ function	run live against prod, output correct
Docs Type cells vs. real annotations (typing.get_type_hints)	151 rows across every docs/models/ field table, all four AGENTS.md traps: 0 drift
Enum ## Values pages vs. code members	15 pages, all match
AGENTS.md cross-table identity for knowledge	identical across all 5 surfaces
tests/README.md vs. pytest --collect-only	all 21 files listed; group counts sum to 450 = the CI gate
scripts/audit_wire.py against prod	7 endpoints, 228 wire keys, 0 unexplained drops
Model fields with no row in their docs page	all 65 pages; the 1 gap found (researchresponse.md) is fixed
scripts/check_drift.py regression suite	22 tests; both fixed bugs proven to fail when their fix is reverted
Every Python block in docs/, README.md, USAGE.md vs. the copy-paste-runnable and timeout_ms rules	107 blocks across 96 files scanned; the 2 genuine violations fixed here

Response composition is not assumed

Results.web, .news and .knowledge are all Optional, and results itself
is Optional. Which sections a given call returns depends on the query and the
parameters, and the published schema marks none of them required, so the SDK does
not assume any particular combination arrives together.

That is a deliberate design choice, and it is why nothing in this PR couples
knowledge to the other sections:

• No docstring, doc page or example states that web or news accompany
  knowledge results.
• The live knowledge tests assert only on results.knowledge, never on
  results.web, so they neither encode nor depend on a particular composition.
• Every documented snippet guards both levels — if res.results: and or [] —
  because either can legitimately be absent.
• docs/models/results.md states the optionality explicitly and shows the guard
  pattern. It was previously a bare field table with no prose, unlike its sibling
  searchresponse.md. The example on that page was executed verbatim against the
  live API and runs clean.

Live testing during this PR saw the section mix vary across queries run with
identical parameters, including queries that returned no sections at all. That is
the behavior the optionality exists to absorb, and it is why composition is
treated as a runtime property rather than a documented guarantee: if it changes
server-side, no SDK change is required.

Two related things deliberately not done:

• No runtime warning about which sections came back. It would fire on
  ordinary, correct responses, and it would assert the SDK knows the server's
  intent.
• No documenting an observed composition as contract. Writing "these are the
  sections you get when knowledge is set" into the docs would be wrong as soon
  as that changes, and nothing would catch the staleness — check_drift.py
  compares field names, enums, endpoints and servers, never descriptions.

The Knowledge enum docstring stays verbatim against
components.schemas.Knowledge.description, per the convention that model
docstrings mirror the published spec. The SDK-authored surfaces — the parameter
tables, both field docstrings, _search_impl's :param knowledge:, and
docs/models/knowledge.md — describe where results land without asserting
anything about the other sections.

Also worth noting: not every query yields knowledge results. The API omits
results.knowledge entirely when nothing relevant is found, so the parsed field
is None rather than an empty list. The live tests use queries re-verified
against the live API.

Surface sweep

Models, exports, request plumbing, shims, tests, docs, examples, changelog, and
version — following the AGENTS.md checklist:

• New models — Knowledge, KnowledgeResult, KnowledgeAttribution, each
  with TypedDict + Pydantic docstrings in lockstep and optional_fields in
  serialize_model. Exported in all three places in models/__init__.py.
• Threading — _build_search_request, _search_impl, search_async, and
  all three SearchShim methods.
• Docs — 3 new model pages, rows added to results.md,
  searchrequestbody.md, docs/sdks/search/README.md, and
  docs/sdks/you/README.md (search section only, not the answer section).
  Type cells mirror the annotations including Optional[...].
• Prose — README.md gains a "Knowledge results" subsection, USAGE.md gains a
  [knowledge] snippet block, examples/api-example-calls.py gains a function
  plus its FUNCTIONS menu entry.
• Tests — 21 unit tests in tests/test_knowledge.py, 6 live tests, 1 perf
  case, and knowledge added to tests/test_param_normalization.py alongside
  its normalized siblings.
• Version — 3.5.0 in pyproject.toml, _version.py, and uv.lock.

Documented skips

Per AGENTS.md, each skipped surface and why:

1. No MIGRATION.md section — additive and non-breaking, nothing to migrate.
2. src/youdotcom/models/searchop.py (SearchRequest) gets no knowledge field.
   The published spec (https://you.com/docs/openapi/web-search.json) defines exactly
   one operation, POST /v1/search, whose body is SearchRequestBody — there is no GET
   operation and no SearchRequest schema to sync against. SearchRequest is a
   query-param model (QueryParamMetadata(style="form", explode=True)) that the SDK
   never puts on a request path (it builds SearchRequestBody), and its fields already
   differ by design: include_domains is a comma-separated Optional[str] here versus
   Optional[List[str]] on the body. Adding knowledge would advertise a capability
   with no public contract basis. It likewise has no extraction, dating to 3.1.0.
   docs/models/searchrequest.md is accurate against the model as it stands; its one
   wrong type cell (livecrawl_formats) is fixed here.
3. No knowledge pricing documentation — the public billing page does not
   mention knowledge, so adding a price claim would be unsupported.
4. MCP parity out of scope for this PR.
5. WebResult.original_thumbnail_url not modeled — prod returns it on web
   results and no published spec declares it, so check_drift.py cannot see it
   from either side. Modeling a field observed only on the wire would make the
   SDK authoritative for behavior the contract does not promise. Recorded in
   KNOWN_WIRE_EXTRAS; it reads like a spec omission worth reporting upstream.
6. FinanceResearchResponse.warnings not modeled — prod sends warnings: []
   at the top level and the sibling research.json declares it (ResearchResponse
   models it), but finance-research.json declares only output. I added the
   field first; check_drift.py --strict then failed on exactly the standing
   mismatch that predicted, so it was reverted. The rule that falls out matches
   snippets: add the field when that endpoint's spec declares it, record it
   when only prod does. tests/test_research.py pins the current state so closing
   the gap later is a deliberate act rather than a silent one.

One former skip was closed instead of kept: the two pre-existing nested response
gaps (AnswerSearchResult.description / .thumbnail_url and
FinanceResearchSource.snippets) are fixed here rather than tracked via
KNOWN_RESPONSE_GAPS, which is now empty.

Two further surfaces were brought up to date in the same pass:

• tests/README.md — listed 11 of the 20 test files and carried wrong counts
  (Contents 12 vs 13, Answer 23 vs 25). Now lists all 21 with groups for extraction,
  knowledge, page_age, stream events and the cross-cutting suites, and counts taken
  from pytest --collect-only. The groups sum to 450, exactly the CI-gate collection.
• Pre-existing Optional type-cell drift across docs/models/ — a mechanical scan
  comparing every Type cell against its real annotation found 11 rows across 8 pages
  with the same Optional[X] ↔ X mismatch this PR fixed in results.md:
  contentsrequest, sourcecontrol, source, searchrequest, researchdetail,
  financeresearchdetail, researchrequest, researchtaskstreameventdata. All
  pre-dated this branch. Each cell fit its existing padded column, so only those 11
  lines changed and table alignment is preserved. The repo now audits clean.

Review notes / open questions

• The public docs currently show the Knowledge example's Python tab using raw
  requests rather than this SDK, and the search guide's optional-parameter
  table does not list knowledge. Both are documentation-site changes rather
  than SDK changes, so they are handled separately from this PR.

[factory-droid]

Droid finished @tyler5673's task —— View job

---

The PR is thorough and consistent across SDK plumbing, models, tests, and docs. The only issue I found is a small correctness gap in the new wire-audit script where list traversal can silently skip mismatches.

[factory-droid]

Droid finished @tyler5673's task —— View job

---

Strong, consistent PR across SDK plumbing, models, tests, docs, and drift tooling. The only actionable gap I found is in the new wire-audit script: the search audits currently walk only results, so new or dropped keys elsewhere in the response can be missed.

[factory-droid]

Droid finished @tyler5673's task —— View job

---

LGTM — I did not find any additional high-confidence, actionable issues beyond items already discussed in existing PR comments.