From c8dda6b74c30050171c2ae63197f8a3b71a17624 Mon Sep 17 00:00:00 2001 From: Tyler Eastman Date: Mon, 21 Sep 2026 12:30:53 -0700 Subject: [PATCH 1/7] feat: add knowledge param and knowledge result models to search (DX-835) Adds support for the new `knowledge` request parameter on POST /v1/search and the knowledge result models that come back with it. Request: `knowledge="core"` on `search()` / `search_async()` and the deprecated `you.search.unified*` shims. Normalized to lowercase like the other enum-typed params; an invalid value raises ValidationError locally before any network call, mirroring the server's 422. All params on these methods are keyword-only, so the addition is non-breaking. Response: `Results.knowledge` is an optional list of `KnowledgeResult`, each carrying `type`, `title`, and `attribution`, plus `description` and an optional `as_of` date for `type: answer` results. `type` is a plain `str` so an unrecognized future kind parses instead of raising. The API omits the key entirely when nothing relevant is found, so the field is None rather than an empty list. Also fixes a blind spot in scripts/check_drift.py: the response-schema check compared top-level fields only, so a new nested field such as `results.knowledge` was invisible. It now recurses through nested object schemas and arrays, and falls back to the previous flat comparison when a response resolves to a oneOf union rather than a single object schema. Recursion surfaced two pre-existing gaps unrelated to this change (AnswerSearchResult.description/.thumbnail_url and FinanceResearchSource.snippets); those are listed in an explicit KNOWN_RESPONSE_GAPS table that reports itself stale once the SDK catches up. Verified: 425 offline tests pass, 6 live knowledge tests pass against prod, mypy clean over 85 files, pylint 10.00/10 on the CI gate, and the drift check is clean in both --verbose and --strict modes. --- CHANGELOG.md | 39 +++ README.md | 29 ++ USAGE.md | 40 +++ docs/models/knowledge.md | 24 ++ docs/models/knowledgeattribution.md | 31 +++ docs/models/knowledgeresult.md | 67 +++++ docs/models/results.md | 9 +- docs/models/searchrequestbody.md | 1 + docs/sdks/search/README.md | 1 + docs/sdks/you/README.md | 1 + examples/api-example-calls.py | 35 +++ pyproject.toml | 2 +- scripts/check_drift.py | 142 +++++++++- src/youdotcom/_shims.py | 6 + src/youdotcom/_version.py | 2 +- src/youdotcom/models/__init__.py | 16 ++ src/youdotcom/models/knowledge.py | 10 + src/youdotcom/models/knowledgeattribution.py | 38 +++ src/youdotcom/models/knowledgeresult.py | 64 +++++ src/youdotcom/models/searchrequestbody.py | 7 + src/youdotcom/models/searchresponse.py | 8 +- src/youdotcom/sdk.py | 25 +- tests/test_knowledge.py | 266 +++++++++++++++++++ tests/test_live.py | 81 ++++++ tests/test_param_normalization.py | 9 +- tests/test_performance.py | 25 ++ uv.lock | 2 +- 27 files changed, 962 insertions(+), 18 deletions(-) create mode 100644 docs/models/knowledge.md create mode 100644 docs/models/knowledgeattribution.md create mode 100644 docs/models/knowledgeresult.md create mode 100644 src/youdotcom/models/knowledge.py create mode 100644 src/youdotcom/models/knowledgeattribution.py create mode 100644 src/youdotcom/models/knowledgeresult.py create mode 100644 tests/test_knowledge.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 2595420..08c43d3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,45 @@ All notable changes to the You.com Python SDK will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [3.5.0] - 2026-09-21 + +Minor release. Adds support for the new `knowledge` parameter on +`POST /v1/search` and the knowledge result models that come back with it. +Purely additive — no breaking changes. + +### Added + +- **`knowledge` parameter on `search()` and `search_async()`** — pass + `knowledge="core"` to request knowledge results backed by licensed data + providers such as encyclopedias, market-data firms, and reference publishers. + They arrive in their own section at `response.results.knowledge`. `"core"` is + the only value the API accepts, and anything else raises `ValidationError` + locally rather than reaching the network, mirroring the server's `422`. +- **`Knowledge` enum** — `Knowledge.CORE`, exported from `youdotcom.models`. + Plain strings are accepted and normalized, so `knowledge="core"` and + `knowledge="CORE"` both work. +- **`KnowledgeResult` and `KnowledgeAttribution` models** — a knowledge result + carries `type`, `title`, and `attribution`, plus `description` and an optional + `as_of` date for `type: answer` results, the only kind returned today. `type` + is modeled as a plain `str` so an unrecognized kind parses instead of raising, + since a new kind may populate a different set of fields. Attribution entries + are credits rather than citations and carry no URL. +- **`Results.knowledge`** — new optional field on the search response container. + Up to 25 results are returned, limited to those relevant to the query. When + none are relevant the API omits the key entirely, so the field is `None` + rather than an empty list and iterating needs an `or []` guard. `count` caps + the web and news sections, not knowledge. + +### Changed + +- **`scripts/check_drift.py` recurses nested response schemas** — the response + check previously compared top-level fields only, so drift inside a nested + object went undetected. Two pre-existing gaps that recursion surfaced + (`AnswerSearchResult.description` / `.thumbnail_url` and + `FinanceResearchSource.snippets`) are listed in an explicit + `KNOWN_RESPONSE_GAPS` table that reports itself stale once the SDK catches up, + rather than being silently ignored. + ## [3.4.0] - 2026-09-08 Minor release. The `metadata` format on the Contents API is now deprecated diff --git a/README.md b/README.md index 20e527f..a947acd 100644 --- a/README.md +++ b/README.md @@ -145,6 +145,35 @@ available and crawls the page live otherwise, `"cache"` returns cached content only (`contents` is omitted for results with none), and `"fetch"` always crawls the page live. +#### Knowledge results + +Pass `knowledge="core"` to add cards backed by licensed data providers — +encyclopedias, market-data firms, reference publishers. They come back in +their own section: + +```python +res = you.search( + query="what is the capital of France", + knowledge="core", +) + +for card in res.results.knowledge or []: + print(card.title) + print(card.description) + print([credit.name for credit in card.attribution]) +``` + +`"core"` is the only value the API accepts; anything else raises +`ValidationError` locally, mirroring the server's `422`. Results are limited +to those relevant to the query, up to 25, and when none are relevant the API +omits the section entirely — so `results.knowledge` is `None` rather than an +empty list. Iterate with `or []`. + +`count` caps the web and news sections, not knowledge. Attribution entries +are credits rather than citations: each names a provider and carries no URL. +`as_of`, when present, is the `YYYY-MM-DD` date the card's underlying data +covers. + ### Contents Clean HTML or Markdown for a list of URLs. diff --git a/USAGE.md b/USAGE.md index 012e571..2adde62 100644 --- a/USAGE.md +++ b/USAGE.md @@ -100,6 +100,46 @@ Unknown keys inside `extraction` raise `ValidationError` locally, and passing `ValueError` — both mirror the server's 422 contract so callers fail-fast. + +```python +# Add knowledge cards backed by licensed data providers. +import os +from youdotcom import You + + +with You( + api_key_auth=os.getenv("YDC_API_KEY"), + timeout_ms=60_000, +) as you: + + res = you.search( + query="what is the capital of France", + knowledge="core", + ) + + for card in res.results.knowledge or []: + print(card.title, card.description) + print([credit.name for credit in card.attribution]) +``` + +`knowledge="core"` requests knowledge results — cards backed by licensed data +providers such as encyclopedias, market-data firms, and reference publishers. +`"core"` is the only value the API accepts; anything else raises +`ValidationError` locally, mirroring the server's 422. + +Results are limited to those relevant to the query, up to 25. When none are +relevant the API omits the section, so `results.knowledge` is `None` rather +than an empty list — iterate with `or []`. `count` caps the web and news +sections, not knowledge. + +Each card carries `type`, `title`, and `attribution`; for `type: answer` — the +only kind returned today — `description` is present and `as_of` is an optional +`YYYY-MM-DD` date covering the card's underlying data. `type` is a plain +string so an unrecognized kind parses rather than raises; ignore a value you +do not recognize. Attribution entries are credits rather than citations: each +names a provider and carries no URL. + + ```python # Tag every outbound request with a caller-identity header so the diff --git a/docs/models/knowledge.md b/docs/models/knowledge.md new file mode 100644 index 0000000..4ac4e67 --- /dev/null +++ b/docs/models/knowledge.md @@ -0,0 +1,24 @@ +# Knowledge + +Requests knowledge results alongside web and news search. + +## Example Usage + +```python +from youdotcom.models import Knowledge + +value = Knowledge.CORE +``` + + +## Values + +| Name | Value | +| ------ | ----- | +| `CORE` | core | + +## Notes + +`core` is the only value the API accepts. Passing anything else raises +`ValidationError` locally rather than reaching the network, mirroring the +server's `422`. diff --git a/docs/models/knowledgeattribution.md b/docs/models/knowledgeattribution.md new file mode 100644 index 0000000..d7b79f5 --- /dev/null +++ b/docs/models/knowledgeattribution.md @@ -0,0 +1,31 @@ +# KnowledgeAttribution + +Display credit for the data behind a [KnowledgeResult](../models/knowledgeresult.md). + +## Example Usage + +```python +from youdotcom.models import KnowledgeAttribution + +credit = KnowledgeAttribution.model_validate({"name": "Encyclopedia Britannica"}) + +print(credit.name) # "Encyclopedia Britannica" +print(credit.source_description) # None when the provider reports none +``` + +Attribution entries arrive nested under `KnowledgeResult.attribution`. Given a +raw payload, `KnowledgeAttribution.model_validate(data)` builds one from a dict +matching `KnowledgeAttributionTypedDict`. + +## Fields + +| Field | Type | Required | Description | Example | +| -------------------- | --------------- | ------------------ | --------------------------------------- | ----------------------- | +| `name` | *str* | :heavy_check_mark: | Data provider for the knowledge result. | Encyclopedia Britannica | +| `source_description` | *Optional[str]* | :heavy_minus_sign: | Description of the provider. | General knowledge | + +## Notes + +These are credits rather than citations: each entry names a provider and +carries no URL. Read them off a result with +`[credit.name for credit in card.attribution]`. diff --git a/docs/models/knowledgeresult.md b/docs/models/knowledgeresult.md new file mode 100644 index 0000000..0d0dabe --- /dev/null +++ b/docs/models/knowledgeresult.md @@ -0,0 +1,67 @@ +# KnowledgeResult + +A single knowledge result. `type` identifies the kind of result and determines which fields it populates. `type`, `title`, and `attribution` are required on every kind. + +For `type: answer`, the only kind currently returned, `description` is required and `as_of` is optional. + +## Example Usage + +```python +import os +from youdotcom import You + +with You(api_key_auth=os.getenv("YDC_API_KEY"), timeout_ms=60_000) as you: + res = you.search(query="what is the capital of France", knowledge="core") + + for card in res.results.knowledge or []: + print(card.type, card.title) + print(card.description) + print([credit.name for credit in card.attribution]) +``` + +Knowledge results arrive in the response rather than being constructed by the +caller. Given a raw payload, `KnowledgeResult.model_validate(data)` builds one +from a dict matching `KnowledgeResultTypedDict`. + +## Fields + +| Field | Type | Required | Description | Example | +| ------------- | ---------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------- | +| `type` | *str* | :heavy_check_mark: | The kind of knowledge result retrieved. `answer` is the only value currently returned. Ignore a value you do not recognize rather than failing on it, since a new kind may populate a different set of fields. | answer | +| `title` | *str* | :heavy_check_mark: | The title of the knowledge result. | Paris is the capital of France | +| `attribution` | List[[models.KnowledgeAttribution](../models/knowledgeattribution.md)] | :heavy_check_mark: | Display credit for the data behind the result. These are credits rather than citations: each entry names a provider and carries no URL. | | +| `description` | *Optional[str]* | :heavy_minus_sign: | Description of the knowledge result, drawn from proprietary licensed data. Required on `type: answer` results. | Paris has been the capital of France… | +| `as_of` | *Optional[str]* | :heavy_minus_sign: | The date the result's underlying data covers, as `YYYY-MM-DD`. Optional, and omitted when the provider reports no date. | 2026-04-26 | + +## Notes + +### The `results.knowledge` key is omitted, not empty + +Knowledge results are limited to those relevant to the query. When none are +relevant the API omits `results.knowledge` entirely rather than returning an +empty array, so `response.results.knowledge` is `None` — check for `None` +before iterating. + +```python +for card in res.results.knowledge or []: + print(card.title) +``` + +### `count` does not cap knowledge + +`count` sets the maximum for the web and news sections. Knowledge has its own +limit of up to 25 results, so `count=1` can still return several knowledge +results. + +### `type` is a plain string + +`type` is modeled as `str` rather than an enum so an unrecognized value parses +instead of raising. A new kind of knowledge result may populate a different +set of fields, so ignore a `type` you do not recognize rather than failing on +it. + +### `as_of` is a string, not a date + +`as_of` stays a `str` in `YYYY-MM-DD` form. Parse it with +`datetime.strptime(card.as_of, "%Y-%m-%d")` when you need a date object, and +expect `None` when the provider reports no date. diff --git a/docs/models/results.md b/docs/models/results.md index 342f0ea..f1f4007 100644 --- a/docs/models/results.md +++ b/docs/models/results.md @@ -3,7 +3,8 @@ ## Fields -| Field | Type | Required | Description | -| -------------------------------------------------- | -------------------------------------------------- | -------------------------------------------------- | -------------------------------------------------- | -| `web` | List[[models.WebResult](../models/webresult.md)] | :heavy_minus_sign: | N/A | -| `news` | List[[models.NewsResult](../models/newsresult.md)] | :heavy_minus_sign: | N/A | \ No newline at end of file +| Field | Type | Required | Description | +| -------------------------------------------------- | ---------------------------------------------------------------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `web` | List[[models.WebResult](../models/webresult.md)] | :heavy_minus_sign: | N/A | +| `news` | List[[models.NewsResult](../models/newsresult.md)] | :heavy_minus_sign: | N/A | +| `knowledge` | Optional[List[[models.KnowledgeResult](../models/knowledgeresult.md)]] | :heavy_minus_sign: | Results backed by licensed data providers. Up to 25 are returned, limited to those relevant to the query. When none are relevant the key is omitted rather than returned as an empty array. | \ No newline at end of file diff --git a/docs/models/searchrequestbody.md b/docs/models/searchrequestbody.md index 91fca65..54089c3 100644 --- a/docs/models/searchrequestbody.md +++ b/docs/models/searchrequestbody.md @@ -12,6 +12,7 @@ | `country` | [Optional[models.Country]](../models/country.md) | :heavy_minus_sign: | The country code that determines the geographical focus of the web results. | | | `language` | [Optional[models.Language]](../models/language.md) | :heavy_minus_sign: | The language of the web results that will be returned (BCP 47 format). | | | `safesearch` | [Optional[models.SafeSearch]](../models/safesearch.md) | :heavy_minus_sign: | Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not. | | +| `knowledge` | [Optional[models.Knowledge]](../models/knowledge.md) | :heavy_minus_sign: | Requests knowledge results alongside web and news search. `"core"` is the only value the API accepts; anything else raises `ValidationError` locally, mirroring the server's `422`. Returns up to 25 results, limited to those relevant to the query — `count` caps the web and news sections, not knowledge. | core | | `livecrawl` | [Optional[models.LiveCrawl]](../models/livecrawl.md) | :heavy_minus_sign: | **Deprecated; use `extraction` instead.** Indicates which section(s) of search results to livecrawl and return full page content. | | | `livecrawl_formats` | Optional[List[[models.LiveCrawlFormats](../models/livecrawlformats.md)]] | :heavy_minus_sign: | **Deprecated; use `extraction.full_page.extraction_formats` instead.** Indicates the format(s) of the livecrawled content. Pass one or both values (`html`, `markdown`). In a GET request, repeat the parameter: `?livecrawl_formats=html&livecrawl_formats=markdown`. In a POST body, provide a JSON array: `["html", "markdown"]`. | | | `extraction` | [Optional[models.Extraction]](../models/extraction.md) | :heavy_minus_sign: | Controls how page content is attached to each result. Preferred over `livecrawl`/`livecrawl_formats`. The two are mutually exclusive; `you.search` raises `ValueError` if both are passed. Top-level `crawl_timeout` is invalid alongside `extraction_mode="highlights"` and is stripped from the body. | | diff --git a/docs/sdks/search/README.md b/docs/sdks/search/README.md index 28d669d..cdf4b6f 100644 --- a/docs/sdks/search/README.md +++ b/docs/sdks/search/README.md @@ -48,6 +48,7 @@ with You( | `country` | [Optional[models.Country]](../../models/country.md) | :heavy_minus_sign: | The country code that determines the geographical focus of the web results. | | | `language` | [Optional[models.Language]](../../models/language.md) | :heavy_minus_sign: | The language of the web results that will be returned (BCP 47 format). | | | `safesearch` | [Optional[models.SafeSearch]](../../models/safesearch.md) | :heavy_minus_sign: | Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not. | | +| `knowledge` | [Optional[models.Knowledge]](../../models/knowledge.md) | :heavy_minus_sign: | Requests knowledge results alongside web and news search. `"core"` is the only value the API accepts; anything else raises `ValidationError` locally, mirroring the server's `422`. Returns up to 25 results, limited to those relevant to the query — `count` caps the web and news sections, not knowledge. | core | | `livecrawl` | [Optional[models.LiveCrawl]](../../models/livecrawl.md) | :heavy_minus_sign: | **Deprecated; use `extraction` instead.** Indicates which section(s) of search results to livecrawl and return full page content. | | | `livecrawl_formats` | Optional[List[[models.LiveCrawlFormats](../../models/livecrawlformats.md)]] | :heavy_minus_sign: | **Deprecated; use `extraction.full_page.extraction_formats` instead.** Indicates the format(s) of the livecrawled content. Pass one or both values (`html`, `markdown`). In a GET request, repeat the parameter: `?livecrawl_formats=html&livecrawl_formats=markdown`. In a POST body, provide a JSON array: `["html", "markdown"]`. | | | `extraction` | [Optional[models.Extraction]](../../models/extraction.md) | :heavy_minus_sign: | Controls how page content is attached to each result. Preferred over `livecrawl`/`livecrawl_formats`. The two are mutually exclusive; `you.search` raises `ValueError` if both are passed. Top-level `crawl_timeout` is invalid alongside `extraction_mode="highlights"` and is stripped from the body. | | diff --git a/docs/sdks/you/README.md b/docs/sdks/you/README.md index adbba12..3f67aa9 100644 --- a/docs/sdks/you/README.md +++ b/docs/sdks/you/README.md @@ -263,6 +263,7 @@ with You( | `country` | [Optional[models.Country]](../../models/country.md) | :heavy_minus_sign: | The country code that determines the geographical focus of the web results. | | | `language` | [Optional[models.Language]](../../models/language.md) | :heavy_minus_sign: | The language of the web results that will be returned (BCP 47 format). | | | `safesearch` | [Optional[models.SafeSearch]](../../models/safesearch.md) | :heavy_minus_sign: | Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not. | | +| `knowledge` | [Optional[models.Knowledge]](../../models/knowledge.md) | :heavy_minus_sign: | Requests knowledge results alongside web and news search. `"core"` is the only value the API accepts; anything else raises `ValidationError` locally, mirroring the server's `422`. Returns up to 25 results, limited to those relevant to the query — `count` caps the web and news sections, not knowledge. | core | | `livecrawl` | [Optional[models.LiveCrawl]](../../models/livecrawl.md) | :heavy_minus_sign: | **Deprecated; use `extraction` instead.** Indicates which section(s) of search results to livecrawl and return full page content. | | | `livecrawl_formats` | Optional[List[[models.LiveCrawlFormats](../../models/livecrawlformats.md)]] | :heavy_minus_sign: | **Deprecated; use `extraction.full_page.extraction_formats` instead.** Indicates the format(s) of the livecrawled content. Pass one or both values (`html`, `markdown`). In a GET request, repeat the parameter: `?livecrawl_formats=html&livecrawl_formats=markdown`. In a POST body, provide a JSON array: `["html", "markdown"]`. | | | `extraction` | [Optional[models.Extraction]](../../models/extraction.md) | :heavy_minus_sign: | Controls how page content is attached to each result. Preferred over `livecrawl`/`livecrawl_formats`. The two are mutually exclusive; `you.search` raises `ValueError` if both are passed. Top-level `crawl_timeout` is invalid alongside `extraction_mode="highlights"` and is stripped from the body. | | diff --git a/examples/api-example-calls.py b/examples/api-example-calls.py index 4aaad63..63dbf83 100755 --- a/examples/api-example-calls.py +++ b/examples/api-example-calls.py @@ -392,6 +392,40 @@ def search_request_with_boost(): print(f" - {result.title or 'Untitled'}: {result.url}") +def search_request_with_knowledge(): + """ + Search API: use `knowledge="core"` to add knowledge cards backed by + licensed data providers (encyclopedias, market-data firms, reference + publishers). They arrive in their own `results.knowledge` section. + + The section is omitted entirely when nothing relevant is found, so + `results.knowledge` is `None` rather than an empty list. `count` caps the + web and news sections, not knowledge. + """ + print("\n🚀 Running Search Request (knowledge)...\n") + + assert you is not None, "SDK client not initialized" + + results = you.search( + query="what is the capital of France", + knowledge="core", + ) + + print("Knowledge results:") + if results.results and results.results.knowledge: + for card in results.results.knowledge: + print(f" - [{card.type}] {card.title}") + if card.description: + preview = card.description[:120].replace("\n", " ") + print(f" {preview}...") + if card.as_of: + print(f" Data as of: {card.as_of}") + credits = ", ".join(credit.name for credit in card.attribution) + print(f" Credit: {credits}") + else: + print("No knowledge results found") + + def content_request_with_max_age(): """ Contents API: use `max_age` to control cache freshness (in seconds). @@ -420,6 +454,7 @@ def content_request_with_max_age(): {"name": "Search Request (extraction)", "fn": search_request}, {"name": "Search Request (deprecated livecrawl)", "fn": search_request_livecrawl_legacy}, {"name": "Search Request (boost_domains)", "fn": search_request_with_boost}, + {"name": "Search Request (knowledge)", "fn": search_request_with_knowledge}, {"name": "Content Request", "fn": content_request}, {"name": "Content Request (max_age)", "fn": content_request_with_max_age}, {"name": "Research Request", "fn": research_request}, diff --git a/pyproject.toml b/pyproject.toml index 236ecd1..c3e29e8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "youdotcom" -version = "3.4.0" +version = "3.5.0" description = "The official You.com Python SDK." authors = [{ name = "You.com" },] readme = "README.md" diff --git a/scripts/check_drift.py b/scripts/check_drift.py index d15baa2..1c40d8f 100644 --- a/scripts/check_drift.py +++ b/scripts/check_drift.py @@ -46,6 +46,16 @@ ("GET", "/v1/search"), } +# Nested response fields the SDK models don't define yet. These predate the +# response check learning to recurse (it previously compared top-level fields +# only, so drift inside a nested object was invisible). Keyed by +# (spec name, dotted field path from the response root). An entry that goes +# stale — the SDK catches up — is reported rather than silently ignored. +KNOWN_RESPONSE_GAPS = { + ("answer", "results.web"): {"description", "thumbnail_url"}, + ("finance-research", "output.sources"): {"snippets"}, +} + # Map (method, path) -> SDK method name. # {task_id} and {task_id}/stream are handled by helpers, not direct methods. EXPECTED_ENDPOINTS = { @@ -77,6 +87,7 @@ ("web-search", "Country", "youdotcom.models", "Country"), ("answer", "Country", "youdotcom.models", "Country"), ("research", "Country", "youdotcom.models", "Country"), + ("web-search", "Knowledge", "youdotcom.models", "Knowledge"), ] # SDK-internal parameters that aren't API params (excluded from drift comparison). @@ -387,8 +398,116 @@ def check_request_params(specs: dict[str, dict[str, Any]]) -> list[str]: return warnings +def _resolve_schema(schema: dict[str, Any], spec: dict[str, Any]) -> dict[str, Any]: + """Follow ``$ref`` and array ``items`` down to the underlying object schema.""" + seen: set[str] = set() + while True: + if "$ref" in schema: + ref = schema["$ref"] + if ref in seen: + return {} + seen.add(ref) + schema = _resolve_ref(ref, spec) + continue + if schema.get("type") == "array" and isinstance(schema.get("items"), dict): + schema = schema["items"] + continue + return schema + + +def _nested_model(annotation: Any) -> Any: + """Return the pydantic model a field annotation points at, or ``None``. + + Unwraps ``Optional[...]`` and ``List[...]`` so nested response objects + more than one level down are still compared. + """ + args = getattr(annotation, "__args__", None) + if args is not None: + for arg in args: + found = _nested_model(arg) + if found is not None: + return found + return None + return annotation if hasattr(annotation, "model_fields") else None + + +def _compare_response_fields( + schema: dict[str, Any], + model: Any, + path: str, + spec: dict[str, Any], + warnings: list[str], + visited: set[Any], + spec_name: str, + field_path: str, +) -> None: + """Compare a spec object schema against a model, recursing into nested objects. + + A top-level-only comparison misses drift inside nested response objects: + a new ``results.knowledge`` array is invisible when ``results`` itself is + unchanged. + """ + if model in visited: + return + visited.add(model) + + schema = _resolve_schema(schema, spec) + props = schema.get("properties") + if not props: + # Nothing to compare against — a `oneOf` union or an unresolvable ref. + # Bail rather than reporting every SDK field as absent from the spec. + return + model_fields = set(model.model_fields.keys()) + + missing_in_sdk = set(props) - model_fields + missing_in_spec = model_fields - set(props) + + known = KNOWN_RESPONSE_GAPS.get((spec_name, field_path), set()) + stale = known - missing_in_sdk + if stale: + warnings.append( + f"[response] {path}: KNOWN_RESPONSE_GAPS entry {stale} is stale — " + f"the SDK model now defines it, so remove the entry" + ) + missing_in_sdk -= known + + if missing_in_sdk: + warnings.append( + f"[response] {path}: spec has fields {missing_in_sdk} " + f"which SDK model doesn't have" + ) + if missing_in_spec: + warnings.append( + f"[response] {path}: SDK model has fields {missing_in_spec} " + f"which spec doesn't define" + ) + + for prop_name, prop_schema in props.items(): + if prop_name not in model_fields: + continue + child_model = _nested_model(model.model_fields[prop_name].annotation) + if child_model is None: + continue + if "properties" in _resolve_schema(prop_schema, spec): + _compare_response_fields( + prop_schema, + child_model, + f"{path}.{prop_name}", + spec, + warnings, + visited, + spec_name, + f"{field_path}.{prop_name}" if field_path else prop_name, + ) + + def check_response_schemas(specs: dict[str, dict[str, Any]]) -> list[str]: - """Check that spec 200 response schema fields match SDK model fields.""" + """Check that spec 200 response schema fields match SDK model fields. + + Endpoints with a single response model are compared recursively, so fields + nested inside response objects are checked too. Endpoints whose response + can be one of several models fall back to a flat union comparison. + """ warnings: list[str] = [] for check in SCHEMA_CHECKS: @@ -408,6 +527,27 @@ def check_response_schemas(specs: dict[str, dict[str, Any]]) -> list[str]: if not schema: continue + model_names = check["sdk_response_models"] + # Recurse only when the response resolves to a single object schema. + # A `oneOf` union resolves to no properties, so it keeps the flat + # comparison below rather than being mistaken for an empty schema. + if len(model_names) == 1 and _resolve_schema(schema, spec).get("properties"): + import importlib + + mod = importlib.import_module("youdotcom.models") + model = getattr(mod, model_names[0]) + _compare_response_fields( + schema, + model, + f"{spec_name} {method} {path} {model_names[0]}", + spec, + warnings, + set(), + spec_name, + "", + ) + continue + spec_fields = _get_schema_properties(schema, spec) sdk_fields = _get_sdk_model_fields(check["sdk_response_models"]) diff --git a/src/youdotcom/_shims.py b/src/youdotcom/_shims.py index 3d8fd8d..b9444fa 100644 --- a/src/youdotcom/_shims.py +++ b/src/youdotcom/_shims.py @@ -58,6 +58,7 @@ def __call__( country: Optional[str] = None, language: OptionalNullable[str] = UNSET, safesearch: Optional[str] = None, + knowledge: Optional[str] = None, livecrawl: Optional[str] = None, livecrawl_formats: Optional[Iterable[str]] = None, extraction: Optional[Union[models.Extraction, Mapping[str, Any]]] = None, @@ -78,6 +79,7 @@ def __call__( country=country, language=language, safesearch=safesearch, + knowledge=knowledge, livecrawl=livecrawl, livecrawl_formats=livecrawl_formats, extraction=extraction, @@ -101,6 +103,7 @@ def unified( country: Optional[str] = None, language: OptionalNullable[str] = UNSET, safesearch: Optional[str] = None, + knowledge: Optional[str] = None, livecrawl: Optional[str] = None, livecrawl_formats: Optional[Iterable[str]] = None, extraction: Optional[Union[models.Extraction, Mapping[str, Any]]] = None, @@ -123,6 +126,7 @@ def unified( country=country, language=language, safesearch=safesearch, + knowledge=knowledge, livecrawl=livecrawl, livecrawl_formats=livecrawl_formats, extraction=extraction, @@ -146,6 +150,7 @@ async def unified_async( country: Optional[str] = None, language: OptionalNullable[str] = UNSET, safesearch: Optional[str] = None, + knowledge: Optional[str] = None, livecrawl: Optional[str] = None, livecrawl_formats: Optional[Iterable[str]] = None, extraction: Optional[Union[models.Extraction, Mapping[str, Any]]] = None, @@ -168,6 +173,7 @@ async def unified_async( country=country, language=language, safesearch=safesearch, + knowledge=knowledge, livecrawl=livecrawl, livecrawl_formats=livecrawl_formats, extraction=extraction, diff --git a/src/youdotcom/_version.py b/src/youdotcom/_version.py index e946998..b2a33e2 100644 --- a/src/youdotcom/_version.py +++ b/src/youdotcom/_version.py @@ -2,7 +2,7 @@ import importlib.metadata __title__: str = "youdotcom" -__version__: str = "3.4.0" +__version__: str = "3.5.0" __openapi_doc_version__: str = "1.0.0" try: diff --git a/src/youdotcom/models/__init__.py b/src/youdotcom/models/__init__.py index 8220763..aa7b6fe 100644 --- a/src/youdotcom/models/__init__.py +++ b/src/youdotcom/models/__init__.py @@ -57,6 +57,12 @@ GetResearchTaskRequest, GetResearchTaskRequestTypedDict, ) + from .knowledge import Knowledge + from .knowledgeattribution import ( + KnowledgeAttribution, + KnowledgeAttributionTypedDict, + ) + from .knowledgeresult import KnowledgeResult, KnowledgeResultTypedDict from .language import Language from .livecrawl import LiveCrawl from .livecrawlformats import LiveCrawlFormats @@ -178,6 +184,11 @@ "FreshnessValueTypedDict", "GetResearchTaskRequest", "GetResearchTaskRequestTypedDict", + "Knowledge", + "KnowledgeAttribution", + "KnowledgeAttributionTypedDict", + "KnowledgeResult", + "KnowledgeResultTypedDict", "Language", "LiveCrawl", "LiveCrawlFormats", @@ -278,6 +289,11 @@ "FreshnessValueTypedDict": ".freshnessvalue", "GetResearchTaskRequest": ".getresearchtaskop", "GetResearchTaskRequestTypedDict": ".getresearchtaskop", + "Knowledge": ".knowledge", + "KnowledgeAttribution": ".knowledgeattribution", + "KnowledgeAttributionTypedDict": ".knowledgeattribution", + "KnowledgeResult": ".knowledgeresult", + "KnowledgeResultTypedDict": ".knowledgeresult", "Language": ".language", "LiveCrawl": ".livecrawl", "LiveCrawlFormats": ".livecrawlformats", diff --git a/src/youdotcom/models/knowledge.py b/src/youdotcom/models/knowledge.py new file mode 100644 index 0000000..9a76e53 --- /dev/null +++ b/src/youdotcom/models/knowledge.py @@ -0,0 +1,10 @@ + + +from __future__ import annotations +from enum import Enum + + +class Knowledge(str, Enum): + r"""Requests knowledge results alongside web and news search.""" + + CORE = "core" diff --git a/src/youdotcom/models/knowledgeattribution.py b/src/youdotcom/models/knowledgeattribution.py new file mode 100644 index 0000000..ec2ef20 --- /dev/null +++ b/src/youdotcom/models/knowledgeattribution.py @@ -0,0 +1,38 @@ + + +from __future__ import annotations +from pydantic import model_serializer +from typing import Optional +from typing_extensions import NotRequired, TypedDict +from youdotcom.types import BaseModel, UNSET_SENTINEL + + +class KnowledgeAttributionTypedDict(TypedDict): + name: str + r"""Data provider for the knowledge result.""" + source_description: NotRequired[str] + r"""Description of the provider.""" + + +class KnowledgeAttribution(BaseModel): + name: str + r"""Data provider for the knowledge result.""" + + source_description: Optional[str] = None + r"""Description of the provider.""" + + @model_serializer(mode="wrap") + def serialize_model(self, handler): + optional_fields = set(["source_description"]) + serialized = handler(self) + m = {} + + for n, f in type(self).model_fields.items(): + k = f.alias or n + val = serialized.get(k, serialized.get(n)) + + if val != UNSET_SENTINEL: + if val is not None or k not in optional_fields: + m[k] = val + + return m diff --git a/src/youdotcom/models/knowledgeresult.py b/src/youdotcom/models/knowledgeresult.py new file mode 100644 index 0000000..dd5f198 --- /dev/null +++ b/src/youdotcom/models/knowledgeresult.py @@ -0,0 +1,64 @@ + + +from __future__ import annotations +from .knowledgeattribution import KnowledgeAttribution, KnowledgeAttributionTypedDict +from pydantic import model_serializer +from typing import List, Optional +from typing_extensions import NotRequired, TypedDict +from youdotcom.types import BaseModel, UNSET_SENTINEL + + +class KnowledgeResultTypedDict(TypedDict): + r"""A single knowledge result. `type` identifies the kind of result and determines which fields it populates. `type`, `title`, and `attribution` are required on every kind. + + For `type: answer`, the only kind currently returned, `description` is required and `as_of` is optional. + """ + + type: str + r"""The kind of knowledge result retrieved. `answer` is the only value currently returned. Ignore a value you do not recognize rather than failing on it, since a new kind may populate a different set of fields.""" + title: str + r"""The title of the knowledge result.""" + attribution: List[KnowledgeAttributionTypedDict] + r"""Display credit for the data behind the result. These are credits rather than citations: each entry names a provider and carries no URL.""" + description: NotRequired[str] + r"""Description of the knowledge result, drawn from proprietary licensed data. Required on `type: answer` results.""" + as_of: NotRequired[str] + r"""The date the result's underlying data covers, as `YYYY-MM-DD`. Optional, and omitted when the provider reports no date.""" + + +class KnowledgeResult(BaseModel): + r"""A single knowledge result. `type` identifies the kind of result and determines which fields it populates. `type`, `title`, and `attribution` are required on every kind. + + For `type: answer`, the only kind currently returned, `description` is required and `as_of` is optional. + """ + + type: str + r"""The kind of knowledge result retrieved. `answer` is the only value currently returned. Ignore a value you do not recognize rather than failing on it, since a new kind may populate a different set of fields.""" + + title: str + r"""The title of the knowledge result.""" + + attribution: List[KnowledgeAttribution] + r"""Display credit for the data behind the result. These are credits rather than citations: each entry names a provider and carries no URL.""" + + description: Optional[str] = None + r"""Description of the knowledge result, drawn from proprietary licensed data. Required on `type: answer` results.""" + + as_of: Optional[str] = None + r"""The date the result's underlying data covers, as `YYYY-MM-DD`. Optional, and omitted when the provider reports no date.""" + + @model_serializer(mode="wrap") + def serialize_model(self, handler): + optional_fields = set(["description", "as_of"]) + serialized = handler(self) + m = {} + + for n, f in type(self).model_fields.items(): + k = f.alias or n + val = serialized.get(k, serialized.get(n)) + + if val != UNSET_SENTINEL: + if val is not None or k not in optional_fields: + m[k] = val + + return m diff --git a/src/youdotcom/models/searchrequestbody.py b/src/youdotcom/models/searchrequestbody.py index 46a21fe..e880925 100644 --- a/src/youdotcom/models/searchrequestbody.py +++ b/src/youdotcom/models/searchrequestbody.py @@ -4,6 +4,7 @@ from .country import Country from .extraction import Extraction, ExtractionTypedDict from .freshnessvalue import FreshnessValue, FreshnessValueTypedDict +from .knowledge import Knowledge from .language import Language from .livecrawl import LiveCrawl from .livecrawlformats import LiveCrawlFormats @@ -32,6 +33,8 @@ class SearchRequestBodyTypedDict(TypedDict): r"""The language of the web results that will be returned (BCP 47 format).""" safesearch: NotRequired[SafeSearch] r"""Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not.""" + knowledge: NotRequired[Knowledge] + r"""Requests knowledge results alongside web and news search.""" livecrawl: NotRequired[LiveCrawl] r"""Deprecated; use `extraction` instead. Indicates which section(s) of search results to livecrawl and return full page content.""" livecrawl_formats: NotRequired[List[LiveCrawlFormats]] @@ -87,6 +90,9 @@ class SearchRequestBody(BaseModel): safesearch: Optional[SafeSearch] = None r"""Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not.""" + knowledge: Optional[Knowledge] = None + r"""Requests knowledge results alongside web and news search.""" + livecrawl: Optional[LiveCrawl] = None r"""Deprecated; use `extraction` instead. Indicates which section(s) of search results to livecrawl and return full page content.""" @@ -132,6 +138,7 @@ def serialize_model(self, handler): "country", "language", "safesearch", + "knowledge", "livecrawl", "livecrawl_formats", "include_domains", diff --git a/src/youdotcom/models/searchresponse.py b/src/youdotcom/models/searchresponse.py index e17262f..72f120c 100644 --- a/src/youdotcom/models/searchresponse.py +++ b/src/youdotcom/models/searchresponse.py @@ -1,6 +1,7 @@ from __future__ import annotations +from .knowledgeresult import KnowledgeResult, KnowledgeResultTypedDict from .newsresult import NewsResult, NewsResultTypedDict from .searchmetadata import SearchMetadata, SearchMetadataTypedDict from .webresult import WebResult, WebResultTypedDict @@ -13,6 +14,8 @@ class ResultsTypedDict(TypedDict): web: NotRequired[List[WebResultTypedDict]] news: NotRequired[List[NewsResultTypedDict]] + knowledge: NotRequired[List[KnowledgeResultTypedDict]] + r"""Results backed by licensed data providers. Up to 25 are returned, limited to those relevant to the query. When none are relevant the key is omitted rather than returned as an empty array.""" class Results(BaseModel): @@ -20,9 +23,12 @@ class Results(BaseModel): news: Optional[List[NewsResult]] = None + knowledge: Optional[List[KnowledgeResult]] = None + r"""Results backed by licensed data providers. Up to 25 are returned, limited to those relevant to the query. When none are relevant the key is omitted rather than returned as an empty array.""" + @model_serializer(mode="wrap") def serialize_model(self, handler): - optional_fields = set(["web", "news"]) + optional_fields = set(["web", "news", "knowledge"]) serialized = handler(self) m = {} diff --git a/src/youdotcom/sdk.py b/src/youdotcom/sdk.py index 53ef1b9..5743e02 100644 --- a/src/youdotcom/sdk.py +++ b/src/youdotcom/sdk.py @@ -42,10 +42,10 @@ def _upper(value: Any) -> Any: def _lower(value: Any) -> Any: """Normalize a plain-string enum value to its lowercase spelling. - Used for ``safesearch``, ``livecrawl``, and ``freshness``, whose enum - members are lowercase (``"STRICT"`` -> ``"strict"``). Date-range freshness - values are unaffected apart from the ``to`` separator, which the API - expects in lowercase anyway. + Used for ``safesearch``, ``knowledge``, ``livecrawl``, and ``freshness``, + whose enum members are lowercase (``"STRICT"`` -> ``"strict"``). Date-range + freshness values are unaffected apart from the ``to`` separator, which the + API expects in lowercase anyway. """ return value.lower() if isinstance(value, str) else value @@ -66,6 +66,7 @@ def _build_search_request( country: Optional[str], language: OptionalNullable[str], safesearch: Optional[str], + knowledge: Optional[str], livecrawl: Optional[str], livecrawl_formats: Optional[Iterable[str]], extraction: Optional[Union[models.Extraction, Mapping[str, Any]]], @@ -147,6 +148,7 @@ def _build_search_request( offset=offset, country=_upper(country), safesearch=_lower(safesearch), + knowledge=_lower(knowledge), livecrawl=_lower(livecrawl), livecrawl_formats=utils.unmarshal( _lower_each(livecrawl_formats), Optional[List[models.LiveCrawlFormats]] @@ -995,6 +997,7 @@ def _search_impl( country: Optional[str] = None, language: OptionalNullable[str] = UNSET, safesearch: Optional[str] = None, + knowledge: Optional[str] = None, livecrawl: Optional[str] = None, livecrawl_formats: Optional[Iterable[str]] = None, extraction: Optional[Union[models.Extraction, Mapping[str, Any]]] = None, @@ -1010,9 +1013,9 @@ def _search_impl( r"""Search via POST /v1/search. Enum-typed parameters (``country``, ``language``, ``safesearch``, - ``livecrawl``, ``livecrawl_formats``, ``freshness``) accept plain - strings in any case -- the SDK normalizes them to the casing the API - expects, so callers don't need to import enum classes. + ``knowledge``, ``livecrawl``, ``livecrawl_formats``, ``freshness``) + accept plain strings in any case -- the SDK normalizes them to the + casing the API expects, so callers don't need to import enum classes. ``livecrawl`` and ``livecrawl_formats`` are deprecated; prefer ``extraction``. The two are mutually exclusive -- passing both @@ -1030,6 +1033,11 @@ def _search_impl( :param language: BCP 47 language code. Omit the argument to use the API default (``"en"``); pass ``None`` to send no language at all. :param safesearch: ``"strict"``, ``"moderate"``, or ``"off"``. + :param knowledge: ``"core"`` -- requests knowledge results alongside + web and news search. Omit to skip them. It is the only value the + API accepts; anything else raises + :class:`pydantic.ValidationError` locally, mirroring the server's + ``422``. :param livecrawl: deprecated. ``"web"``, ``"news"``, or ``"all"``. Use ``extraction`` instead. Mutually exclusive with ``extraction``. :param livecrawl_formats: deprecated. ``["html"]``, ``["markdown"]``, @@ -1075,6 +1083,7 @@ def _search_impl( country=country, language=language, safesearch=safesearch, + knowledge=knowledge, livecrawl=livecrawl, livecrawl_formats=livecrawl_formats, extraction=extraction, @@ -1171,6 +1180,7 @@ async def search_async( country: Optional[str] = None, language: OptionalNullable[str] = UNSET, safesearch: Optional[str] = None, + knowledge: Optional[str] = None, livecrawl: Optional[str] = None, livecrawl_formats: Optional[Iterable[str]] = None, extraction: Optional[Union[models.Extraction, Mapping[str, Any]]] = None, @@ -1207,6 +1217,7 @@ async def search_async( country=country, language=language, safesearch=safesearch, + knowledge=knowledge, livecrawl=livecrawl, livecrawl_formats=livecrawl_formats, extraction=extraction, diff --git a/tests/test_knowledge.py b/tests/test_knowledge.py new file mode 100644 index 0000000..d5645ea --- /dev/null +++ b/tests/test_knowledge.py @@ -0,0 +1,266 @@ +"""Tests for the ``knowledge`` parameter and knowledge results on ``you.search``. + +Locks the contract for the Knowledge launch (``POST /v1/search``): + +- ``knowledge`` is a new enum-typed request parameter. ``"core"`` is the only + value the API accepts; anything else is rejected server-side with ``422``. +- The parameter is normalized to lowercase like the other enum-typed search + parameters, and is omitted from the request body when not supplied. +- ``results.knowledge`` is a list of ``KnowledgeResult``. ``type``, ``title``, + and ``attribution`` are always present; ``description`` and ``as_of`` are + optional. ``source_description`` inside an attribution entry is optional. +- The ``results.knowledge`` key is omitted entirely when no knowledge results + are relevant, so the parsed attribute is ``None`` rather than ``[]``. +- ``KnowledgeResult.type`` is modeled as a plain string: an unrecognized value + parses instead of raising, since a new kind may arrive later. +""" + +import json +from contextlib import contextmanager + +import httpx +import pytest +from pydantic import ValidationError + +from youdotcom import You, models +from youdotcom.models import Knowledge, KnowledgeAttribution, KnowledgeResult + + +@contextmanager +def _capture(response_body: dict): + """Yield ``(You, captured)`` over a mock transport returning ``response_body``.""" + captured: dict = {} + + def handler(request): + captured["body"] = json.loads(request.content) + return httpx.Response( + 200, + headers={"content-type": "application/json"}, + content=json.dumps(response_body), + ) + + client = httpx.Client(transport=httpx.MockTransport(handler)) + try: + with You(api_key_auth="k", server_url="http://mock.local", client=client) as you: + yield you, captured + finally: + client.close() + + +_EMPTY = {"results": {"web": []}, "metadata": {"query": "q"}} + + +def _search_body(**kwargs) -> dict: + """Run one synchronous search; return the JSON body that went over the wire.""" + with _capture(_EMPTY) as (you, captured): + you.search(query="q", **kwargs) + return captured["body"] + + +def _parse(results: dict) -> models.SearchResponse: + """Parse a ``results`` mapping through the response model.""" + return models.SearchResponse.model_validate( + {"results": results, "metadata": {"query": "q"}} + ) + + +# A knowledge result shaped like what prod returns today. +_ANSWER = { + "type": "answer", + "title": "Paris is the capital of France", + "description": "Paris has been the capital of France since the 10th century.", + "as_of": "2026-09-01", + "attribution": [ + { + "name": "Encyclopedia Britannica", + "source_description": "General knowledge", + } + ], +} + + +# --------------------------------------------------------------------------- +# Enum contract +# --------------------------------------------------------------------------- + + +class TestKnowledgeEnum: + def test_core_member(self): + assert Knowledge.CORE.value == "core" + + def test_core_is_the_only_member(self): + """``knowledge=detailed`` is not public yet — shipping it here would + let callers send a value the API rejects with ``422``.""" + assert [m.value for m in Knowledge] == ["core"] + + +# --------------------------------------------------------------------------- +# Request wiring +# --------------------------------------------------------------------------- + + +class TestKnowledgeRequest: + def test_knowledge_lands_on_wire(self): + assert _search_body(knowledge="core")["knowledge"] == "core" + + def test_knowledge_lowercased(self): + assert _search_body(knowledge="CORE")["knowledge"] == "core" + + def test_knowledge_accepts_enum_instance(self): + assert _search_body(knowledge=Knowledge.CORE)["knowledge"] == "core" + + def test_knowledge_omitted_by_default(self): + assert "knowledge" not in _search_body() + + def test_knowledge_does_not_disturb_neighbours(self): + body = _search_body(knowledge="core", count=5, safesearch="off") + assert body["knowledge"] == "core" + assert body["count"] == 5 + assert body["safesearch"] == "off" + + def test_request_body_model_serializes_enum(self): + body = models.SearchRequestBody(query="q", knowledge=Knowledge.CORE) + assert body.model_dump(mode="json")["knowledge"] == "core" + + def test_invalid_value_raises_locally(self): + """``knowledge`` is enum-typed on the request body, so a value the API + would reject with ``422`` raises ``ValidationError`` before any + request is sent -- the same local-mirrors-server pattern as + ``extraction``.""" + with pytest.raises(ValidationError): + _search_body(knowledge="detailed") + + +# --------------------------------------------------------------------------- +# Response parsing +# --------------------------------------------------------------------------- + + +class TestKnowledgeResponse: + def test_parses_full_answer(self): + resp = _parse({"knowledge": [_ANSWER]}) + assert resp.results.knowledge is not None + kr = resp.results.knowledge[0] + assert kr.type == "answer" + assert kr.title == "Paris is the capital of France" + assert kr.description.startswith("Paris has been") + assert kr.as_of == "2026-09-01" + assert kr.attribution[0].name == "Encyclopedia Britannica" + assert kr.attribution[0].source_description == "General knowledge" + + def test_optional_fields_default_to_none(self): + kr = KnowledgeResult.model_validate( + {"type": "answer", "title": "t", "attribution": [{"name": "n"}]} + ) + assert kr.description is None + assert kr.as_of is None + assert kr.attribution[0].source_description is None + + def test_optional_fields_omitted_on_round_trip(self): + """Absent optionals stay absent rather than serializing as null.""" + kr = KnowledgeResult.model_validate( + {"type": "answer", "title": "t", "description": "d", "attribution": [{"name": "n"}]} + ) + dumped = kr.model_dump(mode="json") + assert dumped == { + "type": "answer", + "title": "t", + "attribution": [{"name": "n"}], + "description": "d", + } + + def test_attribution_omits_absent_source_description(self): + dumped = KnowledgeAttribution(name="n").model_dump(mode="json") + assert dumped == {"name": "n"} + + def test_unknown_type_parses(self): + """Forward compat: the spec says to ignore an unrecognized ``type`` + rather than fail, since a new kind may populate different fields.""" + kr = KnowledgeResult.model_validate( + {"type": "some_future_kind", "title": "t", "attribution": [{"name": "n"}]} + ) + assert kr.type == "some_future_kind" + + def test_missing_knowledge_key_is_none(self): + """Prod omits ``results.knowledge`` when nothing is relevant.""" + assert _parse({"web": []}).results.knowledge is None + + def test_multiple_results_preserved(self): + resp = _parse({"knowledge": [_ANSWER, {**_ANSWER, "title": "second"}]}) + assert [r.title for r in resp.results.knowledge] == [ + "Paris is the capital of France", + "second", + ] + + def test_knowledge_alongside_news(self): + resp = _parse({"knowledge": [_ANSWER], "news": []}) + assert resp.results.knowledge is not None + assert resp.results.news == [] + + +# --------------------------------------------------------------------------- +# End to end through the SDK +# --------------------------------------------------------------------------- + + +class TestKnowledgeEndToEnd: + def test_search_returns_parsed_knowledge(self): + with _capture({"results": {"knowledge": [_ANSWER]}}) as (you, captured): + resp = you.search(query="what is the capital of France", knowledge="core") + assert captured["body"]["knowledge"] == "core" + assert resp.results.knowledge[0].title == "Paris is the capital of France" + + +# --------------------------------------------------------------------------- +# Async + deprecated-spelling parity +# --------------------------------------------------------------------------- + + +class TestKnowledgeParity: + @pytest.mark.asyncio + async def test_search_async_sends_knowledge(self): + captured: dict = {} + + def handler(request): + captured["body"] = json.loads(request.content) + return httpx.Response( + 200, + headers={"content-type": "application/json"}, + content=json.dumps({"results": {"knowledge": [_ANSWER]}}), + ) + + async with httpx.AsyncClient(transport=httpx.MockTransport(handler)) as ac: + async with You( + api_key_auth="k", server_url="http://mock.local", async_client=ac + ) as you: + resp = await you.search_async(query="q", knowledge="core") + + assert captured["body"]["knowledge"] == "core" + assert resp.results.knowledge[0].type == "answer" + + def test_deprecated_unified_passes_knowledge(self): + with _capture(_EMPTY) as (you, captured): + with pytest.warns(DeprecationWarning): + you.search.unified(query="q", knowledge="core") + assert captured["body"]["knowledge"] == "core" + + @pytest.mark.asyncio + async def test_deprecated_unified_async_passes_knowledge(self): + captured: dict = {} + + def handler(request): + captured["body"] = json.loads(request.content) + return httpx.Response( + 200, + headers={"content-type": "application/json"}, + content=json.dumps(_EMPTY), + ) + + async with httpx.AsyncClient(transport=httpx.MockTransport(handler)) as ac: + async with You( + api_key_auth="k", server_url="http://mock.local", async_client=ac + ) as you: + with pytest.warns(DeprecationWarning): + await you.search.unified_async(query="q", knowledge="core") + + assert captured["body"]["knowledge"] == "core" diff --git a/tests/test_live.py b/tests/test_live.py index 1d61126..90176cb 100644 --- a/tests/test_live.py +++ b/tests/test_live.py @@ -16,6 +16,7 @@ """ import os +from datetime import datetime import httpx import pytest @@ -29,6 +30,7 @@ ExtractionMode, ExtractionSource, Freshness, + Knowledge, LiveCrawl, LiveCrawlFormats, SafeSearch, @@ -389,6 +391,85 @@ def test_full_page_extraction_source_fetch(self, you_client): ) +@requires_api_key +class TestLiveSearchKnowledge: + """Live tests for the ``knowledge`` parameter on ``POST /v1/search``. + + ``KNOWLEDGE_QUERY`` is verified against prod to return knowledge results. + Coverage is data-dependent, so these assert the shape of what comes back + rather than any specific card. + + These deliberately make no assertion about ``results.web``. Knowledge + coverage varies by query and the web section is not part of the knowledge + contract, so coupling the two would make these tests flaky for reasons + unrelated to this surface. + """ + + KNOWLEDGE_QUERY = "what is the capital of France" + + def test_knowledge_core_returns_results(self, you_client): + with you_client as you: + res = you.search(query=self.KNOWLEDGE_QUERY, knowledge=Knowledge.CORE) + + assert res.results is not None + assert res.results.knowledge, ( + "Expected knowledge results for a query verified to have them; " + "server-side coverage may have changed" + ) + + def test_knowledge_accepts_plain_string(self, you_client): + """Enum-typed params take plain strings, like ``safesearch``.""" + with you_client as you: + res = you.search(query=self.KNOWLEDGE_QUERY, knowledge="core") + + assert res.results.knowledge + + def test_knowledge_result_shape(self, you_client): + with you_client as you: + res = you.search(query=self.KNOWLEDGE_QUERY, knowledge="core") + + assert res.results.knowledge + for kr in res.results.knowledge: + assert kr.type == "answer", "`answer` is the only kind returned today" + assert kr.title + # description is required on type=answer results + assert kr.description + assert kr.attribution, "attribution is required on every kind" + for credit in kr.attribution: + assert credit.name + # as_of is optional; when present it is a bare YYYY-MM-DD date + if kr.as_of is not None: + datetime.strptime(kr.as_of, "%Y-%m-%d") + + def test_knowledge_within_documented_cap(self, you_client): + """Up to 25 knowledge results. ``count`` caps the web/news sections, + not knowledge -- knowledge has its own limit.""" + with you_client as you: + res = you.search(query=self.KNOWLEDGE_QUERY, knowledge="core", count=1) + + assert res.results.knowledge + assert len(res.results.knowledge) <= 25 + + def test_knowledge_omitted_when_not_requested(self, you_client): + """Baseline calls omit ``results.knowledge`` entirely, so the parsed + attribute is ``None`` rather than an empty list.""" + with you_client as you: + res = you.search(query="Python programming language") + + assert res.results is not None + assert res.results.knowledge is None + + @pytest.mark.asyncio + async def test_search_async_knowledge(self, you_client): + async with you_client as you: + res = await you.search_async( + query=self.KNOWLEDGE_QUERY, knowledge="core" + ) + + assert res.results is not None + assert res.results.knowledge + + @requires_api_key class TestLiveContents: """Live tests for the Contents API.""" diff --git a/tests/test_param_normalization.py b/tests/test_param_normalization.py index cdc0365..a56acfa 100644 --- a/tests/test_param_normalization.py +++ b/tests/test_param_normalization.py @@ -3,7 +3,8 @@ The SDK advertises that enum-typed parameters accept plain strings in any case so callers never have to import an enum class. `country`/`language` normalize upward (their enum members are uppercase); `safesearch`, -`livecrawl`, `livecrawl_formats`, and `freshness` normalize downward. +`knowledge`, `livecrawl`, `livecrawl_formats`, and `freshness` normalize +downward. Also pins the three-way `language` contract, which is easy to break: @@ -19,7 +20,7 @@ import pytest from youdotcom import You -from youdotcom.models import Country, Language, LiveCrawl, SafeSearch +from youdotcom.models import Country, Knowledge, Language, LiveCrawl, SafeSearch _SEARCH_BODY = json.dumps({"results": {"web": []}}) @@ -83,6 +84,10 @@ class TestLowercaseParams: def test_safesearch_normalizes_to_lower(self, value): assert _search_body(safesearch=value)["safesearch"] == "strict" + @pytest.mark.parametrize("value", ["core", "CORE", Knowledge.CORE]) + def test_knowledge_normalizes_to_lower(self, value): + assert _search_body(knowledge=value)["knowledge"] == "core" + @pytest.mark.parametrize("value", ["web", "WEB", LiveCrawl.WEB]) def test_livecrawl_normalizes_to_lower(self, value): assert _search_body(livecrawl=value)["livecrawl"] == "web" diff --git a/tests/test_performance.py b/tests/test_performance.py index e6e6fe4..eca2a14 100644 --- a/tests/test_performance.py +++ b/tests/test_performance.py @@ -569,6 +569,31 @@ def call(): if show_detailed: print_detailed_metrics(metrics) + # ---------------------------------------------------------------- + # Knowledge case: `knowledge="core"` adds a licensed-data section to + # the response, which changes payload size and response latency. + # ---------------------------------------------------------------- + + def test_search_with_knowledge_core(self, server_url, api_key, iterations, show_detailed): + """Search with knowledge="core" (adds the results.knowledge section).""" + client = create_timing_client("post_/v1/search") + + with You(server_url=server_url, client=client, api_key_auth=api_key, timeout_ms=90_000) as you: + def call(): + you.search( + query="what is the capital of France", + count=3, + knowledge="core", + server_url=server_url, + ) + + metrics = measure_sdk_call( + call, client, iterations, "Search: knowledge=core" + ) + ALL_METRICS.append(metrics) + if show_detailed: + print_detailed_metrics(metrics) + # ============================================================================ # Contents Endpoint Tests diff --git a/uv.lock b/uv.lock index 5362b2f..bc8321c 100644 --- a/uv.lock +++ b/uv.lock @@ -954,7 +954,7 @@ wheels = [ [[package]] name = "youdotcom" -version = "3.4.0" +version = "3.5.0" source = { editable = "." } dependencies = [ { name = "httpcore" }, From e64094aeed62e8588b1a7c8bbd3cf84c54655d1d Mon Sep 17 00:00:00 2001 From: Tyler Eastman Date: Mon, 21 Sep 2026 13:08:42 -0700 Subject: [PATCH 2/7] fix: address review findings on drift stale logic and live assertions (DX-835) Two of three review findings verified valid and fixed here. check_drift.py: the KNOWN_RESPONSE_GAPS staleness check compared against `missing_in_sdk`, so a field the spec *dropped* was reported as stale with the message "the SDK model now defines it" -- false, and it would fail a strict drift run for the wrong reason. Staleness now compares against the model's own fields (`known & model_fields`). Negative-tested four ways: spec-drops-field emits nothing, SDK-defines-field still reports stale, genuine new drift is still caught, and existing suppressions still hide the known gaps. The previous logic reproduced the false positive on the first case. test_live.py: test_knowledge_result_shape hard-asserted `type == "answer"` and required `description` on every result, contradicting the forward-compat design behind modeling `type` as a plain str. The spec states description is "Required on `type: answer` results" only, and that an unrecognized kind should be ignored rather than failed on. The test now asserts the fields the spec requires of every kind and scopes description to `type == "answer"`. Rejected: rewording the Knowledge docstring to drop "alongside web and news search". That sentence is verbatim from the published OpenAPI spec, which the model docstrings mirror by convention. The live API contradicting it is the server-side bug tracked separately; rewriting the SDK contract to match a launch-blocking bug would desync the SDK from the spec and docs. --- scripts/check_drift.py | 6 +++++- tests/test_live.py | 10 +++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/scripts/check_drift.py b/scripts/check_drift.py index 1c40d8f..0a3bd3d 100644 --- a/scripts/check_drift.py +++ b/scripts/check_drift.py @@ -463,7 +463,11 @@ def _compare_response_fields( missing_in_spec = model_fields - set(props) known = KNOWN_RESPONSE_GAPS.get((spec_name, field_path), set()) - stale = known - missing_in_sdk + # Stale means the SDK model now defines the field, so the suppression no + # longer does anything. Compare against the model, not against what's + # missing: a field the spec dropped is neither missing nor defined, and + # must not be reported as stale. + stale = known & model_fields if stale: warnings.append( f"[response] {path}: KNOWN_RESPONSE_GAPS entry {stale} is stale — " diff --git a/tests/test_live.py b/tests/test_live.py index 90176cb..1f79d35 100644 --- a/tests/test_live.py +++ b/tests/test_live.py @@ -430,13 +430,17 @@ def test_knowledge_result_shape(self, you_client): assert res.results.knowledge for kr in res.results.knowledge: - assert kr.type == "answer", "`answer` is the only kind returned today" + # ``type`` is a plain str so an unrecognized future kind parses + # rather than raising; assert only what the spec requires of every + # kind instead of pinning ``answer`` as the sole value. + assert isinstance(kr.type, str) and kr.type assert kr.title - # description is required on type=answer results - assert kr.description assert kr.attribution, "attribution is required on every kind" for credit in kr.attribution: assert credit.name + # description is required on type=answer results only + if kr.type == "answer": + assert kr.description # as_of is optional; when present it is a bare YYYY-MM-DD date if kr.as_of is not None: datetime.strptime(kr.as_of, "%Y-%m-%d") From 01986d14c1f4776d9e9ac77d25fe92c4cd4b43bd Mon Sep 17 00:00:00 2001 From: Tyler Eastman Date: Mon, 21 Sep 2026 13:23:30 -0700 Subject: [PATCH 3/7] docs: make the README knowledge snippet copy-paste runnable (DX-835) The Knowledge results snippet used a bare `you`, so pasting it alone raised NameError, and it made a network call without timeout_ms. Its direct structural precedent -- `#### Page content extraction`, the same kind of new-search-param subsection at the same heading level -- is fully wrapped with imports, client construction, and timeout_ms=60_000. The bare-style snippets in this README are only the short top-level API-overview blurbs that follow the Quickstart. Now matches the extraction subsection. Verified by extracting the rendered snippet and running it verbatim against prod: prints knowledge card titles, descriptions, and attribution credits. --- README.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index a947acd..0ed574c 100644 --- a/README.md +++ b/README.md @@ -152,15 +152,18 @@ encyclopedias, market-data firms, reference publishers. They come back in their own section: ```python -res = you.search( - query="what is the capital of France", - knowledge="core", -) +import os +from youdotcom import You -for card in res.results.knowledge or []: - print(card.title) - print(card.description) - print([credit.name for credit in card.attribution]) +with You(api_key_auth=os.getenv("YDC_API_KEY"), timeout_ms=60_000) as you: + res = you.search( + query="what is the capital of France", + knowledge="core", + ) + for card in res.results.knowledge or []: + print(card.title) + print(card.description) + print([credit.name for credit in card.attribution]) ``` `"core"` is the only value the API accepts; anything else raises From 98c1fc40337e1ae90b3c928e868b1bb47117a587 Mon Sep 17 00:00:00 2001 From: Tyler Eastman Date: Mon, 21 Sep 2026 14:28:10 -0700 Subject: [PATCH 4/7] docs: mark web and news as Optional in the Results type cells (DX-835) `Results.web` and `Results.news` are `Optional[List[...]]` in `searchresponse.py`, but their `docs/models/results.md` type cells read plain `List[...]`. The wrong cells pre-date this branch, but adding the `knowledge` row re-padded the table and re-emitted both lines, so they are changed lines in this diff. AGENTS.md ("Type cells in field tables come from the annotation") names `Optional[X]` vs `X` as a trap and requires eyeballing every row of a touched model page, not just the rows whose prose changed. Verified with an annotation-vs-docs comparison across all four touched model pages (25 rows): all match, including the new `knowledge` row. --- docs/models/results.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/models/results.md b/docs/models/results.md index f1f4007..a033b95 100644 --- a/docs/models/results.md +++ b/docs/models/results.md @@ -5,6 +5,6 @@ | Field | Type | Required | Description | | -------------------------------------------------- | ---------------------------------------------------------------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `web` | List[[models.WebResult](../models/webresult.md)] | :heavy_minus_sign: | N/A | -| `news` | List[[models.NewsResult](../models/newsresult.md)] | :heavy_minus_sign: | N/A | +| `web` | Optional[List[[models.WebResult](../models/webresult.md)]] | :heavy_minus_sign: | N/A | +| `news` | Optional[List[[models.NewsResult](../models/newsresult.md)]] | :heavy_minus_sign: | N/A | | `knowledge` | Optional[List[[models.KnowledgeResult](../models/knowledgeresult.md)]] | :heavy_minus_sign: | Results backed by licensed data providers. Up to 25 are returned, limited to those relevant to the query. When none are relevant the key is omitted rather than returned as an empty array. | \ No newline at end of file From e8fa8d63854f74fce4c20d68aec533eceabffb94 Mon Sep 17 00:00:00 2001 From: Tyler Eastman Date: Mon, 21 Sep 2026 14:46:28 -0700 Subject: [PATCH 5/7] fix: sync knowledge field docstrings and relax a live description assert (DX-835) Two findings from the second droid-review round, both verified valid. Cross-table identity: the `knowledge` description in docs/sdks/you/README.md, docs/sdks/search/README.md and docs/models/searchrequestbody.md carried the `"core"`-only / ValidationError / 25-result / `count` constraints, but both `SearchRequestBody` field docstrings stopped after the first sentence. AGENTS.md requires those surfaces to read identically. Expanded the docstrings rather than shrinking the tables, matching how `include_domains` and `extraction` already carry SDK-layer guidance past the spec sentence. The `Knowledge` enum class docstring stays spec-verbatim: it is not one of the four cross-table surfaces, `components.schemas.Knowledge.description` is exactly that sentence (re-fetched), and `SearchRequestBody.properties.knowledge` carries no description of its own. `FreshnessValue` and `LiveCrawlFormats` likewise have enum docstrings that differ from their field docstrings. Live assert: `assert kr.description` -> `assert kr.description is not None`. `description` is `Optional[str]`, so `is not None` is the real presence check; truthiness would also enforce non-empty, which AGENTS.md's "`is not None`, not truthiness" rule warns against for server-returned strings. `type`, `title` and `attribution[].name` keep truthiness deliberately: those are required `str` fields, where `is not None` is vacuous (pydantic already guarantees it) and non-empty is the meaningful claim. --- src/youdotcom/models/searchrequestbody.py | 4 ++-- tests/test_live.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/youdotcom/models/searchrequestbody.py b/src/youdotcom/models/searchrequestbody.py index e880925..d7e3b1b 100644 --- a/src/youdotcom/models/searchrequestbody.py +++ b/src/youdotcom/models/searchrequestbody.py @@ -34,7 +34,7 @@ class SearchRequestBodyTypedDict(TypedDict): safesearch: NotRequired[SafeSearch] r"""Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not.""" knowledge: NotRequired[Knowledge] - r"""Requests knowledge results alongside web and news search.""" + r"""Requests knowledge results alongside web and news search. `"core"` is the only value the API accepts; anything else raises `ValidationError` locally, mirroring the server's `422`. Returns up to 25 results, limited to those relevant to the query — `count` caps the web and news sections, not knowledge.""" livecrawl: NotRequired[LiveCrawl] r"""Deprecated; use `extraction` instead. Indicates which section(s) of search results to livecrawl and return full page content.""" livecrawl_formats: NotRequired[List[LiveCrawlFormats]] @@ -91,7 +91,7 @@ class SearchRequestBody(BaseModel): r"""Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not.""" knowledge: Optional[Knowledge] = None - r"""Requests knowledge results alongside web and news search.""" + r"""Requests knowledge results alongside web and news search. `"core"` is the only value the API accepts; anything else raises `ValidationError` locally, mirroring the server's `422`. Returns up to 25 results, limited to those relevant to the query — `count` caps the web and news sections, not knowledge.""" livecrawl: Optional[LiveCrawl] = None r"""Deprecated; use `extraction` instead. Indicates which section(s) of search results to livecrawl and return full page content.""" diff --git a/tests/test_live.py b/tests/test_live.py index 1f79d35..1cd4506 100644 --- a/tests/test_live.py +++ b/tests/test_live.py @@ -440,7 +440,7 @@ def test_knowledge_result_shape(self, you_client): assert credit.name # description is required on type=answer results only if kr.type == "answer": - assert kr.description + assert kr.description is not None # as_of is optional; when present it is a bare YYYY-MM-DD date if kr.as_of is not None: datetime.strptime(kr.as_of, "%Y-%m-%d") From be9b0659c0702ea3023770f1279b8bcc59e954d6 Mon Sep 17 00:00:00 2001 From: Tyler Eastman Date: Mon, 21 Sep 2026 15:32:46 -0700 Subject: [PATCH 6/7] docs: fix pre-existing Optional type cells and refresh the tests inventory (DX-835) Docs-only. Two independent staleness fixes found while auditing every doc surface in the repo. Type cells: 11 rows across 8 model pages declared a bare type where the annotation is Optional -- contentsrequest (urls, formats), sourcecontrol (include_domains, exclude_domains, boost_domains), source (snippets), searchrequest (livecrawl_formats), researchdetail and financeresearchdetail (ctx), researchrequest (output_schema), researchtaskstreameventdata (data). AGENTS.md names Optional[X] <-> X as a type-cell trap. All pre-date this branch. Every cell fits its existing padded column, so only those 11 lines change and table alignment is preserved. The Required cells were already right: is_required() is False for all 11. tests/README.md: listed 11 of the 20 test files and carried wrong counts (Contents 12 vs 13, Answer 23 vs 25). Now lists all 20, adds groups for extraction, knowledge, page_age, stream events and the cross-cutting suites, and labels counts as pytest --collect-only numbers. The groups sum to 425, exactly the CI-gate collection. Verified with a heading-aware audit comparing every field table in docs/models/ against typing.get_type_hints -- 151 rows, all four AGENTS.md traps (Optional<->X, datetime<->date, bool<->int, str<->int): 0 findings left. The 15 enum "## Values" pages were also diffed against their code members: clean, the one apparent hit being contentsformats' intentional "metadata _(deprecated)_" annotation. --- docs/models/contentsrequest.md | 4 +- docs/models/financeresearchdetail.md | 2 +- docs/models/researchdetail.md | 2 +- docs/models/researchrequest.md | 2 +- docs/models/researchtaskstreameventdata.md | 2 +- docs/models/searchrequest.md | 2 +- docs/models/source.md | 2 +- docs/models/sourcecontrol.md | 6 +-- tests/README.md | 57 ++++++++++++++++++++-- 9 files changed, 64 insertions(+), 15 deletions(-) diff --git a/docs/models/contentsrequest.md b/docs/models/contentsrequest.md index ba41c58..21f039f 100644 --- a/docs/models/contentsrequest.md +++ b/docs/models/contentsrequest.md @@ -5,7 +5,7 @@ | Field | Type | Required | Description | Example | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `urls` | List[*str*] | :heavy_minus_sign: | Array of URLs to fetch the contents from. | | -| `formats` | List[[models.ContentsFormats](../models/contentsformats.md)] | :heavy_minus_sign: | Array of content formats to return. All included formats are returned in the response. The "metadata" format is deprecated and will be removed in a future major release. | [
"html",
"markdown"
] | +| `urls` | Optional[List[*str*]] | :heavy_minus_sign: | Array of URLs to fetch the contents from. | | +| `formats` | Optional[List[[models.ContentsFormats](../models/contentsformats.md)]] | :heavy_minus_sign: | Array of content formats to return. All included formats are returned in the response. The "metadata" format is deprecated and will be removed in a future major release. | [
"html",
"markdown"
] | | `crawl_timeout` | *Optional[int]* | :heavy_minus_sign: | Maximum time in seconds to wait for page content. Must be between 1 and 60 seconds. Default is 10 seconds. | 10 | | `max_age` | *OptionalNullable[int]* | :heavy_minus_sign: | Maximum allowed age of cached content in seconds. When set, cached content older than this threshold is ignored and the page is re-fetched. Must be 0 or greater. Default: null (no age limit, cached content is returned regardless of age). | 86400 | \ No newline at end of file diff --git a/docs/models/financeresearchdetail.md b/docs/models/financeresearchdetail.md index ca5a772..951cce1 100644 --- a/docs/models/financeresearchdetail.md +++ b/docs/models/financeresearchdetail.md @@ -9,4 +9,4 @@ | `loc` | List[[models.FinanceResearchLoc](../models/financeresearchloc.md)] | :heavy_check_mark: | The location of the error as a path of segments (strings for field names, integers for byte offsets). | [
"body",
"input"
] | | `msg` | *str* | :heavy_check_mark: | A human-readable description of the error. | Field required | | `input` | [models.FinanceResearchInputUnion](../models/financeresearchinputunion.md) | :heavy_check_mark: | The input value that caused the error. | | -| `ctx` | Dict[str, *Any*] | :heavy_minus_sign: | Additional context about the error. | | \ No newline at end of file +| `ctx` | Optional[Dict[str, *Any*]] | :heavy_minus_sign: | Additional context about the error. | | \ No newline at end of file diff --git a/docs/models/researchdetail.md b/docs/models/researchdetail.md index 8ee1a3b..ef69971 100644 --- a/docs/models/researchdetail.md +++ b/docs/models/researchdetail.md @@ -9,4 +9,4 @@ | `loc` | List[[models.ResearchLoc](../models/researchloc.md)] | :heavy_check_mark: | The location of the error as a path of segments (strings for field names, integers for byte offsets). | [
"body",
"input"
] | | `msg` | *str* | :heavy_check_mark: | A human-readable description of the error. | Field required | | `input` | [models.ResearchInputUnion](../models/researchinputunion.md) | :heavy_check_mark: | The input value that caused the error. | | -| `ctx` | Dict[str, *Any*] | :heavy_minus_sign: | Additional context about the error. | | \ No newline at end of file +| `ctx` | Optional[Dict[str, *Any*]] | :heavy_minus_sign: | Additional context about the error. | | \ No newline at end of file diff --git a/docs/models/researchrequest.md b/docs/models/researchrequest.md index cd83057..22aa3da 100644 --- a/docs/models/researchrequest.md +++ b/docs/models/researchrequest.md @@ -9,4 +9,4 @@ | `research_effort` | [Optional[models.ResearchEffort]](../models/researcheffort.md) | :heavy_minus_sign: | Controls how much time and effort the Research API spends on your question. Higher effort levels run more searches and dig deeper into sources, at the cost of a longer response time.

Available levels:
- `lite`: Returns answers quickly. Good for straightforward questions that just need a fast, reliable answer.
- `standard`: The default. Balances speed and depth, a good fit for most questions.
- `deep`: Spends more time researching and cross-referencing sources. Use this when accuracy and thoroughness matter more than speed.
- `exhaustive`: The most thorough option. Explores the topic as fully as possible, best suited for complex research tasks where you want the highest quality result.
- `frontier`: The highest-quality, longest-running tier (up to 4 hours). Requires `background=true`; returns 422 otherwise. | | `background` | *Optional[bool]* | :heavy_minus_sign: | When true, queue a research task and return a task handle immediately instead of waiting for the result inline. Defaults to synchronous. When enabled, the response is a TaskResponse object with a task_id and stream_url for polling progress via GET /v1/research/{task_id} or streaming via GET /v1/research/{task_id}/stream. | | `source_control` | [Optional[models.SourceControl]](../models/sourcecontrol.md) | :heavy_minus_sign: | Beta. Controls which web sources the research agent searches and visits. Use this to allow specific domains, block specific domains, boost specific domains, filter by recency, or focus web results by country.

`include_domains` and `exclude_domains` cannot be used together. Each domain list is capped at 500 entries. `exclude_domains` also blocks the research agent from visiting pages on those domains during browsing. `boost_domains` gives matching domains a relative ranking boost without filtering out other domains. It can be combined with `exclude_domains` but cannot be combined with `include_domains`. | -| `output_schema` | Dict[str, *Any*] | :heavy_minus_sign: | Beta. Requests structured JSON output in output.content using a supported JSON Schema subset. Supported only with research_effort values standard, deep, and exhaustive. Sending output_schema with research_effort: "lite" returns 422.

Schema rules: Root must be a JSON object. Top-level anyOf is not allowed. Every object must define properties and set additionalProperties: false. Every property must be listed in required. Recursive schemas are not supported.

Limits: Max nesting depth 5, max total properties 100, max total enum values 500, max total schema string budget 25,000. | \ No newline at end of file +| `output_schema` | Optional[Dict[str, *Any*]] | :heavy_minus_sign: | Beta. Requests structured JSON output in output.content using a supported JSON Schema subset. Supported only with research_effort values standard, deep, and exhaustive. Sending output_schema with research_effort: "lite" returns 422.

Schema rules: Root must be a JSON object. Top-level anyOf is not allowed. Every object must define properties and set additionalProperties: false. Every property must be listed in required. Recursive schemas are not supported.

Limits: Max nesting depth 5, max total properties 100, max total enum values 500, max total schema string budget 25,000. | \ No newline at end of file diff --git a/docs/models/researchtaskstreameventdata.md b/docs/models/researchtaskstreameventdata.md index 56634db..ca0abd9 100644 --- a/docs/models/researchtaskstreameventdata.md +++ b/docs/models/researchtaskstreameventdata.md @@ -10,6 +10,6 @@ The event payload. Structure varies by event type. Common fields include type, t | `type` | *Optional[str]* | :heavy_minus_sign: | The event type identifier. | | `task_id` | *Optional[str]* | :heavy_minus_sign: | The task UUID. | | `status` | *Optional[str]* | :heavy_minus_sign: | Current task status when the event was emitted. | -| `data` | Dict[str, *Any*] | :heavy_minus_sign: | Event-specific payload data. | +| `data` | Optional[Dict[str, *Any*]] | :heavy_minus_sign: | Event-specific payload data. | | `error` | *OptionalNullable[str]* | :heavy_minus_sign: | Error message if the event represents an error. | | `sequence` | *Optional[int]* | :heavy_minus_sign: | Event sequence number. | \ No newline at end of file diff --git a/docs/models/searchrequest.md b/docs/models/searchrequest.md index 3c07e4c..96f3102 100644 --- a/docs/models/searchrequest.md +++ b/docs/models/searchrequest.md @@ -13,7 +13,7 @@ | `language` | [Optional[models.Language]](../models/language.md) | :heavy_minus_sign: | The language of the web results that will be returned (BCP 47 format). | | | `safesearch` | [Optional[models.SafeSearch]](../models/safesearch.md) | :heavy_minus_sign: | Configures the safesearch filter for content moderation. This allows you to decide whether to return NSFW content or not. | | | `livecrawl` | [Optional[models.LiveCrawl]](../models/livecrawl.md) | :heavy_minus_sign: | Indicates which section(s) of search results to livecrawl and return full page content. | | -| `livecrawl_formats` | List[[models.LiveCrawlFormats](../models/livecrawlformats.md)] | :heavy_minus_sign: | N/A | | +| `livecrawl_formats` | Optional[List[[models.LiveCrawlFormats](../models/livecrawlformats.md)]] | :heavy_minus_sign: | N/A | | | `include_domains` | *Optional[str]* | :heavy_minus_sign: | A list of domains to restrict search results to. Only results from these domains will be returned. For large domain lists (up to 500), use POST with a JSON array instead. This is a strict allowlist — cannot be combined with `exclude_domains` (returns `422`).

**Important:** Use a single comma-separated value (e.g. `include_domains=nytimes.com,bbc.com`). Repeated parameters (`include_domains=a.com&include_domains=b.com`) are not supported. | nytimes.com,bbc.com | | `exclude_domains` | *Optional[str]* | :heavy_minus_sign: | A list of domains to exclude from search results. Results from these domains will be filtered out. For large domain lists (up to 500), use POST with a JSON array instead. Cannot be combined with `include_domains` (returns `422`).

**Important:** You must use a single comma-separated value (e.g. `exclude_domains=spam-site.com,other-site.com`). Repeated parameters are not supported. | spam-site.com,other-site.com | | `boost_domains` | *Optional[str]* | :heavy_minus_sign: | A list of domains to boost in search ranking. Matching results from these domains receive a relative ranking boost, but results are not limited to these domains. Supports up to 500 domains. Can be combined with `exclude_domains`, but cannot be combined with `include_domains` (returns `422`).

**Important:** You must use a single comma-separated value (e.g. `boost_domains=nytimes.com,wired.com`). Repeated parameters are not supported. | nytimes.com,wired.com | diff --git a/docs/models/source.md b/docs/models/source.md index 174794d..5e51467 100644 --- a/docs/models/source.md +++ b/docs/models/source.md @@ -7,4 +7,4 @@ | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | `url` | *str* | :heavy_check_mark: | The URL of the source webpage. | | `title` | *Optional[str]* | :heavy_minus_sign: | The title of the source webpage. | -| `snippets` | List[*str*] | :heavy_minus_sign: | Relevant excerpts from the source page that were used in generating the answer. | \ No newline at end of file +| `snippets` | Optional[List[*str*]] | :heavy_minus_sign: | Relevant excerpts from the source page that were used in generating the answer. | \ No newline at end of file diff --git a/docs/models/sourcecontrol.md b/docs/models/sourcecontrol.md index cc42a85..c1fdc62 100644 --- a/docs/models/sourcecontrol.md +++ b/docs/models/sourcecontrol.md @@ -9,8 +9,8 @@ Beta. Controls which web sources the research agent searches and visits. Use thi | Field | Type | Required | Description | | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | -| `include_domains` | List[*str*] | :heavy_minus_sign: | Only return results from these domains. Max 500 domains. Cannot be used with exclude_domains or boost_domains. | -| `exclude_domains` | List[*str*] | :heavy_minus_sign: | Never return results from these domains. Max 500 domains. Also blocks the research agent from visiting pages on those domains during browsing. | -| `boost_domains` | List[*str*] | :heavy_minus_sign: | Boost results from these domains without excluding other domains. Max 500 domains. Cannot be used with include_domains. | +| `include_domains` | Optional[List[*str*]] | :heavy_minus_sign: | Only return results from these domains. Max 500 domains. Cannot be used with exclude_domains or boost_domains. | +| `exclude_domains` | Optional[List[*str*]] | :heavy_minus_sign: | Never return results from these domains. Max 500 domains. Also blocks the research agent from visiting pages on those domains during browsing. | +| `boost_domains` | Optional[List[*str*]] | :heavy_minus_sign: | Boost results from these domains without excluding other domains. Max 500 domains. Cannot be used with include_domains. | | `freshness` | *Optional[str]* | :heavy_minus_sign: | Filter results by recency. Accepts `day`, `week`, `month`, `year`, or a custom date range in `YYYY-MM-DDtoYYYY-MM-DD` format. | | `country` | *Optional[str]* | :heavy_minus_sign: | ISO 3166-1 alpha-2 country code, such as US, GB, or DE, to geographically focus web results. | \ No newline at end of file diff --git a/tests/README.md b/tests/README.md index f4e6146..77c321e 100644 --- a/tests/README.md +++ b/tests/README.md @@ -53,13 +53,22 @@ pytest tests/ -v - `test_client.py` - Helper utilities for creating test HTTP clients - `test_search.py` - Tests for the Search API (`/v1/search`) +- `test_extraction.py` - Tests for the `extraction` parameter on `you.search` (model contract, strict validation, wire contract, conflicts, plus-value rule, async) +- `test_knowledge.py` - Tests for the `knowledge` parameter and the knowledge result models on `you.search` +- `test_page_age.py` - Tolerance of non-ISO `page_age` values on search and news results - `test_contents.py` - Tests for the Contents API (`/v1/contents`) - `test_answer.py` - Tests for the Answer API (`/v1/answer`) - `test_direct_methods.py` - Tests for direct methods on `You` (search, contents) - `test_shims.py` - Tests for backward-compat sub-SDK shims with DeprecationWarning +- `test_param_normalization.py` - Tests for plain-string parameter normalization (case folding, `language`, deprecated shims) - `test_research.py` - Tests for the Research API (`/v1/research`) including background mode, output_schema, and source_control - `test_research_helpers.py` - Tests for the hand-maintained `research_helpers` module (background submission, polling, streaming, research_and_wait) +- `test_researchtaskstreamevent.py` - Tests for `ResearchTaskStreamEvent` model contracts and the real SSE decode path - `test_security_env.py` - Tests for environment variable precedence (`YDC_API_KEY` / `YOU_API_KEY_AUTH`) +- `test_attribution.py` - Tests for the `X-Client-Info` attribution header (grammar, edge cases, construction-time validation, wire round-trip, version resolution) +- `test_redaction.py` - Tests for debug-log header redaction +- `test_client_lifecycle.py` - Tests for client teardown in `You.__exit__` / `You.__aexit__` +- `test_root_init.py` - Tests for the `youdotcom` package root module - `test_performance.py` - Performance/instrumentation tests measuring SDK overhead - `test_live.py` - Live API tests that run against the real You.com API (requires API key) @@ -67,6 +76,10 @@ pytest tests/ -v Tests are organized into logical classes using pytest: +Counts below are collected tests (`pytest --collect-only`), so a parametrized case +counts once per parameter set. The groups sum to the 425 tests in the CI gate; +`test_performance.py` and `test_live.py` are excluded from that gate. + **Search API** (10 tests): - Basic search functionality - Search with filters (freshness, country, safesearch) @@ -74,19 +87,35 @@ Tests are organized into logical classes using pytest: - News livecrawl with contents - Error handling (unauthorized, forbidden, unprocessable, internal server error) -**Contents API** (12 tests): +**Extraction** (38 tests): +- Model contract and strict validation +- Wire contract and mutual exclusion with the deprecated `livecrawl` +- Plus-value rule and async parity + +**Knowledge** (21 tests): +- `Knowledge` enum and plain-string normalization +- Request wire contract on `search` / `search_async` +- Response parsing into `KnowledgeResult` / `KnowledgeAttribution` +- End-to-end round-trip and sub-SDK shim parity + +**Page age tolerance** (15 tests): +- ISO values still parse to `datetime` +- Non-ISO values returned verbatim instead of failing the whole response +- Wrong JSON types still raise + +**Contents API** (13 tests): - HTML and Markdown format generation - Single and multiple URL processing - Optional format parameter - Error handling (unauthorized, forbidden, empty URLs) -**Answer API** (23 tests): +**Answer API** (25 tests): - Basic answer functionality - Answer with freshness, country, boost domains - Async answer - Error handling (unauthorized, forbidden, payment required, unprocessable, internal server error) -**Research API**: +**Research API** (34 tests): - Basic research functionality (standard, deep, exhaustive effort) - Background mode (task submission, get_research_task, status polling) - Output schema (structured JSON output, content_type object) @@ -94,13 +123,33 @@ Tests are organized into logical classes using pytest: - Error handling (unauthorized, forbidden, unprocessable entity, 422 combos) - Stream research task (SSE success path + 404/401/403 error paths) -**Research Helpers**: +**Research Helpers** (57 tests): - research_background / research_background_async (TaskResponse return) - poll_research_task / poll_research_task_async (terminal status) - research_and_wait / research_and_wait_async (submit + wait) - stream_research / stream_research_async (tolerant SSE) - RawStreamEvent decoder (_decode_raw_event) +**Research Task Stream Events** (26 tests): +- Known and unknown event names +- Round-trip and declared-type contracts +- End-to-end pin through the real SSE decode path + +**Cross-cutting** (186 tests): +- `X-Client-Info` attribution header: grammar, edge cases, construction-time + validation, wire round-trip, version resolution (91) +- Plain-string parameter normalization: case folding, `language` three-way + contract, deprecated shims (35) +- Environment variable precedence `YDC_API_KEY` / `YOU_API_KEY_AUTH` (17) +- Debug-log header redaction (14) +- Client teardown in `You.__exit__` / `You.__aexit__` (10) +- Direct methods on `You` (8) and backward-compat sub-SDK shims (6) +- `youdotcom` package root module (5) + +**Outside the CI gate**: +- `test_performance.py` (33 tests) - SDK overhead instrumentation +- `test_live.py` (46 tests) - runs against the real API, requires an API key + ### Running Live Tests The `test_live.py` file contains tests that run against the real You.com API. All tests require an API key and are skipped unless `YDC_API_KEY` or `YOU_API_KEY_AUTH` is set: From e405cca7acf37903cc0ac673c98637fae741b3b8 Mon Sep 17 00:00:00 2001 From: Tyler Eastman Date: Mon, 21 Sep 2026 15:45:22 -0700 Subject: [PATCH 7/7] docs: make the as_of parsing note self-contained and add timeout_ms to the retries example (DX-835) Two docs footguns, one from review and one found sweeping every snippet. docs/models/knowledgeresult.md: the `as_of` note told readers to call `datetime.strptime(card.as_of, "%Y-%m-%d")` without an import, so following it raised NameError. Replaced with a fenced block that carries `from datetime import datetime`, matching the pattern docs/models/webresult.md already uses for its own `page_age` datetime note. The block also guards the optional section (`res.results.knowledge or []`) and uses `is not None` rather than truthiness, consistent with the model's Optional[str] annotation. README.md: the Retries example builds its own client but passed no `timeout_ms`, so copying it inherits httpx's 5s default and raises ReadTimeout -- the exact failure the adjacent Timeouts section warns about. AGENTS.md requires an explicit `timeout_ms` on any snippet that triggers a round-trip. The other README snippets that call `you.search` / `you.contents` without a `timeout_ms` are deliberately untouched: they reuse the client constructed in Quick Start, which already passes `timeout_ms=60_000`, so the rule is satisfied once upstream rather than per snippet. --- README.md | 4 ++-- docs/models/knowledgeresult.md | 13 ++++++++++--- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 0ed574c..3da99bc 100644 --- a/README.md +++ b/README.md @@ -378,8 +378,8 @@ retries = RetryConfig( retry_connection_errors=True, ) -with You(api_key_auth=key, retry_config=retries) as you: # whole client - res = you.search(query="...", retries=retries) # or one call +with You(api_key_auth=key, retry_config=retries, timeout_ms=60_000) as you: # whole client + res = you.search(query="...", retries=retries) # or one call ``` Retries apply to `429`, `500`, `502`, `503`, and `504`. diff --git a/docs/models/knowledgeresult.md b/docs/models/knowledgeresult.md index 0d0dabe..8f07f62 100644 --- a/docs/models/knowledgeresult.md +++ b/docs/models/knowledgeresult.md @@ -62,6 +62,13 @@ it. ### `as_of` is a string, not a date -`as_of` stays a `str` in `YYYY-MM-DD` form. Parse it with -`datetime.strptime(card.as_of, "%Y-%m-%d")` when you need a date object, and -expect `None` when the provider reports no date. +`as_of` stays a `str` in `YYYY-MM-DD` form, and is `None` when the provider +reports no date. Parse it when you need a date object: + +```python +from datetime import datetime + +for card in res.results.knowledge or []: + if card.as_of is not None: + print(datetime.strptime(card.as_of, "%Y-%m-%d").date()) +```