🐛 A `-D needs_variant_data=x` Sphinx rejects still strips the inline variant table from the TOML

[chrisjsewell]

sphinx-build -D needs_variant_data=x is rejected by Sphinx — WARNING: cannot override dictionary config setting 'needs_variant_data', ignoring (use -D needs_variant_data.key=val to set individual elements) — but the key stays in config.overrides, and sphinx-needs' TOML reader then treats it as an applied override: _load_variants_from_toml removes data from [variants] and variant_data from [needs] before read_variants (packages/sphinx-needs/src/sphinx_needs/needs.py, the _without_key loop keyed on legacy_key in overridden or f"needs_{legacy_key}" in overridden). The inline map declared in the TOML silently vanishes for an override Sphinx never applied.

Measured (2026-09-30, master at d0edd0e9, while reviewing #2000): a project with needs_from_toml = "ubproject.toml", [variants] data_file = "vd.json" + [variants.data] edition = "pro", built with -D needs_variant_data=x → Sphinx's "cannot override … ignoring" warning, then needs_variant_data resolves from the file alone: edition is gone (rc 0, no sphinx-needs warning). The same happens for the legacy [needs.variant_data] location, and the bare spelling -D variant_data=x (an unknown config value Sphinx also ignores, and the loop also honours) does the same.

The dotted form -D needs_variant_data.k=v is unaffected: Sphinx folds it into the raw config and drops it from overrides, so nothing is stripped and the value lands on the confval (#1991 covers the separate problem that it is then lost under a TOML inline table).

Fix shape. Strip a key only for an override Sphinx actually applied: consult config.overrides for the two needs_variant_data* confval names and skip needs_variant_data when the override value is not a mapping (Sphinx never applies a non-dict override to a dict confval), or — simpler and closer to what a user means — treat only needs_variant_data_file and the dotted needs_variant_data.<k> forms as overrides of the inline table's keys. Drop the bare variant_data / variant_data_file spellings from the predicate: they are not confvals and Sphinx ignores them. One test per form (applied -D needs_variant_data_file=… still strips; rejected -D needs_variant_data=x and the bare spellings do not).

sphinx-mounts (#2000) mirrors the current rule verbatim on the file sphinx-needs is pointed at, so the two tools agree; when this changes, sphinx_mounts.extension._without_overridden follows in the same release window (its docstring names the mirror).

[chrisjsewell]

A second arm of the same predicate, measured while #2005 adopted the rule for sphinx-codelinks: _load_variants_from_toml also strips on the BARE spelling (legacy_key in overridden_keys, i.e. -D variant_data_file=… without the needs_ prefix). Sphinx rejects that as an unknown config value ("unknown config value 'variant_data_file' in override, ignoring") and applies nothing, but the key stays in config.overrides, so the TOML's variant_data_file / data_file is removed for an override that never took effect — the same shape as the dict case above, on a different spelling. codelinks (#2005) skips only on the full confval name (src_trace_<key>) and never for a confval Sphinx cannot set from -D; the fix here would follow that: f"needs_{legacy_key}" only, and needs_variant_data only when Sphinx actually applied it (the dotted form is folded into the raw config and never appears in overrides, so it needs nothing). sphinx-mounts mirrors this predicate verbatim on the pointed file (sphinx_mounts.extension._without_overridden) and changes in the same release.