[czi-fsisenda]

Why

Every Python install brings openai, anthropic and google-genai, with their version pins, even though 18 of our 19 evaluators call one vendor, and a caller who injects their own llm_provider (#405) calls none. This is step 1 of the two-PR plan on DSCR-2648 (#412 is the 2.0 draft). #409 widens the version ranges separately, under DSCR-2625. It adds the extras and a clear error without removing anything from the base install, so no 1.x install changes. Users can move to the new install lines before 2.0 requires them.

Ticket: DSCR-2648 (split from DSCR-2625). Stacked on #405, which it needs: the new test runs every evaluator on an injected llm_provider. The base branch moves to main once #405 merges.

What users see

pip install "learning-commons-evaluators[google]"          # one provider
pip install "learning-commons-evaluators[openai,google]"   # several
pip install "learning-commons-evaluators[all]"             # every provider, same as today

If a provider's SDK isn't installed (from 2.0, or after an uninstall), construction fails before any I/O, naming every extra that's missing:

VocabularyComplexityEvaluator(openai_api_key="...", google_api_key="...")
# ConfigurationError: Vocabulary Complexity Evaluator needs the openai and google-genai packages,
# which are not installed. Install them with: pip install "learning-commons-evaluators[openai,google]"

Changes

• pyproject.toml:
  • Extras openai, anthropic, google and all, named by the Provider values so the error can name the extra directly.
  • The SDKs also stay in dependencies for now, so each pin appears in both places until 2.0.
  • dev depends on [all].
• providers/factory.py:
  • sdk_installed(provider) checks with importlib.util.find_spec, without importing the SDK.
  • missing_sdk_error(...) builds the message.
  • create_provider makes the same check for direct callers.
• BaseEvaluator._validate_sdks:
  • Runs at construction, before the key check, since installing comes before configuring.
  • Covers the providers the instance will call, so a model_override needs its own provider's SDK, and an injected llm_provider needs none.
• tests/unit/test_without_vendor_sdks.py:
  • Runs every evaluator in a child process where no vendor SDK can be imported.
  • Each one must refuse to build its own adapters, run on an injected provider, and import no vendor module.
• New CI job, "Without vendor SDKs": runs that test with the SDKs actually uninstalled.
• Docs:
  • The README install section has an extras table, and the "Bring your own provider" section gains a note on raising errors that classify correctly.
  • The root README, demos/python/requirements.txt ([all]) and docs/local-development.md are updated.

For reviewers

• Mintlify: the quickstart install line and the configuration page in edu-aidt-mintlify-docs need the same update when this releases. That will be a separate PR there.
• Conflict with the floor-widening PR fix(python-sdk): accept older openai and anthropic releases #409: it lowers the pins this PR repeats in the extras. Whichever merges second updates both places.
• Demo pin: [all] works against the current 1.1.0 on PyPI. pip only warns about the unknown extra until this releases.

Testing

• make verify passes (1783 passed, 107 skipped), with coverage at 98.75%. scripts/check.py passes.
• I reproduced the new CI job locally: a fresh venv, the vendor SDKs uninstalled, the test passing. I also checked the real error message in that venv.
• Mutation checks:
  • Removing the meta-path blocker makes the subprocess test fail (Math Standards Alignment builds without its SDK).
  • Disabling _validate_sdks fails it and the evaluator-level tests.
• An independent review pass found no correctness issues. Its one minor point is fixed: a module stubbed into sys.modules without a spec now counts as installed.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!