fix: allow index clear to recover from incompatible index schemas

[Mahnoor-Zaffar]

Fixes #167 — Cannot clear or rebuild an index after an index schema upgrade.

What changed

vidxp index clear --yes and reindexing both validated the active snapshot before proceeding, so an index containing generation manifests with an older index_schema_version raised IndexSchemaError and blocked recovery. Clearing also only published an empty snapshot, leaving the Chroma collections intact with their old embedding dimensions.

This change makes recovery possible after an intentional schema change:

• LocalSnapshotRepository.clear() now reads the active snapshot without per-generation manifest validation (validate_generations=False threaded through read_active/read_snapshot), so an incompatible manifest no longer blocks publishing the empty snapshot. Corrupt snapshot/pointer metadata still fails closed.
• The SQL mirror (SQLSnapshotRepository) gets the same tolerant clear() and threads the flag through _publish.
• LocalIndexBackend.clear() now also discards the vector collections (IndexStorage.clear() on the repository store) so a rebuild recreates them with the current embedding dimensions. Missing/empty stores are tolerated.
• The CLI index clear confirmation and output now state that generated index data and vector collections are removed while imported source media is preserved.

No automatic migration — reindex/status still reject incompatible manifests; the supported recovery is clear, then rebuild from existing imported media.

Validation

• uv run --no-sync ruff check . — passes
• New regression tests (real Chroma):
  • test_clear_recovers_from_incompatible_generation_schema
  • test_clear_discards_incompatible_vector_collections
• uv run --no-sync pytest -q — 793 passed. Pre-existing environment failures only: 8 tests need optional model/deps not installed here (torch, opencv, transformers, av), and 5 infra tests import a top-level utils package (confirmed failing on base too).

[Mahnoor-Zaffar]

Hey! Wanted to flag a few follow-ups that I think would make this fix feel complete for the exact scenarios in #167.

This PR clears the main blocker: running vidxp index clear --yes on a repo with an older schema now works, wipes the vector collections (so dims reset), and keeps imported media untouched. But there are three spots where the flow still kind of dead-ends, and a small change each would tie it all together:

1. Reindex still errors with zero guidance. Going through require_compatible_profile() → read_active() still validates the old manifests, and the shared boundary turns that into a generic "index schema is incompatible with this version." So if someone just tries vidxp index <media> or vidxp index bulk --reindex after upgrading, they're stuck with no hint of what to do. Since we're not auto-migrating old indexes, it'd help to have that error point people at vidxp index clear --yes and then reindex from existing media.

2. vidxp index status crashes on the incompatible snapshot. It validates by default, so you can't even see what state the index is in. Better to show an incompatible state with a clear-and-rebuild hint than to throw.

3. Docs are now stale. docs/local-api.md and INSTALLATION_GUIDE.md still say clear "publishes an empty active snapshot without deleting retained generations," which no longer matches what happens.

Optional extras: cleaning up the orphaned generation directories left behind from the old schema, and adding a CLI-level test for vidxp index clear --yes against an incompatible snapshot (the current tests exercise the repository/backend layers directly).

Net effect: new code → status says incompatible → clear gives the recovery path → reindex works again. Happy to pick these up if that's the direction you'd like.