Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions docs/features/vector-stores-and-embeddings/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ This feature ports the vector store abstractions, embedding generator abstractio
| Vector search | Unified search interface with `search_type` parameter (`"vector"`, `"keyword_hybrid"`) |
| Data model decorator | `@vectorstoremodel` decorator for defining vector store data models (supports Pydantic, dataclasses, plain classes, dicts) |
| Agent tools | `create_vector_search_tool`, `create_upsert_tool`, `create_get_tool`, `create_delete_tool` for agent-usable vector store operations |
| Agent providers | `VectorStoreHistoryProvider` for provider-owned scoped history and `VectorCollectionContextProvider` for caller-owned collections |
| In-memory store | Zero-dependency vector store for testing and development |
| 13+ connectors | Azure AI Search, Qdrant, Redis, PostgreSQL, MongoDB, Cosmos DB, Pinecone, Chroma, Weaviate, Oracle, SQL Server, FAISS |

Expand Down Expand Up @@ -130,6 +131,11 @@ Options considered:
| `SupportsVectorUpsert` | `_vectors.py` | Protocol for collection CRUD |
| `SupportsVectorSearch` | `_vectors.py` | Protocol for vector search |
| `create_vector_search_tool` | `_vectors.py` | Creates AF `FunctionTool` from vector search |
| `create_upsert_tool` | `_vectors.py` | Creates an agent tool for record upserts |
| `create_get_tool` | `_vectors.py` | Creates an agent tool for key-based retrieval |
| `create_delete_tool` | `_vectors.py` | Creates an agent tool for key-based deletion |
| `VectorStoreHistoryProvider` | `_vectors.py` | Stores scoped full history with optional compacted loading and search |
| `VectorCollectionContextProvider` | `_vectors.py` | Adds CRUD and one or more search tools for a caller-owned collection |

## Source Files Reference (SK → AF mapping)

Expand Down Expand Up @@ -437,7 +443,7 @@ Each connector follows the AF package structure:
---

### Phase 8: Vector Store CRUD Tools
**Goal:** Provide a full set of agent-usable tools for CRUD operations on vector store collections.
**Goal:** Provide agent-usable CRUD tools plus generic history and context providers for vector stores.
**Mergeable:** Yes — adds tools without changing existing APIs.

#### 8.1 — `create_upsert_tool` — tool for upserting records into a collection
Expand All @@ -447,6 +453,8 @@ Each connector follows the AF package structure:
- Consider if this overlaps with filtered search and document when to use which
#### 8.3 — `create_delete_tool` — tool for deleting records by key
#### 8.4 — Tests and samples for CRUD tools
#### 8.5 — `VectorStoreHistoryProvider` — provider-owned scoped history with optional embeddings, compaction, and search
#### 8.6 — `VectorCollectionContextProvider` — configurable tools and instructions for a caller-owned collection

---

Expand Down Expand Up @@ -502,9 +510,18 @@ Each connector follows the AF package structure:
- `create_upsert_tool(...)` → tool for upserting records
- `create_get_tool(...)` → tool for retrieving records by key
- `create_delete_tool(...)` → tool for deleting records
- These are separate from search and are placed in a later phase

10. **Score threshold filtering**: Scoring, filter execution, score thresholds, and paging belong to the connector
- Upsert and delete require approval by default; get is read-only by default

10. **Generic vector providers**:
- `VectorStoreHistoryProvider` takes a store because it owns the history model and collection. Every operation
is filtered by application, optional tenant/agent, provider source, and session. Compaction changes only the
messages loaded into model context; optional search still queries the full scoped transcript. Content arrays
can be stored as JSON text or msgspec-encoded MessagePack bytes to match the backing store.
- `VectorCollectionContextProvider` takes a caller-owned collection and adds generated instructions plus
configurable CRUD/search tools. Additional `create_vector_search_tool` instances can expose different filters
and result mappings.

11. **Score threshold filtering**: Scoring, filter execution, score thresholds, and paging belong to the connector
and backing store (ref: [SK .NET PR #13501](https://github.com/microsoft/semantic-kernel/pull/13501)). Core passes
`score_threshold` through without requiring a known distance function or an explicit metric and does not
post-filter returned results, including results without scores. Each connector defines its score units,
Expand Down
4 changes: 3 additions & 1 deletion docs/specs/feature-usage-bit-registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,9 @@ only to approved first-party endpoints.
| 17 | `core.session_store` | Agent session store | `agent_framework.SessionStore` / `FileSessionStore` |
| 18 | `core.agent_hooks` | Agent Hooks middleware | `agent_framework.create_agent_hooks_middleware` |
| 19 | `core.vector_stores` | Vector store abstractions | `BaseVectorCollection` / `BaseVectorSearch` operations |
| 20–31 | _reserved_ | core growth | — |
| 20 | `core.vector_store_history_provider` | Vector-store history provider | `agent_framework.VectorStoreHistoryProvider` |
| 21 | `core.vector_collection_context_provider` | Vector-collection context provider | `agent_framework.VectorCollectionContextProvider` |
| 22–31 | _reserved_ | core growth | — |
| 32 | `orchestration.sequential` | Sequential orchestration | `agent_framework_orchestrations.SequentialBuilder` |
| 33 | `orchestration.concurrent` | Concurrent orchestration | `agent_framework_orchestrations.ConcurrentBuilder` |
| 34 | `orchestration.group_chat` | Group-chat orchestration | `agent_framework_orchestrations.GroupChatBuilder` |
Expand Down
6 changes: 6 additions & 0 deletions python/packages/core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,12 @@ The vector store API is experimental under the shared `VECTOR_STORES` feature ID
filter execution, score thresholds (including provider-defined/default metrics), and paging. Use native backend
execution where available, otherwise an explicit connector-local fallback or reject unsupported options
- **`create_vector_search_tool`** - Creates an agent tool from any `SupportsVectorSearch` implementation
- **`create_upsert_tool` / `create_get_tool` / `create_delete_tool`** - Create agent tools for collection CRUD;
upsert and delete require approval by default, while get does not
- **`VectorStoreHistoryProvider`** - Stores full scoped conversation history in a provider-owned collection;
optional embeddings enable session-scoped history search, while optional compaction affects only loaded context
- **`VectorCollectionContextProvider`** - Adds instructions and configurable CRUD/search tools for a caller-owned
collection, including independently configured additional search tools
- **`InMemoryCollection` / `InMemoryStore`** - Dependency-free, process-local development and test implementation;
cosine scoring scales finite inputs, all metrics reject non-finite scores, and unsupported distance functions
fail before record scanning. Hamming scores/thresholds use the fraction of unequal dimensions, not a count.
Expand Down
10 changes: 10 additions & 0 deletions python/packages/core/agent_framework/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,8 +310,13 @@
"SearchType",
"SupportsVectorSearch",
"SupportsVectorUpsert",
"VectorCollectionContextProvider",
"VectorStoreCollectionDefinition",
"VectorStoreField",
"VectorStoreHistoryProvider",
"create_delete_tool",
"create_get_tool",
"create_upsert_tool",
"create_vector_search_tool",
"register_vectorstoremodel",
"vectorstoremodel",
Expand Down Expand Up @@ -633,8 +638,10 @@
"UsageDetails",
"UserInputRequiredException",
"ValidationTypeEnum",
"VectorCollectionContextProvider",
"VectorStoreCollectionDefinition",
"VectorStoreField",
"VectorStoreHistoryProvider",
"Workflow",
"WorkflowAgent",
"WorkflowBuilder",
Expand Down Expand Up @@ -667,8 +674,11 @@
"create_agent_hooks_middleware_from_emitter",
"create_always_approve_tool_response",
"create_always_approve_tool_with_arguments_response",
"create_delete_tool",
"create_edge_runner",
"create_get_tool",
"create_harness_agent",
"create_upsert_tool",
"create_vector_search_tool",
"detect_media_type_from_base64",
"enqueue_messages",
Expand Down
10 changes: 10 additions & 0 deletions python/packages/core/agent_framework/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -266,8 +266,13 @@ from ._vectors import (
SearchType,
SupportsVectorSearch,
SupportsVectorUpsert,
VectorCollectionContextProvider,
VectorStoreCollectionDefinition,
VectorStoreField,
VectorStoreHistoryProvider,
create_delete_tool,
create_get_tool,
create_upsert_tool,
create_vector_search_tool,
register_vectorstoremodel,
vectorstoremodel,
Expand Down Expand Up @@ -588,8 +593,10 @@ __all__ = [
"UsageDetails",
"UserInputRequiredException",
"ValidationTypeEnum",
"VectorCollectionContextProvider",
"VectorStoreCollectionDefinition",
"VectorStoreField",
"VectorStoreHistoryProvider",
"Workflow",
"WorkflowAgent",
"WorkflowBuilder",
Expand Down Expand Up @@ -622,8 +629,11 @@ __all__ = [
"create_agent_hooks_middleware_from_emitter",
"create_always_approve_tool_response",
"create_always_approve_tool_with_arguments_response",
"create_delete_tool",
"create_edge_runner",
"create_get_tool",
"create_harness_agent",
"create_upsert_tool",
"create_vector_search_tool",
"detect_media_type_from_base64",
"enqueue_messages",
Expand Down
2 changes: 2 additions & 0 deletions python/packages/core/agent_framework/_telemetry.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ class FeatureIndex(IntEnum):
CORE_SESSION_STORE = 17
CORE_AGENT_HOOKS = 18
CORE_VECTOR_STORES = 19
CORE_VECTOR_STORE_HISTORY_PROVIDER = 20
CORE_VECTOR_COLLECTION_CONTEXT_PROVIDER = 21


# This environment variable is reserved by the Foundry hosting environment to
Expand Down
Loading
Loading