Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 50 additions & 14 deletions python/packages/oracle/README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,66 @@
# Agent Framework Oracle vector store

An alpha connector for storing and searching native `VECTOR` columns in Oracle
Database 23ai or newer. `OracleCollection` implements async batch CRUD and
vector search; `OracleStore` shares a client across collections; `OracleSettings`
resolves connection configuration through Agent Framework.
AI Database 26ai. `OracleCollection` implements async batch CRUD and vector
search; `OracleStore` shares a client across collections; `OracleSettings`
describes the connection and pool settings resolved through Agent Framework.

## Installation and setup

```bash
pip install agent-framework-oracle --pre
```

Requires Python 3.10+, Oracle Database 23ai or newer with `COMPATIBLE` set to
Requires Python 3.10+, Oracle AI Database 26ai with `COMPATIBLE` set to
**23.4.0 or higher**, `python-oracledb` 2.2.x or 3.x, and permission to
create/drop tables in the connected user's schema. The connector uses the
driver's async Thin-mode API; it does not initialize the Thick client or
provision a database.
It does not create or alter schemas, vector indexes, or existing tables.

Set `ORACLE_DSN` (for example `localhost:1521/FREEPDB1`), `ORACLE_USER`, and
`ORACLE_PASSWORD`, or pass `dsn`, `user`, and `password` to either constructor.
Credentials are resolved in order: **explicit argument > selected `.env` file
> process environment**. A `.env` file is read only when `env_file_path` is
specified; `env_file_encoding` overrides UTF-8. `password` accepts `str` or
Agent Framework `SecretString` and is unwrapped only when opening a connection.
Missing or empty credentials are errors.
For username/password authentication, set `ORACLE_DSN` (for example
`localhost:1521/FREEPDB1`), `ORACLE_USER`, and `ORACLE_PASSWORD`, or pass
`dsn`, `user`, and `password` to either constructor. Typed driver settings can
also be supplied with the `pool_parameters` mapping. Settings are resolved in
order: **named `dsn`/`user`/`password` argument > `pool_parameters` > selected
`.env` file > process environment > connector pool defaults**. A `.env` file is
read only when `env_file_path` is specified; the selected file must exist, and
`env_file_encoding` overrides UTF-8. `password` accepts `str` or Agent Framework
`SecretString` and is unwrapped only when opening a connection.

Named connection values must be nonempty when supplied, but their presence is
not required by the connector. Authentication and connection-descriptor
completeness are validated by python-oracledb, allowing alternatives such as
`access_token`, `externalauth`, or `host` with `service_name`/`sid`.

Explicit `pool_parameters` keys are forwarded when they are accepted by the
installed `python-oracledb` version, including newly added driver options not
yet declared by the open `OracleSettings` typed dictionary. This preserves
editor discovery for known options while allowing forward-compatible keys.
Unsupported names and typos raise an error; unrecognized `ORACLE_` environment
variables remain ignored.

Recognized scalar `oracledb.create_pool_async()` options can also be supplied as
`ORACLE_` settings. The prefix is removed, the remainder is converted to the
driver's lowercase keyword name, and boolean, integer, and floating-point
values are converted before the pool is created. Password-like values remain
masked until they are passed to the driver. For example:

```dotenv
ORACLE_DSN=localhost:1521/FREEPDB1
ORACLE_USER=example
ORACLE_PASSWORD=database-password
ORACLE_WALLET_LOCATION=/path/to/wallet
ORACLE_WALLET_PASSWORD=wallet-password
ORACLE_MIN=1
ORACLE_MAX=8
ORACLE_HOMOGENEOUS=false
```

The connector uses `min=0` and `max=4` when those options are not configured.
Unrecognized `ORACLE_` variables are ignored. Options that require callbacks,
classes, collections, or SSL context objects are not loaded from text settings;
pass them through `pool_parameters` or use a configured client.

Alternatively, pass a configured `oracledb.AsyncConnection` or
`oracledb.AsyncConnectionPool` as `client` for advanced authentication. A
Expand Down Expand Up @@ -130,8 +166,8 @@ creates and deletes a unique test table.

The unit tests require no Oracle server. To run the live integration test,
provide **all three** `ORACLE_TEST_DSN`, `ORACLE_TEST_USER`, and
`ORACLE_TEST_PASSWORD` for an explicitly designated disposable Oracle 23ai+
schema where the user can create and drop tables:
`ORACLE_TEST_PASSWORD` for an explicitly designated disposable Oracle AI
Database 26ai schema where the user can create and drop tables:

```bash
cd python
Expand All @@ -145,6 +181,6 @@ deletes only that table.

## Documentation

- [Oracle Database AI Vector Search requirements](https://docs.oracle.com/en/database/oracle/oracle-database/26/vecse/overview-ai-vector-search.html)
- [Oracle AI Vector Search requirements](https://docs.oracle.com/en/database/oracle/oracle-database/26/vecse/overview-ai-vector-search.html)
- [python-oracledb 2.2 vector types](https://python-oracledb.readthedocs.io/en/v2.2.0/user_guide/vector_data_type.html)
- [Microsoft Agent Framework](https://learn.microsoft.com/agent-framework/)
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Copyright (c) Microsoft. All rights reserved.

"""Oracle Database 23ai native-vector collections and stores."""
"""Oracle AI Database 26ai native-vector collections and stores."""

from __future__ import annotations

Expand Down
158 changes: 133 additions & 25 deletions python/packages/oracle/agent_framework_oracle/_vector_store.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
# Copyright (c) Microsoft. All rights reserved.

"""Oracle Database 23ai vector collections and stores."""
"""Oracle AI Database 26ai vector collections and stores."""

from __future__ import annotations

import asyncio
import math
from array import array
from collections.abc import AsyncGenerator, Collection, Mapping, Sequence
from collections.abc import AsyncGenerator, Callable, Collection, Mapping, Sequence
from contextlib import asynccontextmanager
from decimal import Decimal
from inspect import signature
from typing import Any, ClassVar, Generic, Protocol, TypeAlias, cast
from uuid import UUID

Expand Down Expand Up @@ -64,12 +65,74 @@ def setinputsizes(self, **kwargs: Any) -> Any: ...
_ORDER_OPERATORS = {"gt": ">", "gte": ">=", "lt": "<", "lte": "<="}


class OracleSettings(TypedDict, total=False):
"""Connection settings resolved from arguments, a selected .env file, or ``ORACLE_`` variables."""
class OracleSettings(TypedDict, total=False, extra_items=Any):
"""Connection and pool settings resolved from arguments, a selected .env file, or ``ORACLE_`` variables."""

dsn: str | None
user: str | None
password: SecretString | None
pool_alias: str | None
min: int | None
max: int | None
increment: int | None
getmode: int | None
homogeneous: bool | None
timeout: int | None
wait_timeout: int | None
max_lifetime_session: int | None
max_sessions_per_shard: int | None
soda_metadata_cache: bool | None
ping_interval: int | None
ping_timeout: int | None
proxy_user: str | None
newpassword: SecretString | None
wallet_password: SecretString | None
access_token: SecretString | tuple[str, str] | Callable[[], str | tuple[str, str]] | None
host: str | None
Comment thread
eavanvalkenburg marked this conversation as resolved.
port: int | None
protocol: str | None
https_proxy: str | None
https_proxy_port: int | None
service_name: str | None
instance_name: str | None
sid: str | None
server_type: str | None
cclass: str | None
purity: int | None
expire_time: int | None
retry_count: int | None
retry_delay: int | None
tcp_connect_timeout: float | None
ssl_server_dn_match: bool | None
ssl_server_cert_dn: str | None
wallet_location: str | None
events: bool | None
externalauth: bool | None
mode: int | None
disable_oob: bool | None
stmtcachesize: int | None
edition: str | None
tag: str | None
matchanytag: bool | None
config_dir: str | None
debug_jdwp: str | None
connection_id_prefix: str | None
sdu: int | None
pool_boundary: str | None
use_tcp_fast_open: bool | None
program: str | None
machine: str | None
terminal: str | None
osuser: str | None
driver_name: str | None
use_sni: bool | None
thick_mode_dsn_passthrough: bool | None
pool_name: str | None


_CREDENTIAL_SETTING_NAMES = {"dsn", "user", "password"}
_ORACLE_SETTING_NAMES = frozenset(OracleSettings.__annotations__)
_CREATE_POOL_PARAMETER_NAMES = frozenset(signature(oracledb.create_pool_async).parameters)


def _quote_identifier(name: str) -> str:
Expand Down Expand Up @@ -218,24 +281,26 @@ def __init__(
dsn: str | None = None,
user: str | None = None,
password: SecretString | None = None,
pool_parameters: Mapping[str, Any] | None = None,
client: OracleClient | None = None,
) -> None:
if client is not None:
if not isinstance(client, (oracledb.AsyncConnection, oracledb.AsyncConnectionPool)):
raise TypeError("client must be an oracledb AsyncConnection or AsyncConnectionPool.")
if any(value is not None for value in (dsn, user, password)):
if any(value is not None for value in (dsn, user, password)) or pool_parameters:
raise ValueError("An Oracle client cannot be combined with connection settings.")
else:
if not isinstance(dsn, str) or not dsn.strip():
raise ValueError("A nonempty Oracle DSN is required.")
if not isinstance(user, str) or not user.strip():
raise ValueError("A nonempty Oracle user is required.")
if not isinstance(password, SecretString) or not password.get_secret_value():
raise ValueError("A nonempty Oracle password is required.")
if dsn is not None and (not isinstance(dsn, str) or not dsn.strip()):
raise ValueError("Oracle dsn must be a nonempty string when provided.")
if user is not None and (not isinstance(user, str) or not user.strip()):
raise ValueError("Oracle user must be a nonempty string when provided.")
if password is not None and (not isinstance(password, SecretString) or not password.get_secret_value()):
raise ValueError("Oracle password must be a nonempty string when provided.")
self._client = client
self._dsn = dsn
self._user = user
self._password = password
self._pool_parameters = dict(pool_parameters or {})
self._pool: oracledb.AsyncConnectionPool | None = None
self._lock = asyncio.Lock()
self._closed = False
Expand All @@ -251,11 +316,21 @@ async def _get_client(self) -> OracleClient:
async with self._lock:
self.ensure_open()
if self._pool is None:
if self._user is None or self._dsn is None or self._password is None:
raise RuntimeError("Oracle connection settings are missing.")
self._pool = oracledb.create_pool_async(
user=self._user, password=self._password.get_secret_value(), dsn=self._dsn, min=0, max=4
)
named_parameters = {
name: value.get_secret_value() if isinstance(value, SecretString) else value
for name, value in (("user", self._user), ("password", self._password), ("dsn", self._dsn))
if value is not None
}
pool_parameters = {
"min": 0,
"max": 4,
**{
name: value.get_secret_value() if isinstance(value, SecretString) else value
for name, value in self._pool_parameters.items()
},
**named_parameters,
}
self._pool = oracledb.create_pool_async(**pool_parameters)
return self._pool

@asynccontextmanager
Expand Down Expand Up @@ -292,24 +367,44 @@ def _create_client(
dsn: str | None,
user: str | None,
password: str | SecretString | None,
pool_parameters: OracleSettings | None,
client: OracleClient | None,
env_file_path: str | None,
env_file_encoding: str | None,
) -> _Client:
if client is not None:
if any(value is not None for value in (dsn, user, password, env_file_path, env_file_encoding)):
raise ValueError("client cannot be combined with dsn, user, password, or .env settings.")
if any(value is not None for value in (dsn, user, password, pool_parameters, env_file_path, env_file_encoding)):
raise ValueError("client cannot be combined with dsn, user, password, pool_parameters, or .env settings.")
return _Client(client=client)
supplied_pool_parameters = dict(pool_parameters or {})
overrides: dict[str, Any] = {
name: value for name, value in supplied_pool_parameters.items() if name in _ORACLE_SETTING_NAMES
}
overrides.update({
name: value for name, value in (("dsn", dsn), ("user", user), ("password", password)) if value is not None
})
settings = load_settings(
OracleSettings,
env_prefix="ORACLE_",
dsn=dsn,
user=user,
password=password,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
**overrides,
)
resolved_pool_parameters: dict[str, Any] = {
name: value for name, value in settings.items() if name not in _CREDENTIAL_SETTING_NAMES and value is not None
}
resolved_pool_parameters.update({
name: value for name, value in supplied_pool_parameters.items() if name not in _ORACLE_SETTING_NAMES
})
if unsupported := resolved_pool_parameters.keys() - _CREATE_POOL_PARAMETER_NAMES:
names = ", ".join(sorted(unsupported))
raise ValueError(f"Oracle pool setting(s) not supported by the installed python-oracledb version: {names}.")
return _Client(
dsn=settings.get("dsn"),
user=settings.get("user"),
password=settings.get("password"),
pool_parameters=resolved_pool_parameters,
)
return _Client(dsn=settings.get("dsn"), user=settings.get("user"), password=settings.get("password"))


def _scalar_field(definition: VectorStoreCollectionDefinition, name: str) -> VectorStoreField:
Expand Down Expand Up @@ -445,7 +540,7 @@ class OracleCollection(
BaseVectorSearch[KeyT, ModelT],
Generic[KeyT, ModelT],
):
"""Store typed records and search Oracle Database 23ai native VECTOR columns."""
"""Store typed records and search Oracle AI Database 26ai native VECTOR columns."""

supported_key_types: ClassVar[set[str] | None] = {"str", "int", "UUID"}
supported_vector_types: ClassVar[set[str] | None] = {"float", "float32", "float64", "int8"}
Expand All @@ -461,12 +556,13 @@ def __init__(
dsn: str | None = None,
user: str | None = None,
password: str | SecretString | None = None,
pool_parameters: OracleSettings | None = None,
client: OracleClient | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
_shared_client: _Client | None = None,
) -> None:
"""Initialize a collection with explicit, selected .env, or environment credentials.
"""Initialize a collection with explicit, .env, or environment connection settings.

Args:
record_type: Registered application model, or dict with an explicit definition.
Expand All @@ -476,12 +572,17 @@ def __init__(
dsn: Oracle connect string; defaults to ``ORACLE_DSN``.
user: Oracle database user; defaults to ``ORACLE_USER``.
password: Database password or AF SecretString; defaults to ``ORACLE_PASSWORD``.
pool_parameters: Additional ``create_pool_async`` settings, including options supported by the installed
driver but not yet declared by ``OracleSettings``. Named ``dsn``, ``user``, and ``password`` arguments
take precedence, followed by this mapping, the selected .env file, process environment, and connector
pool defaults.
client: Borrowed async connection or pool in place of connection settings.
env_file_path: Optional selected .env file; no implicit discovery.
env_file_encoding: Encoding of the selected .env file.
"""
if _shared_client is not None and any(
item is not None for item in (dsn, user, password, client, env_file_path, env_file_encoding)
item is not None
for item in (dsn, user, password, pool_parameters, client, env_file_path, env_file_encoding)
):
raise ValueError("Shared Oracle clients cannot be combined with connection settings.")
super().__init__(
Expand All @@ -497,6 +598,7 @@ def __init__(
dsn=dsn,
user=user,
password=password,
pool_parameters=pool_parameters,
client=client,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
Expand Down Expand Up @@ -874,6 +976,7 @@ def __init__(
dsn: str | None = None,
user: str | None = None,
password: str | SecretString | None = None,
pool_parameters: OracleSettings | None = None,
client: OracleClient | None = None,
embedding_generator: EmbeddingClient | None = None,
env_file_path: str | None = None,
Expand All @@ -885,6 +988,10 @@ def __init__(
dsn: Oracle connect string; defaults to ``ORACLE_DSN``.
user: Oracle database user; defaults to ``ORACLE_USER``.
password: Database password or AF SecretString; defaults to ``ORACLE_PASSWORD``.
pool_parameters: Additional ``create_pool_async`` settings, including options supported by the installed
driver but not yet declared by ``OracleSettings``. Named ``dsn``, ``user``, and ``password`` arguments
take precedence, followed by this mapping, the selected .env file, process environment, and connector
pool defaults.
client: Borrowed async connection or pool in place of connection settings.
embedding_generator: Default local embedding client for child collections.
env_file_path: Optional selected .env file; no implicit discovery.
Expand All @@ -895,6 +1002,7 @@ def __init__(
dsn=dsn,
user=user,
password=password,
pool_parameters=pool_parameters,
client=client,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
Expand Down
Loading
Loading