Skip to content

[Store] Add query-level filter system for metadata filtering #1820

@wachterjohannes

Description

@wachterjohannes

During the discussion and implementation of #1562 (query abstraction), a query-level filter system was part of the original design in #1570 but removed before merge to be addressed separately.

As @pkothmair pointed out, this feature is still missing.

Current State

  • QueryInterface is an empty marker interface — no filter support
  • VectorQuery, TextQuery, HybridQuery carry only search intent, no filter parameter
  • The existing Document\FilterInterface filters document streams during indexing — a different concept
  • Stores like Postgres, Pinecone, Qdrant, ChromaDB, etc. all support metadata filtering natively but there is no unified API for it

Proposal

Introduce a lightweight query-level filter system, starting with basic equality filters on document metadata. This was already discussed and prototyped in #1570 as EqualFilter.

Example Usage

use Symfony\AI\Store\Query\VectorQuery;
use Symfony\AI\Store\Query\Filter\EqualFilter;

$query = new VectorQuery($vector, new EqualFilter('locale', 'en'));
$results = $store->query($query, ['limit' => 10]);

$query = new TextQuery('space adventure', new EqualFilter('category', 'scifi'));
$results = $store->query($query, ['limit' => 5]);

Open Questions

  • Should filters live on the query object or in the $options array?
    In [Store] Open Store::query(Vector) method for different query types #1562, @Guikingone and @ahmed-bhs argued for keeping execution options (limit, minScore) in $options and search intent (filters) on the query — which aligns with the current query abstraction design.
  • Should we start with just EqualFilter or also include range/comparison filters?
  • How to handle stores that don't support filtering — throw UnsupportedFeatureException or silently ignore?

Related

cc @Guikingone @ahmed-bhs @pkothmair

Metadata

Metadata

Assignees

No one assigned

    Labels

    StoreIssues & PRs about the AI Store component

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions