Releases: Query-farm/vgi-python
Release list
v0.20.0
Protocol 1.2.0: schema-qualified (schema, name) dispatch — the owning schema is carried on every by-name RPC (bind plus the unary aggregate and table-buffering calls), so a function name declared in two schemas of one catalog resolves to the implementation the caller named instead of mis-routing.
Result-cache key carries the owning schema, so two same-name functions in different schemas no longer cross-serve each other's cached results.
Breaking: requires the vgi 1.2.0 extension.
v0.19.0
Fixed
Cross-catalog function dispatch over HTTP. Two catalogs served by one worker process could resolve to the wrong implementation: SELECT b.main.test_same_name_catalog(1) returned twin_a:1. The call answered from the wrong catalog rather than failing.
MetaWorker distinguished sub-workers with a 4-byte prefix on the attach_opaque_data, but init()/bind() stripped that prefix before handing the request to the sub-worker — so the value persisted into an HTTP stream state carried no routing key. On rehydrate the resolver fell back to a registry scan and returned whichever sub-worker declared the name first.
The catalog name is now sealed into the attach plaintext at catalog_attach and read back on every dispatch. It sits inside the AEAD deliberately: the signing key is process-wide, so a routing header outside the seal would be client-editable and could steer one catalog's attach into another. Being inside also means it survives state serialization, so a rehydrate landing on a different instance routes correctly — no process-local map, and no positional index that need not agree across deploys.
Ambiguous dispatch now raises. When several sub-workers declare the same function name and the call carries no attach naming the catalog, MetaWorker refuses instead of returning the first declarer — turning a lost routing key into a plausible wrong answer is how this stayed invisible.
Upgrading
The attach_opaque_data envelope format changed, so an attach minted by 0.18.x is not routable by 0.19.0 and vice versa. Attaches are process-scoped, so an ordinary restart is unaffected. An HTTP state token held across a rolling upgrade will fail to route — loudly, rather than resolving to the wrong catalog.
CI
The integration matrix now runs the full sqllogictest suite on every transport. The launch lane previously ran only 2 of ~290 files, and the http lane was silently skipping 136 files after a SIGKILL fixture killed its shared worker — VGI_TEST_DEDICATED_WORKER is now confined to the subprocess lane. The shm lane moved onto the launcher, cutting it from ~14m to ~3m.
v0.18.0
Breaking
Function resolution is now keyed by (schema, name), not name alone.
A worker may register the same function name in more than one catalog schema. Previously the worker kept a flat name -> classes registry, so two such functions collided as overloads and any call raised Ambiguous function call — registration was schema-aware, but dispatch was not.
BindRequest gains schema_name, and the worker builds a second index keyed by (lowercased schema, function name). A schema-qualified lookup is exact, so example.main.f(x) and example.data.f(x) reach different implementations. Callers that send no schema search every schema and get a cross-schema ambiguity error naming the schemas involved. Functions declared via the legacy functions list register into the catalog's default_schema.
Client now requires schema_name. It is a required keyword on table_function, table_in_out_function, scalar_function, table_buffering_function, and table_scan_resumable, so a Client-driven bind is never ambiguous. vgi-client gains --schema (defaults to main).
Protocol version 1.0.0 → 1.1.0. Requires a VGI DuckDB extension built against 1.1.0; older clients are rejected at dispatch with a directional ProtocolVersionError.
Other changes
MetaWorkerroutesbind/initonattach_opaque_data, which is what identifies the catalog. Its no-attach fallback now prefers a sub-worker declaring the name in the requested schema rather than matching the bare name — previously a second catalog declaring the samemain.<fn>was unreachable.- The wire field stays nullable for
COPYhandler binds, which are advertised at catalog level and carry no schema. - Removed the
cache_interleavedfixture. It declaredsupports_batch_indexand then emitted descendingbatch_indexon a single stream — a protocol violation that only survived because the DuckDB extension's standalonevgi_table_function()path never wired that flag, so the monotonicity check never ran. That path has been removed from the extension. - New fixtures covering both collision shapes: one name across two schemas of a catalog, and one schema and name across two catalogs served by a single worker process.
v0.17.0
Minor release. Requires vgi-rpc >= 0.26.0, which changes what workers served through this package put on the wire.
Changed
HTTP response-codec negotiation is now client-authoritative. VGI's own X-VGI-Accept-Encoding outranks the generic Accept-Encoding. cpp-httplib (the DuckDB extension's HTTP client) injects Accept-Encoding: deflate, gzip, br, zstd — gzip before zstd — so walking that list first picked gzip and ignored the zstd-first order VGI states. gzip compression dominated large Arrow responses: 432ms vs ~40ms of zstd for 200MB of bodies.
identity is a first-class accept token. A client can now explicitly demand an uncompressed body; previously "no compression" was only reachable by accident, when nothing on offer happened to be producible. Content-Encoding: identity on a request passes through instead of 415ing.
VGI-Supported-Encodings is always advertised, including present-but-empty when compression is disabled. Absent and empty mean different things: absent is a legacy server (clients assume {zstd}), empty is a server positively stating it speaks no compression.
zstd level default drops 3 → 1, compression on by default. On Arrow IPC bodies level 1 measured 4.4x faster and smaller on hex-digest data, 2.7x faster and smaller on mixed numeric, and 1.4x faster for 2% more bytes on compressible text.
Fixed
- Lint gate:
ruff format/ import ordering for the compute-ladder fixtures added in 00aad32 (the embedded pytest-ruff plugin only coverstests/, notvgi/). - Regenerated the landing-conformance golden for the four new example-worker functions (
collatz_steps,hash_rounds,passthru,sha256_hex): 203 → 207 functions.
v0.16.1
Patch release.
Fixed
filter-pushdown: drop a fully-degraded AND instead of emitting WHERE () (#e3c50a6)
An AND whose children all failed to parse produced AndFilter(children=()), which rendered as (). Callers appending WHERE {clause} when the clause is truthy then built a syntactically invalid statement:
ParserException: syntax error at or near ")"
... "trade_canceled", "_partition_date" FROM src WHERE () LIMIT 50000
_parse_filter returns None for a join-key filter whose keys column cannot be resolved (graceful degradation — DuckDB filters client-side). Dropping such a child from an AND is safe on its own, but when every child dropped the conjunction was left empty. Triggering it required a query joining against a pushdown-enabled table where none of the dynamic join-key columns resolved, so it surfaced intermittently rather than reproducibly.
The OR branch already guarded this case; the AND branch now does too, returning None so the filter is reported unusable.
_filter_to_sql is additionally hardened against any other construction path: an empty AND renders as TRUE, an empty OR as FALSE — the correct identities, notably FALSE for OR so an empty disjunction cannot silently widen the result set.
v0.16.0
Adds the catalog_schema_contents RPC schema updates the current community vgi extension expects (published 0.15.0 predates them). Unblocks downstream workers (e.g. vgi-adbc) that attach over the latest extension.
v0.15.0
required_filters (AND-of-ORs)
Replaces the flat, AND-only Table.required_field_filter_paths with a unified
required_filters field in conjunctive normal form: an AND (outer list) of
OR-groups (inner lists) of dotted-path column references. A group is satisfied
when any one of its paths has a WHERE filter; every group must be satisfied.
required_filters=(("accession_number",), ("ticker", "cik"))
# → "accession_number AND one of (ticker, cik)"This expresses the common "ticker OR cik" requirement the old flat list could
not. Wire type is now list<list<utf8>>, kept as the trailing TableInfo
field so older extensions degrade gracefully to no-enforcement.
Breaking: clean rename, no backward compatibility. Pair with the matching
VGI DuckDB extension build that reads required_filters.
v0.14.0
HTTP result-cache conditional revalidation
Requires vgi-rpc >= 0.24.0, which surfaces the /init request metadata to a producer's first process() over HTTP. Together with the VGI extension's C++ change (validators on the /init request), result-cache conditional revalidation (304 / not_modified) now works over both transports, not just subprocess.
vgi-rpc>=0.24.0;cache/revalidate.testnow runs on the http lane too (ci skip removed).- No worker code change:
TableProducerState.process()already reads the validators.
Verified: revalidate.test passes over HTTP against published vgi-rpc 0.24.0.
v0.13.0
Per-schema doc in describe.json; refreshed shared landing page v3 (Markdown tables, schema/function descriptions on expand, two-tier green/blue Cupola CTA, INSTALL vgi FROM community line, footer copyright).
v0.12.0
Catalog macros on the landing page + landing-page fixes
- describe.json now surfaces catalog macros (scalar + table) in the schema
functionslist — fixes workers whose callable functions are declared as DuckDB macros (e.g. vgi-volcanos) showing 0 functions. Defaulted macro params render as named args with their default; untyped params showANY. - Landing page: Markdown rendering for catalog/table/view docs; Arrow→DuckDB column/argument type display (
int64→BIGINT,dictionary<…>→VARCHAR); table/view descriptions shown only on expand; footer schema-version line removed.