Building with an unvendored duckdb library

[m-kuhn]

In QGIS we are shipping duckdb via gdal drivers and also as part of the python duckdb package.

When loading data via gdal functionality and when the geoparquet downloader plugin is active (which makes use of the duckdb python package) we get a crash.
The extract of the stack trace below shows that the python library is involved, even though it should not be involved in here.

My best guess is, that this is because of an ODR violation because duckdb is statically linked into the python package and also loaded via gdal.

Is it possible to build the python package linked against an already built (shared) duckdb library?

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   duckdb.cpython-312-darwin.so           0x30bcc01e8 duckdb::BitpackingScanState<long long, long long>::BitpackingScanState(duckdb::ColumnSegment&) + 80
1   duckdb.cpython-312-darwin.so           0x30bcbe784 duckdb::unique_ptr<duckdb::SegmentScanState, std::__1::default_delete<duckdb::SegmentScanState>, true> duckdb::BitpackingInitScan<long long>(duckdb::ColumnSegment&) + 44
2   libduckdb.dylib                        0x336c2a6c8 duckdb::ColumnData::BeginScanVectorInternal(duckdb::ColumnScanState&) + 156
3   libduckdb.dylib                        0x336c2a7a8 duckdb::ColumnData::ScanVector(duckdb::ColumnScanState&, duckdb::Vector&, unsigned long long, duckdb::ScanVectorType, unsigned long long) + 76
4   libduckdb.dylib                        0x336c64a98 duckdb::StandardColumnData::Scan(duckdb::TransactionData, unsigned long long, duckdb::ColumnScanState&, duckdb::Vector&, unsigned long long) + 124
5   libduckdb.dylib                        0x336c50bd0 void duckdb::RowGroup::TemplatedScan<(duckdb::TableScanType)0>(duckdb::TransactionData, duckdb::CollectionScanState&, duckdb::DataChunk&) + 2032

[evertlammerts]

The Python package always compiles the vendored engine and links it statically, in 2.0.0 and on main, and there is no option to link a shared libduckdb instead. We also export the engine's C++ symbols on purpose, for usable stack traces. It is likely that we will move the duckdb python package to the new C API v2 in the not too far off future (over the course of the v2 line, I suspect), when that stabilizes. We might support dynamic linking then as well, but we have to figure out the trade offs there first.