Expose minDistance bindings#20
Closed
estebanzimanyi wants to merge 12 commits into
Closed
Conversation
Replaces the regex-over-builder/meos.h parser in build_pymeos_functions.py with a JSON loader that consumes the IDL produced by MEOS-API. build_header.py keeps producing builder/meos.h for CFFI cdef compilation; only the wrapper-generation step now reads the structured catalog. Picks up 13 functions the regex silently dropped (meos_error, tfloat_avg_value, geo_get_srid, geog_from_binary, meos_basetype + family, temptype_subtype/_all, mfjson skiplist_make) and normalises meos_set_spatial_ref_sys_csv's argument from a raw const char* cast to an encoded str. json_object-bearing signatures are filtered out to match build_header.py's undefined-types skip.
build_pymeos_functions.py used to carry two hardcoded sets enumerating which params are extra Python returns (22 entries) and which accept None (52 entries). Both sets are now populated from the IDL's shape field, which MEOS-API ships in meta/meos-meta.json so every binding reads the same catalog. result_parameters stays local because the override (out-param replaces the bool return when truthy) is PyMEOS-CFFI ergonomic, not a fact about the C API. Re-vendoring the IDL also picks up the missing outputArrays entries for tpoint_as_mvtgeom, which previously took gsarr / timesarr as Python lists rather than as out-parameters; the regenerated wrapper now returns (result, gsarr, timesarr).
The previous build_pymeos_functions.py output_parameters set was missing tpoint_as_mvtgeom's gsarr and timesarr entries, which made the wrapper take both as user-facing list arguments rather than as output parameters; the shape-driven consolidation corrects that and the regenerated wrapper now returns (result, gsarr, timesarr). The smoke test now reflects on the wrapper's signature with inspect.signature so the gsarr / timesarr parameters are asserted absent from the user-facing arglist. Also adds an explicit meos_initialize(None) call to exercise the shape.nullable path for tz_str.
The previous attempt called meos_initialize(None) and immediately meos_finalize() to exercise shape.nullable for tz_str; the double-init / finalize cycle aborts with SIGABRT (exit 134) on Linux, which masks the actual codegen signal. Move the nullable check to inspect.signature so the annotation is verified statically (the parameter must accept None) and the live MEOS lifecycle stays a single happy-path init/finalize. tpoint_as_mvtgeom's output-parameter regression guard is unchanged.
Vendor the MEOS 1.4 IDL (3544 functions, +1149 over 1.3) and regenerate pymeos_cffi/functions.py plus builder/meos.h. Add the three new public headers (meos_cbuffer.h, meos_pose.h, meos_rgeo.h) to the builder lists. Restore the GEOSContextHandle_t entry in undefined_types so the cdef strips geos_get_context, which is exported by libmeos but its handle type is forward-declared opaque. The IDL was generated by MEOS-API against MobilityDB master (HEAD dd4ccd3) with the shape metadata catalog merged in. Version bumped to 1.4.0a1.
CI was failing the smoke import with 'undefined symbol: temptype_continuous' because the regen used a stale libmeos.so for the symbol filter. Re-ran build_header.py against a fresh master MEOS build with -DCBUFFER=ON -DPOSE=ON so the filter sees the renamed temptype_supports_linear and the new cbuffer/pose/rgeo symbols. Also include the three new headers (meos_cbuffer.h, meos_pose.h, meos_rgeo.h) in build_pymeos.py's #include preamble so the cffi compile sees the prototypes from each module, and enable -DCBUFFER=ON -DPOSE=ON in pr_build.yml so CI builds a libmeos that exports the cbuffer/pose/rgeo functions referenced by the cdef.
Two fixes that surface PyMEOS test failures earlier:
1. The previous IDL was generated against the MobilityDB source tree
which pulls in duplicate GSERIALIZED definitions from
build/postgis/liblwgeom/liblwgeom.h. Libclang ended up canonicalising
the geometry pointer to const int * for ~400 functions, so the cffi
casts would emit at the wrong type. Regenerating against the clean
/usr/local/include tree restores const GSERIALIZED * everywhere.
2. _load_shape_pairs only added a function's count parameter to
output_parameters when the shape carried an outputArrays list. For
plain T *foo(..., int *count) cases (stbox_quad_split) the
arrayReturn.lengthFrom={kind:param,name:count} entry is enough; lift
the count detection out of the outputArrays loop so it applies
regardless. stbox_quad_split now returns (STBox *, int) instead of
taking count as an input.
The functions.py shrinkage (~3200 lines) comes from collapsing
duplicate Datum-shape suffix variants whose tail was the same as their
Datum-form base. ~3500 line growth from the GSERIALIZED rewrap (extra
casts for the corrected pointer type).
Three additions that let downstream wrappers (PyMEOS in particular) talk to MEOS 1.4 without dropping down to raw cffi: - float_to_datum / int_to_datum: Python-side equivalents of MEOS's Float8GetDatum / Int32GetDatum. They use _ffi.new + reinterpret-cast to assemble a Datum without requiring the helpers to be exported from libmeos. - tbox_expand_float / tbox_expand_int / tbox_shift_scale_float / tbox_shift_scale_int: thin wrappers around the underlying tbox_expand_value and tbox_shift_scale_value that handle the Datum/MeosType boilerplate. MEOS 1.4 collapsed the per-base-type variants into the Datum-shaped form; these helpers re-expose the typed call shape that PyMEOS expects. - The is_output_parameter heuristic for trailing int *count was comparing against the rendered annotation string ending in '*' but the codegen now emits Annotated[_ffi.CData, 'int *'] which ends in ']. Switched the check to the raw ctype so tbool_values / tfloat_values / tint_values / tgeo_values etc. all return (array, count) tuples instead of forcing the count as an input.
For void returns and the non-bool result_param path, the C return value was assigned to a local that ruff F841'd. Skip the assignment in those cases and have mi_span_span's modifier reintroduce it (it needs both out_result and the C-side count as a tuple). After the regen ruff check pymeos_cffi/ builder/ now passes with zero errors.
Adds mindistance_tgeo_tgeo and tgeoarr_tgeoarr_mindist to the cffi cdef header, the IDL, the generated functions wrappers, and the __all__ export list, following the nad_tgeo_tgeo pattern. These are the MEOS spatial-min kernels from MobilityDB #1007; the build CI compiles against MobilityDB master where the symbols are defined. The functions.py block is the verbatim codegen output ruff-formatted to the committed style.
estebanzimanyi
force-pushed
the
bump/meos-1.4
branch
from
a4f3dbe to
7421d77
Compare
Member
Author
|
Subsumed by #19 ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds mindistance_tgeo_tgeo and tgeoarr_tgeoarr_mindist to the cffi cdef header, the IDL, the generated functions wrappers, and the all export list, following the nad_tgeo_tgeo pattern. These are the MEOS spatial-min kernels from MobilityDB #1007; the build CI compiles against MobilityDB master where the symbols are defined, and the functions.py block is the verbatim codegen output ruff-formatted to the committed style. Stacked on the MEOS 1.4 bump.