Skip to content

Remote py apis - #2675

Draft
shivamka1 wants to merge 99 commits into
db_v4from
remote-py-apis
Draft

Remote py apis#2675
shivamka1 wants to merge 99 commits into
db_v4from
remote-py-apis

Conversation

@shivamka1

@shivamka1 shivamka1 commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Transport abstraction for the RemoteGraph client

Motivation

The existing Python RemoteGraph client is write-only, with each of ~15
mutation methods inlining its own Jinja template and JSON-parsing chain.
This PR consolidates that plumbing into a single Transport abstraction
that every operation flows through, and adds a minimal composable read
surface on top of the same seam so the client can now do lazy view
composition like rg.window(0, 10).node("ben").degree() end-to-end.

What shipped

  • New Transport trait in
    raphtory-graphql/src/client/transport.rs
    async fn execute(&self, op: &Op) -> Result<Option<Prop>, ClientError>.
  • Op { Read(ReadExpr) | Write(WriteOp) } in
    op.rs — read side is a
    recursive expression tree, write side is a flat enum of arg structs.
  • GraphqlTransport in
    graphql_transport.rs
    renders write ops via the existing Jinja templates (moved from client
    wrappers), walks read expressions into nested GraphQL queries, parses
    responses.
  • All 15 individual writes migrated through the transport, byte-identical
    wire behavior: 7 on RemoteGraph, 4 on RemoteNode, 4 on RemoteEdge.
    Each method body shrinks from ~30–50 lines to ~5–15 lines.
  • Minimal read surface: Root, Window, Node, Degree — enough for
    rg.window(s, e).node(id).degree() end-to-end. Only the terminal fires
    an RPC.
  • Renames: GraphQLRemote{Graph,Node,Edge}Remote*,
    RaphtoryGraphQLClientRemoteClient, raphtory_client.rs
    remote_client.rs. Python-facing names unchanged.
  • Cleanup: client field removed from RemoteNode/RemoteEdge — they
    now hold only transport and view state. Kept on RemoteGraph pending
    batch-method migration (see below).
  • Bug fix: PyRemoteGraph::node/edge now delegate to the Rust methods,
    so the accumulated view chain (window(...)) is correctly propagated
    when descending to a node — previously silently dropped.
  • Python bindings added for window() and degree().

Testing

  • 46 pre-existing raphtory-graphql tests still green.
  • 3 new Rust tests in graphql_transport.rs (unit tests for the read
    render/parse; one integration test spawning a real server).
  • 3 new Python tests in
    test_remote_graph_transport.py:
    test_add_and_degree, test_windowed_degree, test_view_chain_propagation.

TODO

  • Two Python-side batch mutations (add_nodes, add_edges) still call
    client.query(...) directly (existing TODOs). Migrating them removes
    the last client field.
  • Read surface expansion (layer, at, rolling, nodes, ids, name, count,
    properties, ...).
  • One hard-shape algorithm end-to-end (e.g. WCC) to lock in the
    return-type conversion pattern.
  • Iteration + paging for for n in g.nodes: ....
  • Small LLM eval to retire the product bet.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite Current: 2b624f9 Previous: 9823ef7 Ratio
lotr_graph/num_edges 4 ns/iter (± 0) 0 ns/iter (± 0) +∞
lotr_graph/num_nodes 5 ns/iter (± 0) 1 ns/iter (± 0) 5
lotr_graph/has_node_nonexisting 5 ns/iter (± 0) 2 ns/iter (± 0) 2.50
lotr_graph/graph_latest 3 ns/iter (± 0) 0 ns/iter (± 0) +∞
lotr_graph_materialise/materialize 8050631 ns/iter (± 40172) 1564816 ns/iter (± 35303) 5.14
lotr_graph_window_100/num_nodes 15 ns/iter (± 0) 5 ns/iter (± 0) 3
lotr_graph_window_100_materialise/materialize 8089774 ns/iter (± 64519) 1669150 ns/iter (± 10700) 4.85
lotr_graph_window_10/has_node_existing 143 ns/iter (± 9) 62 ns/iter (± 11) 2.31
lotr_graph_window_10_materialise/materialize 3345959 ns/iter (± 47014) 971980 ns/iter (± 4278) 3.44
lotr_graph_subgraph_10pc/has_node_nonexisting 5 ns/iter (± 0) 2 ns/iter (± 0) 2.50
lotr_graph_subgraph_10pc_materialise/materialize 2009904 ns/iter (± 22468) 334634 ns/iter (± 1287) 6.01
lotr_graph_subgraph_10pc_windowed/has_node_existing 149 ns/iter (± 9) 62 ns/iter (± 14) 2.40
lotr_graph_subgraph_10pc_windowed_materialise/materialize 1217703 ns/iter (± 11569) 230399 ns/iter (± 2617) 5.29
lotr_graph_window_50_layered/num_edges_temporal 155717 ns/iter (± 2232) 70121 ns/iter (± 7586) 2.22
lotr_graph_window_50_layered/has_node_existing 390 ns/iter (± 24) 129 ns/iter (± 12) 3.02
lotr_graph_window_50_layered/has_node_nonexisting 5 ns/iter (± 0) 2 ns/iter (± 0) 2.50
lotr_graph_window_50_layered/graph_latest 85415 ns/iter (± 2064) 36649 ns/iter (± 916) 2.33
lotr_graph_window_50_layered_materialise/materialize 30515575 ns/iter (± 95819) 3488825 ns/iter (± 24948) 8.75
lotr_graph_persistent_window_50_layered/num_edges_temporal 646047 ns/iter (± 24446) 192686 ns/iter (± 1569) 3.35
lotr_graph_persistent_window_50_layered/has_node_existing 442 ns/iter (± 435) 174 ns/iter (± 83) 2.54
lotr_graph_persistent_window_50_layered/has_node_nonexisting 5 ns/iter (± 0) 2 ns/iter (± 0) 2.50
lotr_graph_persistent_window_50_layered/iterate_exploded_edges 3480647 ns/iter (± 10244) 1659940 ns/iter (± 19402) 2.10
lotr_graph_persistent_window_50_layered/graph_latest 137704 ns/iter (± 4154) 57549 ns/iter (± 4809) 2.39
lotr_graph_persistent_window_50_layered_materialise/materialize 53184087 ns/iter (± 282815) 5298035 ns/iter (± 147912) 10.04

This comment was automatically generated by workflow using github-action-benchmark.

github-actions Bot and others added 20 commits July 3, 2026 22:37
…Degree/OutDegree/Name) with Python bindings + tests
…ypes, ExcludeNodes) + Valid, DefaultLayer, and graph Path/Namespace/Name terminals
shivamka1 and others added 30 commits July 27, 2026 12:09
…ates), reject non-finite floats, validate new_graph type
…) so .first().sum() doesn't run as .sum().first()
…efore tempdir teardown

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ne blocking_compute per nested read, not per source)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant