Remote py apis - #2675
Draft
shivamka1 wants to merge 99 commits into
Draft
Conversation
…ree read wired end-to-end
…ata, update_metadata, delete_edge) through Transport
…date_metadata) through Transport
…tadata) through Transport
…aph.node/edge to Rust methods (fixes view-chain preservation in Python)
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
| Benchmark suite | Current: 2b624f9 | Previous: 9823ef7 | Ratio |
|---|---|---|---|
lotr_graph/num_edges |
4 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph/num_nodes |
5 ns/iter (± 0) |
1 ns/iter (± 0) |
5 |
lotr_graph/has_node_nonexisting |
5 ns/iter (± 0) |
2 ns/iter (± 0) |
2.50 |
lotr_graph/graph_latest |
3 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph_materialise/materialize |
8050631 ns/iter (± 40172) |
1564816 ns/iter (± 35303) |
5.14 |
lotr_graph_window_100/num_nodes |
15 ns/iter (± 0) |
5 ns/iter (± 0) |
3 |
lotr_graph_window_100_materialise/materialize |
8089774 ns/iter (± 64519) |
1669150 ns/iter (± 10700) |
4.85 |
lotr_graph_window_10/has_node_existing |
143 ns/iter (± 9) |
62 ns/iter (± 11) |
2.31 |
lotr_graph_window_10_materialise/materialize |
3345959 ns/iter (± 47014) |
971980 ns/iter (± 4278) |
3.44 |
lotr_graph_subgraph_10pc/has_node_nonexisting |
5 ns/iter (± 0) |
2 ns/iter (± 0) |
2.50 |
lotr_graph_subgraph_10pc_materialise/materialize |
2009904 ns/iter (± 22468) |
334634 ns/iter (± 1287) |
6.01 |
lotr_graph_subgraph_10pc_windowed/has_node_existing |
149 ns/iter (± 9) |
62 ns/iter (± 14) |
2.40 |
lotr_graph_subgraph_10pc_windowed_materialise/materialize |
1217703 ns/iter (± 11569) |
230399 ns/iter (± 2617) |
5.29 |
lotr_graph_window_50_layered/num_edges_temporal |
155717 ns/iter (± 2232) |
70121 ns/iter (± 7586) |
2.22 |
lotr_graph_window_50_layered/has_node_existing |
390 ns/iter (± 24) |
129 ns/iter (± 12) |
3.02 |
lotr_graph_window_50_layered/has_node_nonexisting |
5 ns/iter (± 0) |
2 ns/iter (± 0) |
2.50 |
lotr_graph_window_50_layered/graph_latest |
85415 ns/iter (± 2064) |
36649 ns/iter (± 916) |
2.33 |
lotr_graph_window_50_layered_materialise/materialize |
30515575 ns/iter (± 95819) |
3488825 ns/iter (± 24948) |
8.75 |
lotr_graph_persistent_window_50_layered/num_edges_temporal |
646047 ns/iter (± 24446) |
192686 ns/iter (± 1569) |
3.35 |
lotr_graph_persistent_window_50_layered/has_node_existing |
442 ns/iter (± 435) |
174 ns/iter (± 83) |
2.54 |
lotr_graph_persistent_window_50_layered/has_node_nonexisting |
5 ns/iter (± 0) |
2 ns/iter (± 0) |
2.50 |
lotr_graph_persistent_window_50_layered/iterate_exploded_edges |
3480647 ns/iter (± 10244) |
1659940 ns/iter (± 19402) |
2.10 |
lotr_graph_persistent_window_50_layered/graph_latest |
137704 ns/iter (± 4154) |
57549 ns/iter (± 4809) |
2.39 |
lotr_graph_persistent_window_50_layered_materialise/materialize |
53184087 ns/iter (± 282815) |
5298035 ns/iter (± 147912) |
10.04 |
This comment was automatically generated by workflow using github-action-benchmark.
…eld from RemoteGraph
…e impls from Py* types to op types
…Degree/OutDegree/Name) with Python bindings + tests
…)/.dst() navigation
…optional-string machinery
…ypes, ExcludeNodes) + Valid, DefaultLayer, and graph Path/Namespace/Name terminals
…ighbour accessors
… wrong answers under view chains
…est, layer, shrink_*, etc.)
…est, layer, shrink_*, etc.)
…p through base_graph on all Remote types
…ates), reject non-finite floats, validate new_graph type
…silently succeeding
…) so .first().sum() doesn't run as .sum().first()
…sted), view-faithful to local
…ests; move RBAC tests to private repo
…efore tempdir teardown Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…er parity on path types
…isplay prints the inverse)
…rde as single wire source of truth
…0=auto heuristic, and the op.rs render leak
…per-member deep-clone cliff
…ne blocking_compute per nested read, not per source)
…ollecting the full history
…n InputTime, add filter serde goldens
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transport abstraction for the RemoteGraph client
Motivation
The existing Python
RemoteGraphclient is write-only, with each of ~15mutation methods inlining its own Jinja template and JSON-parsing chain.
This PR consolidates that plumbing into a single
Transportabstractionthat every operation flows through, and adds a minimal composable read
surface on top of the same seam so the client can now do lazy view
composition like
rg.window(0, 10).node("ben").degree()end-to-end.What shipped
Transporttrait inraphtory-graphql/src/client/transport.rs—async fn execute(&self, op: &Op) -> Result<Option<Prop>, ClientError>.Op { Read(ReadExpr) | Write(WriteOp) }inop.rs— read side is arecursive expression tree, write side is a flat enum of arg structs.
GraphqlTransportingraphql_transport.rs—renders write ops via the existing Jinja templates (moved from client
wrappers), walks read expressions into nested GraphQL queries, parses
responses.
wire behavior: 7 on
RemoteGraph, 4 onRemoteNode, 4 onRemoteEdge.Each method body shrinks from ~30–50 lines to ~5–15 lines.
Root,Window,Node,Degree— enough forrg.window(s, e).node(id).degree()end-to-end. Only the terminal firesan RPC.
GraphQLRemote{Graph,Node,Edge}→Remote*,RaphtoryGraphQLClient→RemoteClient,raphtory_client.rs→remote_client.rs. Python-facing names unchanged.clientfield removed fromRemoteNode/RemoteEdge— theynow hold only
transportand view state. Kept onRemoteGraphpendingbatch-method migration (see below).
PyRemoteGraph::node/edgenow delegate to the Rust methods,so the accumulated view chain (
window(...)) is correctly propagatedwhen descending to a node — previously silently dropped.
window()anddegree().Testing
raphtory-graphqltests still green.graphql_transport.rs(unit tests for the readrender/parse; one integration test spawning a real server).
test_remote_graph_transport.py:test_add_and_degree,test_windowed_degree,test_view_chain_propagation.TODO
add_nodes,add_edges) still callclient.query(...)directly (existingTODOs). Migrating them removesthe last
clientfield.properties, ...).
return-type conversion pattern.
for n in g.nodes: ....