widget ready

[entlein]

Report every PxL /Vis.json here

[ConstanzeTU]

Accepted pxl + vis.json — px/dx_evidence_graph (commit da0c7a4, ships cloud v0.0.27-pre-v0.0)

User-accepted verbatim on 2026-08-19. This is the pair baked into the cloud image.

dx_evidence_graph.pxl

# Copyright 2018- The Pixie Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

import px


def _consulted(start_time: str, clickhouse_dsn: str, view: str, order_id: str):
    orders = px.DataFrame('dx_anomaly_orders', clickhouse_dsn=clickhouse_dsn, start_time=start_time)
    orders = orders[orders.order_id == order_id]
    orders = orders[['order_id', 'pod', 'lo', 'hi']]
    src = px.DataFrame(view, clickhouse_dsn=clickhouse_dsn, start_time=start_time)
    j = src.merge(orders, how='inner', left_on=['pod'], right_on=['pod'], suffixes=['', '_ord'])
    j = j[j.row_time >= j.lo]
    j = j[j.row_time <= j.hi]
    return j.drop(['row_time', 'event_time', 'lo', 'hi', 'order_id', 'pod_ord'])


def evidence_graph(start_time: str, clickhouse_dsn: str, table: str):
    df = px.DataFrame(table, clickhouse_dsn=clickhouse_dsn, start_time=start_time)
    df.from_entity = px.Pod(df.subject_pod)
    df.to_entity = df.target
    return df[['from_entity', 'to_entity', 'uniqueID', 'rule', 'process',
               'target', 'target_kind', 'severity', 'alert', 'subject_pod']]


def kubescape(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__kubescape_logs', order_id)


def redis(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__redis_events', order_id)


def conn(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__conn_stats', order_id)


def http(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__http_events', order_id)


def dns(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__dns_events', order_id)


def pgsql(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__pgsql_events', order_id)


def mysql(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__mysql_events', order_id)


def dc_snoop(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__dc_snoop', order_id)


def stack_trace(start_time: str, clickhouse_dsn: str, order_id: str):
    return _consulted(start_time, clickhouse_dsn, 'dx_src__stack_trace', order_id)


def orders(start_time: str, clickhouse_dsn: str):
    df = px.DataFrame('dx_anomaly_orders', clickhouse_dsn=clickhouse_dsn, start_time=start_time)
    return df[['uniqueID', 'rule', 'pod', 'order_id', 'lo', 'hi']]

vis.json

{
  "variables": [
    {"name": "start_time", "type": "PX_STRING", "description": "Window start.", "defaultValue": "-6h"},
    {"name": "clickhouse_dsn", "type": "PX_STRING", "description": "forensic_db DSN: user:pass@host:port/db.", "defaultValue": "forensic_analyst:changeme-analyst@clickhouse-forensic-soc-db.clickhouse.svc.cluster.local:9000/forensic_db"},
    {"name": "graph_table", "type": "PX_STRING", "description": "L1 kill-chain graph source.", "defaultValue": "dx_kubescape_anomalies"},
    {"name": "order_id", "type": "PX_STRING", "description": "Pick an order_id from the Orders panel at the bottom; every protocol panel snaps to that order's consulted records.", "defaultValue": ""}
  ],
  "globalFuncs": [
    {"outputName": "g_graph", "func": {"name": "evidence_graph", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "table", "variable": "graph_table"}]}},
    {"outputName": "g_kube", "func": {"name": "kubescape", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_redis", "func": {"name": "redis", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_conn", "func": {"name": "conn", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_http", "func": {"name": "http", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_dns", "func": {"name": "dns", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_pgsql", "func": {"name": "pgsql", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_mysql", "func": {"name": "mysql", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_dcsnoop", "func": {"name": "dc_snoop", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_stack", "func": {"name": "stack_trace", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}, {"name": "order_id", "variable": "order_id"}]}},
    {"outputName": "g_orders", "func": {"name": "orders", "args": [{"name": "start_time", "variable": "start_time"}, {"name": "clickhouse_dsn", "variable": "clickhouse_dsn"}]}}
  ],
  "widgets": [
    {"name": "Evidence graph (subject pod -> target)", "position": {"x": 0, "y": 0, "w": 12, "h": 4}, "globalFuncOutputName": "g_graph", "displaySpec": {"@type": "types.px.dev/px.vispb.Graph", "adjacencyList": {"fromColumn": "from_entity", "toColumn": "to_entity"}, "edgeWeightColumn": "severity", "edgeColorColumn": "severity", "edgeLabelColumn": "rule", "edgeThresholds": {"mediumThreshold": 5, "highThreshold": 8}, "edgeHoverInfo": ["uniqueID", "rule", "process", "target", "severity"], "edgeLength": 500}},
    {"name": "kubescape_logs (anomalies) for order", "position": {"x": 0, "y": 4, "w": 12, "h": 4}, "globalFuncOutputName": "g_kube", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "redis_events", "position": {"x": 0, "y": 8, "w": 6, "h": 4}, "globalFuncOutputName": "g_redis", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "conn_stats", "position": {"x": 6, "y": 8, "w": 6, "h": 4}, "globalFuncOutputName": "g_conn", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "http_events", "position": {"x": 0, "y": 12, "w": 6, "h": 4}, "globalFuncOutputName": "g_http", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "dns_events", "position": {"x": 6, "y": 12, "w": 6, "h": 4}, "globalFuncOutputName": "g_dns", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "pgsql_events", "position": {"x": 0, "y": 16, "w": 6, "h": 4}, "globalFuncOutputName": "g_pgsql", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "mysql_events", "position": {"x": 6, "y": 16, "w": 6, "h": 4}, "globalFuncOutputName": "g_mysql", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "dc_snoop (file access)", "position": {"x": 0, "y": 20, "w": 6, "h": 4}, "globalFuncOutputName": "g_dcsnoop", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "stack_trace (profiler)", "position": {"x": 6, "y": 20, "w": 6, "h": 4}, "globalFuncOutputName": "g_stack", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}},
    {"name": "ORDERS (pick an order_id -> paste into the order_id variable)", "position": {"x": 0, "y": 24, "w": 12, "h": 4}, "globalFuncOutputName": "g_orders", "displaySpec": {"@type": "types.px.dev/px.vispb.Table"}}
  ]
}

[ConstanzeTU]

v0.0.28-pre-v0.0 — order_id deep-links + kubescape detail restored in the graph popup

Commit: e168d19df77447d1e9595e959d3ea88d6468bf4c (branch fix/ae-protocol-export-pxexport, tag release/cloud/v0.0.28-pre-v0.0, build run 32350329109)

dx_evidence_graph.pxl

• evidence_graph() left-joins dx_anomaly_orders on (uniqueID, rule) and emits
  order_link = px.script_reference(order_id, 'px/dx_evidence_graph', {start_time, clickhouse_dsn, graph_table, order_id}).
  One edge per order (order_id granularity = uniqueID+RuleID+Disc since rc23).
• orders() now takes graph_table, renders order_id as the same clickable deep-link (order column), and drops raw lo/hi from the display (still in dx_order_records).

vis.json

• edgeHoverInfo: ["order_link", "rule", "alert", "process", "target", "severity", "uniqueID"] — restores the alert header (kubescape message, e.g. "Unexpected process launched: cat ...") dropped at 57268e1, adds the order link.
• ORDERS panel moved from the bottom to directly under the graph (y:4), retitled "click an order to filter all panels"; order_id variable description updated (no more copy-paste workflow).

graph.tsx (UI)

• ST_SCRIPT_REFERENCE columns in edgeHoverInfo are excluded from the hover tooltip text and rendered as a <ScriptReference> router link at the top of the click-to-pin edge popup (popup drag skips anchor clicks). Clicking the link re-runs the script with order_id set, so all panels snap to that order.

Behavior: click an edge (e.g. R0001) -> pinned popup shows [order deep-link] / rule / alert header / process / target / severity. Clicking the link or an ORDERS-row link filters every protocol panel to that order. Live verification on rig 6a85d889 pending image.

[ConstanzeTU]

v0.0.29-pre-v0.0 — order deep-link opens the EXACT consulted set

Commit: 8a7dd35e8a0e7598770d46135986a601ca44d481 (branch fix/ae-protocol-export-pxexport, tag release/cloud/v0.0.29-pre-v0.0; supersedes v0.0.28's panel read path, deep-link mechanics unchanged)

• Protocol/dark panels behind the order_id deep-link now read the stamped dx_order_records rows for that order (order_id + src_table filter; identity columns vector/source/pod/remote_addr/path/comm/dns_name) instead of reconstructing a pod ±300s window over the dx_src__* views. This is the dx#136 stamping model the dx_order_records DDL documents as the intended panel read path ("The panels read THIS ... instead of a ±300s time window").
• kubescape panel: joins dx_src__kubescape_logs on (uniqueID, RuleID) of the selected order — keeps the full anomaly seed rows (alert header, cmdline, process tree).
• evidence_graph()/orders() deep-links, vis.json, and the graph.tsx popup link are unchanged from e168d19df (v0.0.28).

Live verification on rig 6a85d889 once the image is up: stamped panels populate from the 82 existing dx_order_records, per-order counts match dx stamped counts.

[ConstanzeTU]

v0.0.29 amended before build: d72d9045c6970f36044d6d20e9eaf84fd321eed6 — ORDERS panel drops uniqueID (no analyst value) and instead joins the graph source (dx_kubescape_anomalies) on (uniqueID, rule) to show the human-readable alert text (e.g. 'Unexpected process launched: cat ...'), the same description as the graph popup. Display columns: order (deep-link) / rule / alert / pod. Tag release/cloud/v0.0.29-pre-v0.0 re-pointed to this commit (previous 8a7dd35 tag was never published). Also confirmed live: v0.0.28 is deployed and serving the deep-link widget (/bundle-oss.json carries script_reference + order_link).

[ConstanzeTU]

v0.0.31-pre-v0.0 — differential stack-trace flamegraph panel. Commit 1d24ba592, tag release/cloud/v0.0.31-pre-v0.0 (image building).

New stack_diff(start_time, clickhouse_dsn, order_id) + a full-width StackTraceFlameGraph widget below the existing panels. For the selected order it diffs the pod's stacks DURING the attack window [lo,hi] against its BASELINE stacks before the attack (row_time < lo), keyed off dx_anomaly_orders. delta = attack_count - baseline_count drives the differential colouring (red = frames that spiked during the attack). Reuses Pixie's StackTraceFlameGraph (semicolon-folded stacks, differenceColumn).

Notes verified live (user pasted into the widget): groupby over the ClickHouse-DSN source compiles + runs; the one bug was a merge suffixing the join key (diff.stack_trace -> diff.stack_trace_atk), now fixed. Also restores the v0.0.30 dx_src__ protocol panels + keeps the graph order_link deep-link + ORDERS alert-text column. Test recipe: run argocd functional-tests (benign baseline), wait >5min so those samples fall before lo, then attack -> the flamegraph baseline = normal render stacks, attack window = malicious render stacks.

[ConstanzeTU]

v0.0.32-pre-v0.0 — NEW script dx/evidence_graph (SOC, MITRE). Commit babc61c51 (tag release/cloud/v0.0.32-pre-v0.0, build 32496716717). px/dx_evidence_graph untouched.

• New dir src/pxl_scripts/dx/evidence_graph/ (pxl+vis.json+manifest); added dx to the bundle dir list in src/pxl_scripts/Makefile (the list is explicit, not globbed).
• Description: 'SOC pixie, works only with clickhouse enabled.'
• MITRE ATT&CK prominent: edge label = rule_mitre (ruleID · technique, e.g. 'R0011 · T1071'); tactic+technique in the edge popup and as leading columns in the kubescape panel + ORDERS. Source = BaseRuntimeMetadata.mitreTactic/mitreTechnique via new views dx_kubescape_mitre / dx_src__kubescape_mitre.
• Data model = dx_orders + dx_ord__* (kubescape_uid+rule_id join). Order deep-links (script_reference -> dx/evidence_graph) on graph edges + ORDERS. Differential stack-trace flamegraph via dx_orders_win (Int64 lo/hi).
• Caveat: the 3 mitre/window views are currently rig-only on 6a87e018 (not yet in AE schema.sql) per user choice — script works on that rig; needs the views baked for durability elsewhere. SQL: scratchpad/mitre_views.sql.