FE-711: Add Metrics to Experiments

[kube]

🌟 What is the purpose of this PR?

🔗 Related links

• ...

🚫 Blocked by

• ...

🔍 What does this change?

• ...

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• does not modify any publishable blocks or libraries, or modifications do not need publishing
• modifies an npm-publishable library and I have added a changeset file(s)
• modifies a Cargo-publishable library and I have amended the version
• modifies a Cargo-publishable library, but it is not yet ready to publish
• modifies a block that will need publishing via GitHub action once merged
• I am unsure / need advice

📜 Does this require a change to the docs?

The changes in this PR:

• are internal and do not require a docs change
• are in a state where docs changes are not yet required but will be
  • this is tracked in: Insert Link Here
• require changes to docs which are made as part of this PR
• require changes to docs which are not made in this PR
  • Provide more detail here
• I am unsure / need advice

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph
• affected the execution graph, and the turbo.json's have been updated to reflect this
• I am unsure / need advice

⚠️ Known issues

🐾 Next steps

🛡 What tests cover this?

❓ How to test this?

1. Checkout the branch / view the deployment
2. Try X
3. Confirm that Y

📹 Demo

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview, Comment	May 26, 2026 9:11pm
petrinaut	Ready	Preview, Comment	May 26, 2026 9:11pm

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	May 26, 2026 9:11pm

[cursor]

PR Summary

Medium Risk
Published core API and worker message shape change, plus user-supplied expression metrics compiled at runtime; well covered by tests but affects experiment execution and streaming behavior.

Overview
Monte Carlo experiments no longer stream a built-in place token count distribution; they use configurable metrics (place means, transition firing counts, or compiled expression code) with scalar or per-run distribution output, optional run/time aggregation, and mergeable accumulators.

petrinaut-core exposes the new metric APIs, a per-run SimulationFrameReader for sampling, worker metricFrames / init metricSpecs, and experiment metrics (replacing distributions), including a local experiment path when metrics are defined without a worker.

Petrinaut UI requires at least one metric when creating an experiment (drawer editor with LSP for expressions), stores metricFrames / latestMetricFramesById, and charts results per metric instead of a single place timeline; the standalone Simulate Metrics tab is hidden from the mode switcher.

^{Reviewed by Cursor Bugbot for commit 74a64da. Bugbot is set up for automated code reviews on this repo. Configure here.}

[augmentcode]

🤖 Augment PR Summary

Summary: This PR introduces first-class Monte Carlo “experiment metrics” for Petrinaut experiments, including runtime/user-defined metric evaluation and UI controls to configure and view metric outputs.

Changes:

• Exports new Monte Carlo metric APIs and types from @hashintel/petrinaut-core (metric specs, user-defined metrics, accumulators).
• Adds a SimulationFrameReader-based per-run frame reader so metrics can safely inspect place/transition state.
• Implements numeric and histogram accumulator utilities plus tests for merge/monoid behavior.
• Adds metric spec compilation (including expression metrics via compileMetric) into user-defined metric configs.
• Extends createMonteCarloExperiment() with a local execution path when metric callbacks/specs are provided (worker cannot receive executable code).
• Updates experiments React state/provider to store metric specs, metric frames, and latest-by-id.
• Adds UI for defining metrics when creating an experiment (including LSP diagnostics for expression metrics).
• Updates experiment viewing UI to optionally show token-count timeline and a new metrics summary section.
• Adds architecture/proposal/usage docs for Monte Carlo metrics direction.

Technical Notes: Worker-backed experiments remain distribution-only; experiments with expression/aggregated metrics run locally to avoid posting JS code across the worker boundary.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

[graphite-app]

Metric ID validation inconsistency allows duplicate IDs with different whitespace. The code trims the ID when checking if empty (line 136) but adds the untrimmed ID to the Set (line 142), and checks for duplicates using the untrimmed ID (line 139). This means metric IDs like " test" and "test" would both pass validation as unique IDs, causing potential conflicts.

for (const metricSpec of input.metricSpecs) {
  const trimmedId = metricSpec.id.trim();
  if (trimmedId === "") {
    throw new Error("Metric id is required");
  }
  if (metricIds.has(trimmedId)) {
    throw new Error(`Metric id "${trimmedId}" is duplicated`);
  }
  metricIds.add(trimmedId);
  // ...
}

Spotted by Graphite



Is this helpful? React 👍 or 👎 to let us know.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c8d52c6. Configure here.}

[cursor]

Progress reports run 0's frame, not global frame

Medium Severity

getProgressFromResult and progressFromResult derive frameNumber and time from simulator.getRunSummary(0), which returns run 0's individual frame number. If run 0 completes early (e.g., no firable transitions), its frameNumber freezes while other runs continue advancing. The old code sourced these values from the distribution metric's context.frameNumber, which was the global simulator frame number — always reflecting the latest advance across all runs.

Additional Locations (1)

• libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/worker/monte-carlo.worker.ts#L36-L47

 

^{Reviewed by Cursor Bugbot for commit c8d52c6. Configure here.}

[cursor]

Worker cancel handler doesn't reset userMetrics array

Low Severity

The cancel handler sets simulator = null and isInitialized = false but does not reset userMetrics = []. The old code cleared distributionMetric = null at this point, and the init error handler in the same file correctly resets userMetrics = []. This inconsistency leaves stale metric objects and their accumulated frame data in worker memory after cancellation until the worker is re-initialized or terminated.

 

^{Reviewed by Cursor Bugbot for commit c8d52c6. Configure here.}