Skip to content

feat(workflow): mark the pinned version in the revision history - #8577

Draft
yangzhang75 wants to merge 2 commits into
apache:mainfrom
yangzhang75:pin/5-anchor
Draft

yangzhang75 wants to merge 2 commits into
apache:mainfrom
yangzhang75:pin/5-anchor

Conversation

@yangzhang75

@yangzhang75 yangzhang75 commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Which version is the public one should be answerable, and the answer should be a row the author
can already see — the version panel is where they go to look at, and restore, an earlier version.

  • A pin leaves an anchor in the revision history. Its delta is the identity patch, so replaying
    that row returns what was published however many edits pile up later. An existing version row
    cannot stand in: one replays to the content as it was before the change it records, so it
    names the wrong thing by exactly one save. Pinning again with nothing changed reuses the anchor
    rather than adding a twin, and workflow.published_version_id names it.
  • The anchor is a marker, not an edit. It is always shown, so the author can find and restore
    what the public has, but it must not start the panel's aggregation window — the save it was
    pinned from is seconds older and would otherwise be folded into it and disappear.
  • The public view is dated by the version on show, not by the author's most recent private
    edit, and the share dialog names the same date from the same row, so the two cannot disagree.
  • Pinning takes the row's lock first. Two pins racing would otherwise both read before either
    wrote, and both insert an anchor.

Any related issues, documentation, discussions?

Closes #7941
Part of #7828. Design discussion: #7128. Stacked on #8576; until that merges this PR shows
its commit too, and the review here is the last one.

How was this PR tested?

7 new cases in WorkflowPublishSpec and WorkflowVersionResourceSpec (572 in the dashboard
package):

  • an anchor the author can identify in the panel, and the save it was pinned from still visible
    beside it rather than folded into it;
  • pinning again unchanged reusing the anchor instead of adding a second one;
  • the version the pin was taken from surviving after the pin is dropped;
  • the public view dated by the pinned version, and the same date reported by the share dialog;
  • a workflow that is following carrying no anchor at all.

scalafmtCheckAll clean.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @aglinxinyuan, @tanishqgandhi1908
    You can notify them by mentioning @aglinxinyuan, @tanishqgandhi1908 in a comment.

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 4 better · 🔴 5 worse · ⚪ 6 noise (<±5%) · 0 without baseline

Compared against main 3726dd7 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 389 0.238 25,651/30,788/30,788 us 🔴 +20.5% / 🔴 +140.9%
🔴 bs=100 sw=10 sl=64 800 0.488 125,421/135,560/135,560 us 🟢 -5.8% / 🔴 +49.8%
⚪ bs=1000 sw=10 sl=64 920 0.561 1,083,719/1,165,419/1,165,419 us ⚪ within ±5% / 🔴 +37.5%
Baseline details

Latest main 3726dd7 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 389 tuples/sec 425 tuples/sec 968.33 tuples/sec -8.5% -59.8%
bs=10 sw=10 sl=64 MB/s 0.238 MB/s 0.259 MB/s 0.591 MB/s -8.1% -59.7%
bs=10 sw=10 sl=64 p50 25,651 us 21,286 us 10,646 us +20.5% +140.9%
bs=10 sw=10 sl=64 p95 30,788 us 34,010 us 14,120 us -9.5% +118.0%
bs=10 sw=10 sl=64 p99 30,788 us 34,010 us 16,498 us -9.5% +86.6%
bs=100 sw=10 sl=64 throughput 800 tuples/sec 842 tuples/sec 1,252 tuples/sec -5.0% -36.1%
bs=100 sw=10 sl=64 MB/s 0.488 MB/s 0.514 MB/s 0.764 MB/s -5.1% -36.1%
bs=100 sw=10 sl=64 p50 125,421 us 118,680 us 83,736 us +5.7% +49.8%
bs=100 sw=10 sl=64 p95 135,560 us 143,875 us 92,464 us -5.8% +46.6%
bs=100 sw=10 sl=64 p99 135,560 us 143,875 us 108,561 us -5.8% +24.9%
bs=1000 sw=10 sl=64 throughput 920 tuples/sec 920 tuples/sec 1,321 tuples/sec 0.0% -30.3%
bs=1000 sw=10 sl=64 MB/s 0.561 MB/s 0.561 MB/s 0.806 MB/s 0.0% -30.4%
bs=1000 sw=10 sl=64 p50 1,083,719 us 1,085,986 us 811,260 us -0.2% +33.6%
bs=1000 sw=10 sl=64 p95 1,165,419 us 1,142,211 us 847,636 us +2.0% +37.5%
bs=1000 sw=10 sl=64 p99 1,165,419 us 1,142,211 us 879,277 us +2.0% +32.5%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,513.90,200,128000,389,0.238,25650.74,30787.87,30787.87
1,100,10,64,20,2499.85,2000,1280000,800,0.488,125421.46,135559.87,135559.87
2,1000,10,64,20,21747.49,20000,12800000,920,0.561,1083718.59,1165419.02,1165419.02

@codecov-commenter

codecov-commenter commented Sep 17, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.74797% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.34%. Comparing base (3726dd7) to head (43494d3).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...shboard/user/workflow/WorkflowPublishService.scala 94.87% 1 Missing and 1 partial ⚠️
...ra/web/resource/dashboard/SearchQueryBuilder.scala 50.00% 1 Missing ⚠️
...esource/dashboard/WorkflowSearchQueryBuilder.scala 97.95% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8577      +/-   ##
============================================
+ Coverage     92.33%   92.34%   +0.01%     
- Complexity     5045     5046       +1     
============================================
  Files          1260     1260              
  Lines         54067    54165      +98     
  Branches       6755     6764       +9     
============================================
+ Hits          49921    50018      +97     
- Misses         2486     2489       +3     
+ Partials       1660     1658       -2     
Flag Coverage Δ *Carryforward flag
access-control-service 77.38% <ø> (ø) Carriedforward from f44c37f
agent-service 99.16% <ø> (ø) Carriedforward from f44c37f
amber 87.37% <96.74%> (+0.06%) ⬆️
computing-unit-managing-service 60.48% <ø> (ø) Carriedforward from f44c37f
config-service 87.37% <ø> (ø) Carriedforward from f44c37f
file-service 82.00% <ø> (ø) Carriedforward from f44c37f
frontend 96.55% <ø> (ø) Carriedforward from f44c37f
notebook-migration-service 83.73% <ø> (ø) Carriedforward from f44c37f
pyamber 98.57% <ø> (ø) Carriedforward from f44c37f
workflow-compiling-service 72.43% <ø> (ø) Carriedforward from f44c37f

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@yangzhang75
yangzhang75 force-pushed the pin/5-anchor branch 3 times, most recently from 6c57b03 to c2e9a09 Compare September 28, 2026 21:13
@yangzhang75
yangzhang75 marked this pull request as ready for review October 6, 2026 18:42
@yangzhang75
yangzhang75 marked this pull request as draft October 7, 2026 01:36
@yangzhang75
yangzhang75 force-pushed the pin/5-anchor branch 3 times, most recently from 7385c46 to 699d08a Compare October 13, 2026 00:55
yangzhang75 and others added 2 commits October 18, 2026 15:06
Search and the listings it feeds were reading the author's live columns,
which for a pinned workflow is the one copy the public cannot open. A
draft would turn up in a public search under a title nobody has seen, and
the card would advertise a name the detail page does not show.

Each filter is now applied to whichever copy the caller may see:
`onVisibleCopy` builds the same filter twice -- over the live columns for
rows the caller was granted access to, over the frozen ones for rows they
reach only because the workflow is public -- and ORs the two. A
disjunction over bare columns rather than a CASE, so each side stays
eligible for its own fulltext index. Unpinned public rows fall back to
the live columns, so a following workflow searches exactly as it does now.

Listings carry two more things from the same query: the frozen name and
description to show a viewer without granted access, and whether the copy
on show is behind the author's working copy. `constructWhereClause` takes
`includePublic` for this; the other builders accept and ignore it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The author can pin a version, and their working copy then moves on. This
gives them a way back to what the public is seeing: pinning leaves an
anchor in the revision history they already use, and the panel marks it
as the one currently public, so restoring the published version is the
restore they already know.

The anchor is a version row whose delta is the identity patch, so
replaying it returns exactly what was pinned however many edits pile up
after. An existing row cannot stand in: a version row replays to the
content as it was *before* the change it records. Pinning content that is
already the pinned one reuses its anchor rather than adding a twin, and
the read that decides takes a row lock so two pins racing cannot both
insert.

Two consequences the anchor forces:

- It must not start the version panel's aggregation window. It lands
  seconds after the save it freezes, and the panel folds close-together
  versions into the newest, which would hide the author's own save behind
  a row they never made.
- The revision history is now readable only with granted access, or while
  nothing is pinned. Replaying a version folds deltas back from the
  author's current content, so listing versions of a pinned workflow would
  hand a public viewer the very edits the pin is holding back.

`publish-status` carries the pinned version's date, read from the version
row so the dialog and the panel print one value rather than two clocks'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mark the pinned version in the revision history

2 participants