Generic Remotion leaderboard video for benchmark composite scores - #352
Open
HeyGarrison wants to merge 19 commits into
Open
Generic Remotion leaderboard video for benchmark composite scores#352HeyGarrison wants to merge 19 commits into
HeyGarrison wants to merge 19 commits into
Conversation
…e scores Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor License AgreementAll contributors are covered by a CLA. |
…st logo fallback Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ed leaderboard Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ion range Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…top 5 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…e video Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… gap Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ks site; relabel 'even more' Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…reveal timings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…s and race Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…-one finishes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…sors label Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Read any benchmark result JSON via BENCHMARK arg or env - Derive title from benchmark path mapping, score from compositeScore (or dax phases) - Use dotcom-aligned logo manager (providerBrandId, logomarkSlug, MISSING_LOGO_BRAND_IDS) - Fix ComputeSDK logo/title alignment in Composition Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
| return BENCHMARK_TITLES[benchmark] ?? humanizeBenchmark(benchmark); | ||
| } | ||
|
|
||
| function getResultPath(benchmark: string): string { |
There was a problem hiding this comment.
P2: Path traversal vulnerability in benchmark result resolution
Unsanitized BENCHMARK env/arg used in path.resolve enables directory traversal to read arbitrary files.
Validate benchmark names against an allowlist and ensure resolved paths remain within the intended results directory.
AI prompt
Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior.
<file name="packages/remotion-video/src/prepare.ts">
<violation number="1" location="packages/remotion-video/src/prepare.ts:94">
<priority>P2</priority>
<title>Path traversal vulnerability in benchmark result resolution</title>
<evidence>The `getResultPath` function constructs a filesystem path using an unsanitized `benchmark` parameter (`process.argv[2]` or `process.env.BENCHMARK`) passed directly into `path.resolve`: `return path.resolve(PKG_ROOT, '../../results', benchmark, 'latest.json');`. An attacker controlling the `BENCHMARK` environment variable or CLI argument can traverse outside the intended `results` directory to read arbitrary files on the filesystem (e.g., `BENCHMARK=../../../etc/passwd`).</evidence>
<recommendation>Sanitize the `benchmark` input before using it in a path. Validate it against an allowlist of known benchmark names, or reject values containing path traversal sequences (e.g., `..`, absolute paths). Use `path.join` combined with a check that the resolved path remains within the intended base directory.</recommendation>
</violation>
</file>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Genericizes
packages/remotion-videoso the same leaderboard composition can be rendered for any benchmark inresults/<benchmark>/latest.json.prepare.tsnow accepts a benchmark viapnpm generate <benchmark>orBENCHMARK=.... It derives the title from a small mapping (with aBENCHMARK_TITLEenv override), reads the result JSON, and uses each provider'scompositeScorewhen present. Forsandbox-dax(which has nocompositeScore), it falls back to the medianphasesCompleted / 7 * 100.src/logo.tsusingproviderBrandId,logomarkSlug, andMISSING_LOGO_BRAND_IDSsocloud-run→google-cloud-run,browseruse→browser-use, andlightning/sailfall back to the local SVGs served by dotcom. Display names use the samecapitalizehelper as dotcom (CodeSandbox,Cloud Run,Lightning AI, etc.).Composition.tsxkeeps the 30-second, dark-mode, full-width race layout, but the header now places the dark ComputeSDK wordmark and title in a single flex container so they stay vertically aligned and the logo is sized larger than the title.Try it
Sample output
Opening frame (frame 0):
Final leaderboard at frame 840:
Watch the 30s race video
Only composite scores are displayed; no median/p95/p99 metrics appear in the video.
Link to Devin session: https://app.devin.ai/sessions/e276f3c69f6a4e4783b1d3f09f60d930
Requested by: @HeyGarrison