feat: Codolio-style tracker, portfolio and contest manager - #19
Open
kiro-agent[bot] wants to merge 7 commits into
Open
kiro-agent[bot] wants to merge 7 commits into
kiro-agent[bot] wants to merge 7 commits into
Conversation
Brings the three Codolio pillars to CodeOvertake as a new API surface, without touching the existing NSUT leaderboard behaviour. Accounts - User model with portfolio fields, JWT auth, bcrypt passwords, GitHub OAuth SSO - requireAuth / optionalAuth middleware; long-lived tokens for the extension Question Tracker (My Workspace) - Shared Problem catalog keyed by (platform, slug) with URL parsing for 11 platforms, so any problem link resolves to one row - Metadata auto-fetch: LeetCode GraphQL, Codeforces problemset API, the GeeksforGeeks practice API (which also yields company tags), HTML fallback - TrackedQuestion joins user to problem and carries status, star, custom tags. Status lives here, so solving once reflects in every sheet containing it Linked notes - One note can link many problems, so a pattern note written once appears on every related question. Notes with no links are standalone Spaced repetition - SM-2 derived scheduling over four confidence ratings - Retention Rating models memory as exponential decay, exp(-days/stability) - Daily queue of 5 weakest questions, persisted so it is stable until midnight, unlocking at 20 solved; streaks tracked on completed queues Sheets - Curated and custom sheets with topic/subtopic hierarchy, drag-drop reorder, public/private, email collaborators, follow-to-track, CSV bulk import that creates topics on the fly - Six seeded curated sheets (Blind 75, DP/Graph Mastery, CP-31, Quick Revision, GfG Essentials); all 182 refs verified to resolve to real metadata Company kits - All-Time / Last 6 Months / Last 45 Days buckets over Problem.companies Portfolio - C-Score across DSA/CP/Dev pillars with a balance factor - Platform verification by echoing a one-time code, on-demand sync with a 15 minute cooldown, projects with GitHub repo picker, reorder and upvotes, education/experience, C-Score leaderboard limited to verified profiles Contest tracker - Live aggregation from LeetCode, Codeforces, CodeChef and AtCoder with calendar/list queries, platform filters and Google Calendar links Platform registry is now two-tier: the four leaderboard platforms are unchanged, while portfolios can additionally link AtCoder (full stats) plus GeeksforGeeks and HackerRank as link-only, since their user-stats APIs are unavailable and shipping scrapers that silently return zeros would corrupt scores.
… extension Frontend - AuthContext with persisted sessions; login, signup and GitHub OAuth callback - api.ts now attaches the bearer token automatically and exports typed clients for every new endpoint - My Workspace: filterable question list, add-by-URL or by-name modal, and a detail panel showing the schedule plus every note linked to the problem - Notes: editor with a linked-questions picker, so one note can surface on many - Daily Revision: queue cards with four confidence ratings, retention gauge, memory breakdown, four-week decay forecast and a revision heatmap - Sheets: explore/mine/following, hierarchical detail view with progress, follow-to-track, CSV import, collaborators and owner settings - Company Kits: company index and kit detail with the three preparation modes - Portfolio: public /u/:handle with C-Score radar, platform cards, projects and background; Edit Profile covering platforms, verification, projects and work - Contest Tracker: month calendar re-bucketed to local time, upcoming panel, platform filters, countdowns and Add-to-Google-Calendar - Layout reworked into primary nav + More menu + account menu, and the footer gained a Tracker column Extension (Manifest V3) - Badges the toolbar icon on recognised problem pages, resolves the problem through the API, then saves it to the workspace with status, star, tags, an optional linked note and an optional target sheet - Authenticates with a rotatable pairing token rather than a session JWT, since it cannot refresh one; requireAuth accepts both Backend - middlewares/auth.js: resolveUserFromToken() accepts a session JWT or a 48-hex extension token, shared by requireAuth and optionalAuth - app.js: CORS now allows chrome-/moz-extension origins, whose ids vary per install, alongside the configured frontend origin
The seed scripts were silently untracked. backend/.gitignore ignored `scripts/`
and `data/` wholesale, so scripts/seedContent.js and scripts/data/*.js were
never committed even though package.json exposes four npm scripts for them and
the README documents them. Since seedContent.js is the only writer of
`isCurated: true`, a fresh clone had an empty Explore Sheets tab and no company
kits at all. The ignore rules are now anchored to actual data files (/data/,
*.csv, *.xlsx) so source under scripts/ is tracked.
Security
- SSRF: POST /api/problems/resolve is unauthenticated and reached the
`platform: 'other'` fallback, which let any caller make the server GET an
arbitrary URL and read the scraped <title> back (cloud metadata endpoints,
private hosts). The generic scraper now refuses any host outside an explicit
allowlist, unknown hosts are never fetched at all, redirects are capped at 2,
and the endpoint is rate limited to 30/min.
- GitHub OAuth honoured a caller-supplied redirect_uri, so the authorization
code could be delivered to an attacker-controlled host. It must now point at
FRONTEND_URL.
- Dropped normalizeEmail() from the auth routes: it strips Gmail dots while the
OAuth path does not, so the same person could end up with two accounts. Both
paths now share the lowercase+trim rule in authService.
Correctness
- workspaceService.addQuestion returned before saving staged tags/star when the
same request also marked the question solved, silently dropping them. That is
the extension's main flow ("save as solved" with tags).
- resolveByUrl left metadataFetchedAt null on a partial result, which its own
staleness check read as stale, so every partial problem was re-fetched on
every single request. Partial results are now stamped and retried on a
shorter TTL, tracked via a new metadataPartial flag. The partial path also
no longer overwrites rating/externalId/isPremium/acceptanceRate.
- Sheet followerCount is now derived from the join collection instead of $inc,
so concurrent follows cannot double-count and the value cannot drift negative.
- portfolioService.syncPlatforms skipped adapters that threw without marking
lastFetchFailed, so a hard failure looked like a successful sync.
- Company kits sorted by companies.frequency, which ranks by whichever tag in
the array is highest rather than the requested company's. getCompanyKit now
projects that company's own tag in an aggregation and sorts on it.
- Added explicit isCurated guards to updateSheet/deleteSheet.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…rofile
Two related changes: Clerk becomes the identity provider (which is what brings
"Continue with Google" and every other social provider), and the pre-existing
owner-less `Student` records finally get a way to be attached to a real person.
Clerk replaces the hand-rolled auth
- Removed email/password signup/login, the bcrypt hashes, the JWT signing
(utils/jwt.js), and the hand-rolled GitHub OAuth dance. This app no longer
stores credentials at all.
- `clerkUserId` is now the identity on User. The document is a mirror: it holds
the data we own (handle, portfolio, platform links, workspace, C-Score) plus a
cache of Clerk's email/name/avatar.
- Providers are toggled in the Clerk Dashboard, so adding Google, Apple, Discord
etc. needs no code change - the prebuilt <SignIn> picks them up. Styled to the
app's dark palette via a shared appearance object.
- Session tokens are verified with @clerk/backend, networkless when CLERK_JWT_KEY
is set, and checked against authorizedParties so a token minted for another app
on the same instance cannot be replayed here.
- Users are provisioned just in time on their first authenticated request rather
than waiting on `user.created`, because webhooks are eventually consistent and
someone who signs up and immediately calls the API would otherwise 401.
- POST /api/webhooks/clerk (Svix-signed, raw body mounted before express.json)
syncs `user.updated` and cascades `user.deleted`. Deletion removes what is
exclusively theirs, detaches what others depend on (sheet collaborators,
project upvotes), and releases any leaderboard claim.
- nameOverridden/avatarOverridden flags stop a later Clerk sync from reverting a
name the user deliberately changed in our own Edit Profile screen.
- GitHub OAuth tokens are read from Clerk on demand for the repo picker instead
of being stored, so a database dump contains no usable GitHub credentials.
- The browser extension keeps its rotatable pairing token: a service worker has
no context in which to refresh a short-lived Clerk token. Both credential types
resolve through the same requireAuth.
Claiming a leaderboard profile
The Student collection predates accounts - records were created by whoever typed
in a roll number, carry no owner, and are guarded only by a 24h cooldown, so
anyone knowing a roll number can edit them.
The key observation is that such a record already contains a proof primitive: the
coding handles on it. Proving control of a handle *already on the record* is
strong evidence you own it, needs no college infrastructure, and reuses the
one-time-code mechanism the portfolio already had. Three paths, strongest first:
1. platform-verified - the record's handle is one already verified on your
portfolio. Instant.
2. platform-code - paste a code into that platform's profile; we read it
back from the handle ON THE RECORD, never from caller input, so you cannot
point us at your own account.
3. institute-email - a Clerk-verified address on INSTITUTE_EMAIL_DOMAIN plus
either the roll number in the address or a name match against the official
student lookup. A college address alone cannot grab a classmate's profile.
Ownership is adopted progressively rather than by cutover: unclaimed records keep
the original open-with-cooldown behaviour so nobody who has not signed up is
locked out, while a claimed record becomes owner-only (and skips the cooldown,
since the owner is editing their own data). Every claim permanently closes one
more open record.
Also closed a spoofing hole this exposed: `rollno` was free text on the profile
form, so anyone could point their portfolio at someone else's ranking. It is now
only ever written by a verified claim.
Handles are masked in the claim UI so probing roll numbers cannot harvest
someone's usernames, only one claim can be pending per record, codes expire after
an hour, attempts are capped, and there is an admin reassign/unclaim escape hatch.
Verification
224 tests pass against a real MongoDB: 131 feature tests, 69 new tests covering
the Clerk model and all three claim paths, and 24 regression tests from the
earlier review. Frontend builds and every new/changed .tsx typechecks clean.
Clerk's own signature verification cannot be exercised without a live instance,
so the suite authenticates via the extension pairing-token path - the same
requireAuth entry point - and separately asserts that unsigned JWTs and unsigned
webhooks are rejected.
Three workflows. `ci.yml` gates every PR and, once green, deploys a live beta environment for that branch. `pr-cleanup.yml` tears it down on close. `deploy.yml` ships master to production and rolls back if it fails to come up. Preview environments - The preview job runs `needs: [backend, frontend]`, so a build CI already knows is broken never gets deployed and never wastes review time. - Each PR gets its own MongoDB database inside the existing cluster (`codeovertake_pr_<n>`), derived from the base URI by utils/mongoUri.js. A preview therefore cannot read or corrupt production data, and teardown is one dropDatabase call. Databases are cheap; separate clusters are not. - Previews run with DISABLE_CRON=true. Without it every open PR would schedule its own nightly student refresh and 6-hourly contest sync, so N open PRs means N times this repo's load on GitHub, LeetCode, Codeforces and CodeChef. The boot-time contest sync still runs so the calendar has data to test against. - Curated sheets and company kits are seeded best-effort: an empty catalog makes most of the app untestable, but a slow upstream should not fail a working preview. - The PR comment is upserted rather than appended, so pushing 20 times leaves one comment, not 20. Deployment - Secrets are written to a 0600 env file on the host instead of being passed as -e flags, keeping them out of `docker inspect` and the process list. - production-up.sh records the running image before changing anything and rolls back to it if the new container fails its health check, so a bad merge cannot leave the API down. It still exits non-zero: a rollback is a failed deploy. - deploy.yml additionally verifies the *public* URL, which catches reverse-proxy and DNS problems that an in-host check cannot see. - Every deploy job short-circuits with a notice when DEPLOY_HOST is unset, so the pipeline is useful before any hosting is configured rather than permanently red. Making CI meaningful - Moved the test suite into backend/test/ so it can actually run in CI. It was written earlier but lived outside the repo, which made it worthless to anyone else. 266 tests: 42 pure-function units plus 224 integration tests against a real MongoDB and the real Express app over HTTP. - Added a units suite covering the security-relevant edges: the CORS wildcard must be a single label, so `https://evil.vercel.app.attacker.com` is rejected; preview URIs must never point at the production database; spaced-repetition intervals must always progress. - features.test.js calls live platform APIs on purpose, so run.js retries a failed suite once. That absorbs an upstream blip without hiding a reproducible failure. - Added `npm run lint` (parses all 106 files) as a stand-in for the linter this project does not have. Frontend is now type-checked - Added tsconfig.json and `npm run typecheck`, and fixed the two errors that previously made a type gate impossible: HeadOn.tsx called useRef with no argument, and main.tsx imported a path with an explicit .tsx extension. `strict` is deliberately left off - the app leans on implicit any in many places, and turning it on would produce hundreds of errors and make the gate useless. It catches wrong props, bad imports and misused APIs today, and can be tightened incrementally. Also adds deploy/Caddyfile.example. Without a reverse proxy a preview is reachable on a plain HTTP port, which works for curl, Postman or a local frontend but is blocked by the browser from an HTTPS Vercel preview. The example shows the wildcard-TLS setup that removes that limitation.
… content and jobs
Everything privileged in this app was previously reachable only through a single
shared ADMIN_SECRET header, which meant there was no way to know who had done
what. This adds a real panel at /admin with role-based access on top of Clerk and
an append-only audit log behind every write.
Access model
------------
One middleware (middlewares/adminAuth.js) accepts either path so a route cannot
accidentally accept only the weaker of the two:
* a signed-in, non-suspended User.isAdmin account, which produces an
attributable actor in the audit log;
* the x-admin-secret header compared in constant time, for scripts, cron and
CI, which have no session. Flagged as req.adminViaSecret so the audit trail
can tell them apart.
A valid non-admin session is answered 403 directly rather than falling through to
the secret check, so the reason is never ambiguous. Anonymous is 401, wrong
credential is 403 — the old middleware answered 403 for both, and the
auth-claims test that asserted this was updated along with two new cases for a
wrong and a short secret.
isAdmin is recomputed on every Clerk sync from ADMIN_EMAILS (verified addresses
only, so you cannot self-promote by adding someone else's email to your own Clerk
account) and from the Clerk `role` public metadata, which means removing someone
from either actually revokes access. A manual promotion from the panel sets
adminGrantedManually so a sync does not silently undo it.
Guard rails: no self-demotion, self-suspension or self-deletion; an admin must
be demoted before being suspended or deleted; the last remaining admin cannot be
removed.
Audit log
---------
models/AuditLog.js + utils/audit.js record actor, action, target, a before/after
metadata diff and an outcome. The panel can filter by action, target type or
exact target id, expand any diff and export a page as JSON. Entries attributed
to the shared secret are visually flagged, because they are unattributable by
construction.
Backend
-------
services/adminService.js covers students (including edits that deliberately
bypass the 24h cooldown and the ownership check, recorded as an override),
accounts, claims, the problem catalog, sheets and contests, plus an in-memory
job registry.
Job state is progress reporting, not a queue: it is per-process, so a restart
clears it, and a second copy of a running job is refused because two concurrent
full student refreshes would double the load on every platform API and race on
the same documents. The start of a job is always audited, so that record
survives a restart even though the status does not.
Two rate limits rather than one: 300/15min for ordinary panel traffic and
10/15min for job triggers. The pre-existing single 3/15min limit on the whole
router would have made the panel unusable after three clicks.
Frontend
--------
frontend/src/app/components/admin/ — a role-gated shell with tab navigation
mirroring the settings page, and shared dense-table primitives. Destructive
actions use a two-click arm/disarm button rather than window.confirm(), which is
easy to click through by reflex. The nav link is hidden for non-admins as a
convenience only: the route re-checks with /api/admin/whoami on mount and every
endpoint re-checks the role.
Tests and docs
--------------
75 new checks in backend/test/admin.test.js covering the authorization matrix,
suspension lockout, every guard rail, the cooldown bypass, audit diffs, delete
refusal while a problem is still referenced, curation, and the job registry.
343 checks pass overall. README gains an Admin panel section and the admin
endpoint table; .env.example documents ADMIN_EMAILS and ADMIN_SECRET.
…ut user
A signed-in user hitting /api/auth/me and getting {"error":"Authentication
required"} had no way to find out why, because verifyClerkToken caught every
verification error and returned null. "Not a Clerk token", "expired", "signed by
a different instance" and "your secret key is invalid" were all flattened into
the same 401 with no log line.
That silence is the actual defect. The reasons are now classified:
* secret-key-invalid and the jwk-* family mean this server cannot verify
anything at all — no token would ever succeed. Answering 401 tells the caller
to fix credentials, which is the one thing that cannot help, so these now
raise and surface as 503.
* token-invalid-authorized-parties logs the azp it received, the list it
accepts, the issuer, and the env var to change. This is the most common cause
of the symptom and was completely invisible.
* token-invalid-signature and token-invalid-algorithm log that the frontend and
backend are probably pointing at different Clerk instances.
* token-invalid stays silent: the 48-hex extension pairing token lands there on
every extension request, so logging it would drown the signal.
* expired / not-yet-valid stay silent too; the client refreshes.
Diagnostics are throttled to one per reason per minute, because a broken deploy
emits one per request and only the first is informative.
authorizedParties now also accepts the literal entries of ALLOWED_ORIGINS — an
origin already trusted for CORS is by definition a legitimate party — and strips
trailing slashes, which would otherwise never match azp since the comparison is
exact. Wildcard CORS patterns are skipped because azp matching cannot express
them. CLERK_AUTHORIZED_PARTIES is documented for everything else.
server.js prints the effective auth configuration at boot: which key is in use,
the accepted origin list, and a warning if no Clerk key is set at all. A
frontend origin missing from that list is now a glance rather than an
investigation.
New test/clerk-session.test.js (24 checks) covers the branch every browser
request actually uses and which nothing tested before: the other suites
authenticate through the extension pairing token because minting a Clerk session
token needs a live instance. It generates an RSA keypair, points CLERK_JWT_KEY at
the public half — Clerk's own networkless verification mode — and mints genuinely
signed tokens to assert the happy path, azp mismatch, absent azp, expiry,
not-yet-valid, wrong signing key, 503-not-401 on an unverifiable server,
extension tokens surviving Clerk being broken, and suspension still returning 403.
367 checks pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the feature set of codolio.com into CodeOvertake, turning it from a leaderboard into Leaderboard + Tracker + Portfolio + Contest Manager.
The existing NSUT leaderboard is deliberately untouched — same four platforms, same scoring, same rankings. Everything here is additive.
What's included
Design decisions worth reviewing
Solved status lives only on
TrackedQuestion. Sheets store no progress, which is what makes "solve once, marked everywhere" work and keeps collaborators from seeing each other's progress. Sheet views join progress in at read time.The platform registry is now two-tier.
getAllPlatforms()still returns exactly the original four leaderboard platforms, socron/updateData.js,rankingService,studentServiceand the leaderboard routes are untouched and existing scores stay comparable.getPortfolioPlatforms()returns seven and is used only byUserandportfolioService.Student.jsandSnapshot.jsare unchanged.GeeksforGeeks and HackerRank are registered as link-only (
statsSupported: false). Both render user profiles entirely client-side and expose no usable public stats API — HackerRank's/rest/hackers/:user/profilenow 404s for every handle, and the GFG profile page ships no server-side data payload. Shipping scrapers that silently return zeros would drag real C-Scores down, so they store the handle and render a link instead. GFG problems are still fully trackable: its practice API returns title, difficulty and company tags, which seed the company kits for free. AtCoder is a full adapter (rating history + solved counts).Spaced repetition is SM-2-derived scheduling plus a separate memory model: retention for one question is
exp(-daysSince / stability), where stability grows with each confident repetition. The Retention Rating is the mean across everything solved. The maths is pure and dependency-free inutils/spacedRepetition.js.C-Score takes the best platform per pillar (so linking three CP sites doesn't triple-count one skill), blends DSA/CP/Dev at 45/30/25, then multiplies by a balance factor worth up to +15% — that multiplier is what makes it reward breadth.
Verification
All third-party integrations were tested against live endpoints, not mocked.
.tsxtypechecks with no errors.Fixed during review
A self-review pass caught two issues worth calling out:
backend/.gitignoreignoredscripts/anddata/wholesale. SinceseedContent.jsis the only writer ofisCurated: true, a fresh clone would have had an empty Explore Sheets tab and no company kits, whilepackage.jsonadvertised four npm scripts that would fail withMODULE_NOT_FOUND. The ignore rules are now anchored to real data files.POST /api/problems/resolve. That endpoint is unauthenticated and its unknown-host fallback let a caller make the server GET any URL and read the scraped<title>back. The generic scraper now refuses hosts outside an allowlist, never fetches unknown hosts, and is rate limited.Also fixed: tags being dropped when a request both tagged and solved a question (the extension's main flow), a metadata re-fetch loop on partial results,
followerCountdrift under concurrent follows, silent success on a thrown sync adapter, and company kits ranking by the wrong company's frequency.Notes for the reviewer
JWT_SECRETis now required for accounts to work; see the expanded.env.example. GitHub OAuth vars are optional and only gate SSO.chrome-extension:///moz-extension://origins, since the extension id differs per install.frontend/src/app/components/HeadOn.tsx:110has a pre-existing type error (useRefwith no argument under current@types/react). Left alone as it's unrelated and harmless at runtime, but it would need fixing before atscgate could be added.npm run seed:content --prefix backendafter deploying to populate the curated sheets and company kits. It's idempotent.