Calmr

Exposure therapy that breathes with the patient. A patient's heart rate, from a Bluetooth strap or a laptop webcam, steers a live Visko Orbis video scene. While they settle, the exposure climbs one step at a time; when their pulse spikes it eases off; on panic the scene is replaced by a calm refuge within seconds.

Built for the Visko Orbis Online Challenge (September 2026). Nothing is pre-rendered, mocked or simulated.

Live: trycalmr.vercel.app · Launch post: on X · Dashboard, unlocked: trycalmr.vercel.app/app?access=bVj57bYQtVVGRQkHnFoxfpE396jJpbGh · Access code: bVj57bYQtVVGRQkHnFoxfpE396jJpbGh

_{Screen recordings of live Orbis sessions driven by the dashboard: the ladder sped up 3.6×, the panic refuge in real time. The heart rate here came from the labelled manual override.}

_{Their own places, image-to-world: the opening scene grows from a photo of a meeting room and the refuge from a photo of a beach. Sped up 2.4×.}

For judges

1. Open trycalmr.vercel.app/app?access=bVj57bYQtVVGRQkHnFoxfpE396jJpbGh in Chrome or Edge on a laptop. The link unlocks live sessions on its own. If you open /app without it, type the access code bVj57bYQtVVGRQkHnFoxfpE396jJpbGh into the box in the deck and press Unlock. No hardware is needed.
2. Pick a scenario and leave the heart-rate source on Manual. Press Calibrate: thirty seconds at the default 70 BPM sets the baseline.
3. Press Start. The first picture takes about ten seconds: six to connect, four more to the first frame. The line above the scene says what Calmr is doing and why; the ladder in the deck steps up about every twelve seconds while the pulse stays at baseline.
4. Drag the override up to about 95 BPM: the scene eases off a level. Press Panic: a calm refuge replaces it about six seconds later.
5. Release Panic and drag back down: the opening scene returns. Press Stop to open the session report.
6. On a second run, add a photo of a room under Their own places and press Start: the scene grows out of it. During the run, hold V and say "step up", "calm place" or a change like "the lights dim". Try Patient view too.

Every live second is paid for on our Reactor credits, so sessions are capped at five minutes and renew themselves while someone is at the controls (not from a background tab, and not after 10 minutes without a click or key). Leaving the page ends the session too. Please press Stop when you are done.

Why it has to be live video

Exposure therapy works by holding a patient in a feared situation until their body settles, then raising the challenge. Recorded video or VR can't do that: it plays the same way whatever the patient's heart is doing. Calmr's scene doesn't exist until the patient's physiology asks for the next step, and it can't be rendered ahead because nobody knows when the pulse will settle or spike. Orbis keeps one world running and changes it mid-stream, which is exactly the interaction exposure therapy needs. With image-to-world the world can even be the patient's own: the meeting room they dread, grown from a photo and filled with people one step at a time.

What's in it

• Real biofeedback, three honest sources. A Bluetooth heart-rate strap or watch (standard Heart Rate profile, with beat-to-beat intervals for HRV), webcam pulse estimated in the browser (MediaPipe face tracking and the POS algorithm), or a manual override that is labelled as such everywhere, including the report. Sources never stand in for each other: with no trustworthy reading the scene holds. The webcam shows its confidence and says when its signal is too weak to trust, and a reading expires as soon as its camera frames stop.
• Personal baseline. Thirty seconds of rest sets this person's resting heart rate. Zones are measured from it: settled up to +12 %, elevated to +30 %, high to +50 %, panic above. 85 BPM is high for someone resting at 60 and calm for someone resting at 80.
• Habituation driven by the body. In Auto, the ladder steps up only after the heart rate has come back down to half of its peak on the current step (the clinical habituation rule, applied to physiology) and the step has had time to show. A spike eases off one level. In Manual, the therapist clicks any rung, steps up or down, or types a direction of their own.
• A cause-and-effect line above the scene. It says, in plain words, what the heart rate is doing relative to baseline and what Calmr is doing about it: "Settled near baseline: stepping up to level 2 of 4 (Every chair filled)."
• Four exposure ladders, each checked on a live stream: Public Speaking, Heights, Claustrophobia and Open Water. Every step is one visible action.
• Panic refuge. One press, or a pulse 50 % above baseline, restarts the scene as a sunlit forest, meadow or beach behind a breathing overlay, on screen about six seconds later. It holds until the pulse settles, then the opening scene returns. If a session reaches its time cap while the calm place is on screen, it renews into the calm place, never back into the fear.
• Patient view. One click turns the player into a full-screen scene for the patient, with a paced breathing guide and a 0 to 10 anxiety rating (keys 0 to 9, + for 10). Where the browser does not allow full screen (iPhone Safari, an embedded page) it covers the whole window instead.
• Sound. Each level can carry a guided sound caption (a murmuring audience, wind past the railing, birdsong in the refuge), or the sound can be generated from the picture alone.
• Session report. Heart rate against the ladder over time with the refuge shaded, peak, mean, recovery after panic, highest level, step-ups and ease-offs, time per level, anxiety ratings and HRV. Export JSON or CSV, or print it as a PDF.
• Their own places (image-to-world). A photo of the patient's real feared place becomes the opening scene, and the ladder escalates inside it. A photo of their safe place becomes the refuge Panic opens. Photos are cropped to Orbis's 832×480 frame in the browser, re-encoded so location data is dropped, and uploaded only to Reactor when a scene starts from them.
• Speak it, see it. Push-to-talk voice for the therapist: hold V or the mic and say "step up", "ease off", "level 2", "calm place", "release" or "rating six". Any other sentence goes to the scene as the next prompt. Controls only match a whole utterance, so "a man steps up to the lectern" is a scene change, not a command.
• Proof in one frame. With the webcam as the source, the measured pulse, the tracked face regions and the raw pulse wave sit in a corner of the scene they steer. Record saves the whole tab as a video file on this computer; Save last 30 s downloads the scene itself as an MP4 from Reactor's recorder.
• Honest running costs. The cost meter multiplies streamed seconds by Reactor's live price, never a guess. Every session is capped (five minutes by default) and renews only for someone at the controls: never from a background tab, and not after 10 minutes without a click or key. Stop, closing the tab or leaving the page deletes the session on Reactor so billing ends, and an access code guards a public deployment.
• Works on a phone. Below 1000 px the dashboard becomes one column with a drawer menu holding every section, the steering mode and the session's quick actions. Start/Stop and Panic stay in a bar under the thumb, and the live scene pins to the top while the deck scrolls beneath it.

Dashboard. Telemetry, the cause-and-effect line, the live scene and the exposure ladder.	Session report. Heart rate against the ladder, with an honest note when the heart rate was set by hand.
Patient view. Only the scene, the breathing guide and a rating.	Claustrophobia, live: the airy room closes in; panic opens a clearing.

_{On a phone. One column with Stop and Panic under the thumb, and a drawer menu for the steering mode and every section. Live session, manual override.}

Image-to-world, live. Left: the photos as Calmr cropped them for Orbis. Right: the live scenes Orbis grew from them, the opening scene and the refuge.

More live stills: Public Speaking, Heights, Open Water.

How the steering works

Bluetooth strap ─┐
webcam rPPG ─────┼─▶ BPM ─▶ zone vs baseline ─▶ habituation engine ─▶ ladder step ─▶ set_prompt / restart ─▶ Orbis ─▶ live video
manual override ─┘                                (auto or manual)

Zone (vs baseline)	Auto does	Public Speaking on screen
Settled, within +12 %	step up once habituated	a few people walk in, then every chair fills, then the rear wall opens onto a hall
Elevated, +12 to +30 %	hold	the scene stays put
High, +30 to +50 %	ease off one level	the last step is undone
Panic, above +50 % or the button	restart into the refuge	a sunlit forest path about 6 s later

Before a baseline is calibrated, fixed bands apply (76, 106 and 136 BPM). Zone changes need 3 BPM of hysteresis and a two-second dwell. With no reading the ladder pauses.

What we learned about Orbis

Every rule above came from watching real streams.

• One visible action per prompt. "The walls recede as seats fill and the lights brighten" barely changed the room. "A few people walk in and sit down in the empty chairs" did.
• A change takes 10 to 15 seconds to show, and each set_prompt replaces the last. Steps that come faster than that get lost, so ladder steps wait for the scene.
• Big camera moves don't happen. "The camera rises up the side of the building" left us on the ground floor. The Heights scene now starts high and uses small moves: walk to the railing, look down.
• Morphing away from a crowded room took 10 to 25 seconds. Restarting generation (reset, a complete scene, start) in the same session shows a new scene in about 6 seconds, so panic restarts.
• The first chunk after start carries no frames. The overlay stays until real pixels arrive.
• Keep Reactor's prompt preparation. Sending the same text with passthrough: true was accepted faster but did not transform the scene.
• Give image-to-world the exact frame. Orbis resizes a starting image to 832×480 without cropping, so Calmr crops to that frame itself. The image anchors the whole run: text steps still work inside it, and people walked into the photo's meeting room on the first step.
• Upload once, reuse on every restart. The first upload and set_image took 1.8 s; reusing the upload later in the session took 0.1 s, so a panic into the patient's own safe place is barely slower than a stock refuge.
• set_image broadcasts its own conditions_ready. Calmr waits for it before sending the prompt, so it is never mistaken for the prompt's confirmation.
• Reactor's recorder gives the scene back. requestClip(30) returned the last 30 s as a 1080p, 30 fps MP4 with sound, ready about 20 s after the request.
• Check that a photo run used the photo. generation_started reports image_conditioned; Calmr logs it when a run given a photo started without one. The same event reports a budget of 2000 chunks per run, about an hour, longer than any session cap.
• Sound captions change mid-run. set_audio_prompt during generation applies from the next chunk, so each ladder step can carry its own sound.
• Leaving a page does not end its session. When Back or a new address put the dashboard in the browser's back-forward cache, Reactor still reported the session ACTIVE, and billing, until its cap. Calmr now asks the server to end it on every pagehide, and GET /sessions/{id} confirmed it CLOSED afterwards.

Measured on live sessions

Measurement	Value
Delivered frame rate (WebRTC stats)	18 fps at 2560×1440
Chunk cadence, where prompts land	1.81–1.85 s
Prompt accepted by Reactor	0.4–3.6 s (up to 12 s once on a slow network)
Start to first presented frame	4.0–4.1 s
Panic press to refuge on screen	4.6–6.4 s (6.8 s when the refuge grows from a photo)
Connect (token, session, WebRTC)	6–8 s
Photo upload and set_image, first time / reused	1.8 s / 0.1 s
Save last 30 s, from the request to the MP4	19.7 s (1920×1080, 30 fps, 27 MB)
Stop, including the server-side delete	about 1 s
Rate, from Reactor's pricing API	$0.0097 per second
Chunk budget per run (max_chunks)	2000, about 61 min

Setup

1. Create a Reactor account at reactor.inc and an API key (user icon → API Keys, starts with rk_).

2. Copy .env.example to .env.local:

   REACTOR_API_KEY=rk_...
   ORBIS_SESSION_SECONDS=300     # cap per session, 60..3600; renewed while someone is there
   CALMR_ACCESS_CODE=            # set on any public deployment
   

   The key is only read inside route handlers. The browser gets a session-scoped token that can open one session.

3. npm install, then npm run dev -- -p 3111. The landing page is http://localhost:3111 and the dashboard http://localhost:3111/app. Webcam and Bluetooth need localhost or HTTPS.

Node 20.9 or newer.

Deploying

Set REACTOR_API_KEY, ORBIS_SESSION_SECONDS and CALMR_ACCESS_CODE in the host's environment and serve over HTTPS. With an access code set, starting a live session requires it; /app?access=CODE unlocks it from a link and removes the code from the address bar. /api/token also allows at most 6 tokens per client and 20 overall per minute, and after 10 wrong codes in 10 minutes a client waits before it may try again. Use a long random code.

Scripts

Command	What it does
npm run dev -- -p 3111	dev server
npm run build / npm start	production build and server
npm run lint	ESLint
npx next typegen && npx tsc --noEmit	typecheck
node --test src/lib/*/*.test.ts	184 unit tests: habituation engine, baseline zones, photo framing and prompts, voice commands, pulse DSP, landmark smoothing and the inset, Bluetooth parsing and HRV, report statistics, tab recording

Layout

src/app/page.tsx                   landing route
src/components/landing/            landing page
src/app/app/page.tsx               dashboard route
src/app/api/                       token (access code, rate limit), session delete, access check, live pricing
src/lib/server/                    Reactor token minting and session delete, access code and rate limits
src/lib/steering/                  scenarios and ladders, habituation engine, baseline zones (+ tests)
src/lib/report/                    session record and statistics (+ tests)
src/lib/rppg/                      webcam pulse extraction (+ tests)
src/lib/ble/                       Bluetooth Heart Rate parsing and HRV (+ tests)
src/lib/image/                     photo framing for image-to-world (+ tests)
src/lib/voice/                     spoken command parser (+ tests)
src/lib/recording/                 tab recording formats and names (+ tests)
src/hooks/                         live session, steering, baseline, recorder, sensor, webcam, voice, tab recorder, fullscreen
src/components/                    dashboard, deck, phone menu and action bar, telemetry, cause-and-effect line, ladder, their own places, voice, pulse inset, capture, patient view, report
public/mediapipe/                  self-hosted face-tracking model and runtime
public/landing/, docs/media/       recordings and stills from live sessions

Honest limitations

• The recordings and stills above use the manual override so the steering is easy to follow. The Bluetooth parser is unit-tested against the GATT spec but has not yet been run with a strap on a patient.
• Webcam pulse is the least dependable source. In a test on a real face on 1 October (MacBook Pro camera, overhead room light, a phone reading 64 BPM), the camera's own exposure and colour adjustments, visible on the wall behind as well, were larger than the pulse, so Calmr showed "no signal" throughout rather than a number. Camera-based pulse is also known to give a weaker signal on darker skin. It needs steady, even light from the front and a still face; a strap is the dependable source.
• Orbis sometimes drifts: a scene can wander after many steps. The refuge restart and the opening-scene restart give it a clean start.
• Voice was tested end to end on live Orbis with a scripted stand-in for the browser's speech recogniser, because the test browser has no microphone. How well real speech is heard depends on the browser's speech service; in Chrome the audio goes to Google unless on-device recognition is available.
• The pulse inset has shown a real face and its colour wave, but no recording yet shows a webcam pulse driving the scene.
• Stop is disabled while a session is still connecting; Reset cancels it. The cost meter counts streamed time only, so it slightly undercounts the seconds spent connecting. A renewal at the time cap starts the ladder again from the opening step (or in the calm place, if that was on screen).
• Image-to-world was tested with public-domain photos. Reactor moderates content, so Calmr asks for photos of places, not people.
• The landing page background replays one live Calmr session at 1.5×: the meeting room grown from a photo fills with people while the heart rate is settled, holds when it rises, opens the forest refuge when it spikes and returns when it settles. The scene is that session's own output from Save last 30 s, joined at frame-exact overlaps, graded darker for legibility, with the two restarts shown as short dissolves; the readout beside it is that run's own heart rate and narrative, set with the manual override.
• Calmr is a prototype, not a medical device, and must not be used for clinical decisions. Pulse is estimated in the browser; no camera frames leave the device.

Credits

Test photos from Wikimedia Commons, both CC0: Quiet meeting room and Boscombe beach.