Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2644,6 +2644,23 @@ loadable play content. Everything a user does must be visible to connected peers
2 (every canvas panel >= 1 texel per Quest pixel). `api.claimInput('sticks')` + the world-pan stick REEL (stick Y
pushes/pulls a one-grip world, as Edit does an object); `addSetting({onLevels: true})` draws a choice as tabs on the
VR Levels page, which now lists every level.
- **ROADMAP 59 (59-hub) — TOOLS & CONNECTIONS, FEATURE STATUS, SKIES OFF THE BUNDLE.** `lib/tools/toolStatus.js` is a
LEAF registry (svelte/store only): `registerToolRows(section, readable)` per section (TOOL_SECTIONS order: chat,
mesh, animation, aibox, agents, desktop — a section shows once a source registers, so a later roadmap adds its
rows with ONE call) and `registerFeature({id, label, section, needs, fix?, ready, row})`. A row = status (`ready |
off | setup | error`), Test, Enable / Disable over the EXISTING switches (provider: active + master; voice:
additive `aiStt.enabled`, absent = on; desktop: `tools:blender`) and the features it turns on. NO KEY MOVED:
Settings ▸ AI became Settings ▸ Tools & connections (`AiSettings.svelte` keeps the provider forms as sub-pages,
`ToolsHub.svelte` is the main view; `'ai'` is a deep-link ALIAS in settingsNav). T5: a button whose tool is missing
STAYS VISIBLE — `toolLinks.explainMissing(id)` toasts "<label> needs <needs>: set one up in Settings ▸ Tools &
connections." and opens the row (`toolsFocus` scrolls + flashes it); the assistant's system prompt lists the same
missing features (`toolSummary`). `builtinTools.js` registers chat/voice/mesh/desktop; Test clients import lazily.
Skies (`hdri/skyFetch.js`): the five bundled `.hdr` live in the PACKS repo `hdri/` (jsDelivr, `VITE_HDRI_BASE` >
`VITE_PACKS_BASE` > packs@format-1); an export reads its own `hdri/<file>`; Cache Storage `tp-skies-v1` keeps a
picked sky (keyed by URL, older ref dropped); a CDN miss is `hdriStatus: 'unavailable'` = the flat preset + a retry
on `online`, never an error. e2e: `h.routeSkies` (OPT-IN — a route disables the context's HTTP cache) serves them
from a local packs checkout. Suites: vitest `toolStatus` (incl. the storage migration); e2e `tools-connections`,
`sky-cdn`.

## UI design system (redesign epic)

Expand Down
18 changes: 17 additions & 1 deletion docs/settings-storage-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,22 @@ Order: "VR override", then the whole `VRSettingsSection` (see its table), then t
| Fine-tune — Reset (`#nudge-reset`) | *action* `resetRoomNudge()` | deletes current room's entry in `colocation-nudge-v1` | — | DEVICE | lib/colocationNudge.js:153 | |
| Ghost hands (`#colocated-ghost-hands`) | `colocatedGhostHands` | `colocatedGhostHands` | `true` | DEVICE | lib/colocationPresence.js:69 (writer :249) | bool str, read `!== 'false'`; load-write |

### Tools & connections (was AI — 59 T4)

59 T4 renamed the page and turned its main view into the Tools & connections hub
(`components/menu/settings/ToolsHub.svelte` over `lib/tools/toolStatus.js` + `lib/tools/builtinTools.js`).
**No key moved**: every row below is still written by the same store and key. The hub's per-row
controls map onto them:

| UI element | Store | Key(s) | Default | Scope | Defined at | Notes |
|---|---|---|---|---|---|---|
| Chat / mesh provider row — Enable / Disable | `setAiActiveProvider(id)` + `setAiEnabled(on)` · `setMeshActiveProvider(id)` + `setMeshGenEnabled(on)` | `aiActiveProvider` + `aiEnabled` · `meshActiveProvider` + `meshGenEnabled` | — | DEVICE | lib/tools/builtinTools.js | Enable = make it active + master on; Disable = master off (the active pointer stays) |
| Voice typing row — Enable / Disable | `setSttConfig({enabled})` | `aiStt` (`enabled`) | absent = on | DEVICE | lib/ai/stt.js | stored only as `enabled: false`; absent on every config saved before 59 |
| Desktop tools ▸ Blender — Enable / Disable | `blenderEnabled` / `setBlenderEnabled` | `tools:blender` | absent = on | DEVICE | lib/tools/builtinTools.js | stored only as `'false'` |
| Any row — Test | *action* `runToolTest(row)` | none | — | — | lib/tools/toolStatus.js | the result is in memory (`toolTests`), never stored |

The rows of the old page, still the forms behind each row's *Set up*:

### AI

| Row label | store | storage key | default | scope | defined at | encoding / notes |
Expand Down Expand Up @@ -333,7 +349,7 @@ Each row writes through the row's `set()`; `put(store, key, v)` (settingsSchema.
| Face edit limit (`#vr-set-faceCap`, free number ≥ 10 on desktop) | `vrFaceCap` | `vrFaceCap` | `2500` (`VR_FACE_CAP`) | DEVICE | schema :305; lib/faceEdit.js:110 (const :107) | num str (int); load-write |
| Vertex edit limit (`#vr-set-vertexCap`, free number ≥ 10) | `vrVertexCap` | `vrVertexCap` | `800` (`VR_VERTEX_CAP`) | DEVICE | schema :316; lib/meshEdit.js:1779 (const :1776) | num str (int); load-write |

## AiSttSettings.svelte (inside AI)
## AiSttSettings.svelte (inside Tools & connections ▸ Voice typing ▸ Set up)

| Row label | store | storage key | default | scope | defined at | encoding / notes |
|---|---|---|---|---|---|---|
Expand Down
3 changes: 2 additions & 1 deletion scripts/export-manifest.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ function excluded(rel) {
/^llms(-full)?\.txt$/.test(rel) ||
/^cloud-plugin.*\.js$/.test(rel) ||
rel.startsWith('templates/') ||
// 37-hdri: ~7.5 MB of bundled HDRIs — buildExport copies only the one the scene shows
// 37-hdri: buildExport copies only the sky the scene shows (59 T3: from the CDN — the .hdr
// files left the bundle; the preview cards stay excluded here as before)
rel.startsWith('hdri/') ||
rel.startsWith('.')
);
Expand Down
2 changes: 2 additions & 0 deletions scripts/hdri-bake.cjs
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
#!/usr/bin/env node
// 59 T3: the .hdr files themselves live in the PACKS repo's hdri/ now (served by jsDelivr) — after a
// bake, move any copied .hdr there (never keep it in static/: the release zip must have none).
// 37-hdri — bake the bundled HDRIs: copy each source .hdr into static/hdri/, write a small
// tone-mapped PNG card next to it, and print the numbers the preset table in
// src/lib/hdri/catalog.js is written from (sun direction, flat fallback colours).
Expand Down
2 changes: 1 addition & 1 deletion scripts/third-party-notices.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ const VENDORED = [
{ name: 'Press Start 2P', version: 'static/fonts', license: 'OFL-1.1', url: 'https://fonts.google.com/specimen/Press+Start+2P', file: 'static/fonts/LICENSE-press-start-2p.txt', source: 'static/fonts' },
{ name: 'Noto Sans subsets (3D label fonts)', version: 'static/fonts/unicode-font-resolver', license: 'OFL-1.1', url: 'https://github.com/notofonts', file: 'static/fonts/unicode-font-resolver/LICENSE-noto.txt', source: 'static/fonts/unicode-font-resolver (scripts/fetch-troika-fonts.cjs)' },
{ name: 'unicode-font-resolver data', version: 'v1.0.1', license: 'MIT', url: 'https://github.com/lojjic/unicode-font-resolver', file: 'static/fonts/unicode-font-resolver/LICENSE-unicode-font-resolver.txt', source: 'static/fonts/unicode-font-resolver (scripts/fetch-troika-fonts.cjs)' },
{ name: 'Poly Haven HDRIs', version: 'static/hdri', license: 'CC0-1.0', url: 'https://polyhaven.com/license', file: 'static/hdri/CREDITS.md', source: 'static/hdri' }
{ name: 'Poly Haven HDRIs', version: 'static/hdri', license: 'CC0-1.0', url: 'https://polyhaven.com/license', file: 'static/hdri/CREDITS.md', source: 'static/hdri (cards) + the packs repo hdri/ (the .hdr files, downloaded on first use)' }
];

/** always in the bundle though nothing imports them by name (the compiler and the kit emit the imports) */
Expand Down
15 changes: 5 additions & 10 deletions src/components/menu/AddMenu.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
// Enter/click spawns at the menu's ground point and replicates.
import { minimalScroll } from '$lib/ui/minimalScroll.js';
import { untrack } from 'svelte';
import { addMenu, aiAssistantHidden, showToast, settingsOpen, settingsSection } from '../../stores/appStore.js';
import { aiReady } from '$lib/ai/providers';
import { addMenu, aiAssistantHidden } from '../../stores/appStore.js';
import { explainMissing } from '$lib/tools/toolLinks.js'; // 59 T5
import { primitivesCatalog } from '$lib/primitivesCatalog';
import { spawnAtPoint } from '$lib/addObjects';
import { rightDragMove, inputContextMenu } from '$lib/searchMenuUx';
Expand Down Expand Up @@ -102,16 +102,11 @@
dragged = false; // the next open re-anchors to wherever it is invoked from
}

// the menu's last row opens the AI assistant — the AiHudButton's own path (Settings › AI
// with a toast when no provider is set up yet), opening rather than toggling
// the menu's last row opens the AI assistant — the AiHudButton's own path (59 T5: a missing
// provider says so and opens its row in Tools & connections), opening rather than toggling
function openAi() {
close();
if (!aiReady()) {
showToast('Enable an AI provider in Settings to use the assistant');
settingsSection.set('ai');
settingsOpen.set(true);
return;
}
if (explainMissing('assistant')) return;
aiAssistantHidden.set('');
}

Expand Down
29 changes: 12 additions & 17 deletions src/components/menu/AiAssistant.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,7 @@
import {
aiAssistantHidden,
aiPromptBarOpen,
showToast,
settingsOpen,
settingsSection
showToast
} from '../../stores/appStore';
import WindowChrome from '../ui/WindowChrome.svelte';
import Icon from '../ui/Icon.svelte';
Expand All @@ -22,6 +20,8 @@
import { focusStack } from '$lib/windowFocus';
import { tabbable } from '$lib/windowTabs';
import { sttConfig, sttReady } from '$lib/ai/stt';
import { explainMissing, openToolRow } from '$lib/tools/toolLinks.js'; // 59 T5
import { derived } from 'svelte/store';
import { dictation, startDictation, stopDictation } from '$lib/ai/sttCapture';

let prompt = $state('');
Expand All @@ -30,6 +30,8 @@
let pillInput: any = $state(null);
let promptInput: any = $state(null);

/** 59 T5: the mic's tooltip says what is missing (the press opens its row) */
const voiceReady = derived(sttConfig, (c) => sttReady(c));
const hasProvider = $derived($aiProviders.length > 0 && !!$aiActiveProvider);
const pillVisible = $derived($aiPromptBarOpen && $aiEnabled && hasProvider && $aiAssistantHidden !== '');

Expand All @@ -52,12 +54,7 @@
return;
}
if ($dictation.state !== 'idle') return;
if (!sttReady($sttConfig)) {
showToast('Set up Voice typing in Settings ▸ AI to talk to the assistant', [
{ label: 'Open Settings', action: () => { settingsSection.set('ai'); settingsOpen.set(true); } }
]);
return;
}
if (explainMissing('voiceTyping')) return; // 59 T5: says what is missing, opens its row
const ok = await startDictation('desktop');
if (!ok && $dictation.error) showToast('Voice typing: ' + $dictation.error);
}
Expand Down Expand Up @@ -183,16 +180,14 @@
{/each}
</select>
{:else}
<!-- 38 NOTES-38 #24 (design page): the hint is the way there — the same Settings › AI
deep link the provider toast uses -->
<!-- 38 NOTES-38 #24 (design page): the hint is the way there — 59 T5: the Chat providers
row in Settings › Tools & connections, the same place the provider toast opens -->
<span class="ai-hint flex-1"
>No provider yet. <button
>No chat provider yet. <button
type="button"
id="ai-hint-setup"
class="ai-hint-link"
onclick={() => {
settingsSection.set('ai');
settingsOpen.set(true);
}}>Set one up in Settings › AI</button
onclick={() => openToolRow('chat')}>Set one up in Tools &amp; connections</button
></span
>
{/if}
Expand All @@ -215,7 +210,7 @@
class:ai-mic-on={$dictation.state === 'recording'}
aria-label={$dictation.state === 'recording' ? 'Stop voice typing' : 'Voice typing'}
aria-pressed={$dictation.state === 'recording'}
title={$dictation.state === 'recording' ? 'Stop and type what was said' : 'Voice typing (Settings ▸ AI)'}
title={$dictation.state === 'recording' ? 'Stop and type what was said' : $voiceReady ? 'Voice typing' : 'Voice typing needs a voice typing server — set one up in Settings ▸ Tools & connections'}
disabled={$aiBusy || $dictation.state === 'transcribing'}
onclick={toggleMic}
>
Expand Down
53 changes: 11 additions & 42 deletions src/components/menu/AiSttSettings.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,19 @@
// (safeStorage, plaintext like every AI key here).
//
// 37-settings (R21): the provider is a segmented control with its status as a badge; the server form
// is a SUB-PAGE ("AI › Voice typing server") reached by a NavRow. `subpage` renders that form (the
// is a SUB-PAGE ("AI › Voice typing server") reached by a NavRow. 59 T4: the NavRow is the Tools &
// connections hub's Voice typing row now, and the preset picker moved into the sub-page.
// `subpage` renders that form (the
// AI page passes it while the sub-page is open). Same store, same key (aiStt).
import { getContext } from 'svelte';
import Section from '../ui/Section.svelte';
import SettingRow from '../ui/SettingRow.svelte';
import NavRow from '../ui/NavRow.svelte';
import Segmented from '../ui/Segmented.svelte';
import Button from '../ui/Button.svelte';
import Badge from '../ui/Badge.svelte';
import { NAV_CONTEXT } from '$lib/settingsNav';
import { STT_PRESETS, sttConfig, setSttConfig, applySttPreset, sttReady, sttPresetFor, transcribe, describeSttError } from '$lib/ai/stt.js';
import { STT_PRESETS, sttConfig, setSttConfig, applySttPreset, sttReady, sttPresetFor, transcribe, describeSttError, silentWav } from '$lib/ai/stt.js';

/** @type {{subpage?: boolean}} */
let { subpage = false } = $props();
const nav = /** @type {any} */ (getContext(NAV_CONTEXT));

/** @type {{ok: boolean, detail: string} | null} */
let testResult = $state(null);
Expand All @@ -33,28 +31,6 @@
/** @param {Event} e */
const value = (e) => /** @type {HTMLInputElement} */ (e.currentTarget).value;

/** half a second of silence as a 16 kHz mono WAV — enough for the server to answer, nothing to transcribe */
function silentWav() {
const samples = 8000;
const buffer = new ArrayBuffer(44 + samples * 2);
const view = new DataView(buffer);
const text = (/** @type {number} */ at, /** @type {string} */ s) => [...s].forEach((c, i) => view.setUint8(at + i, c.charCodeAt(0)));
text(0, 'RIFF');
view.setUint32(4, 36 + samples * 2, true);
text(8, 'WAVE');
text(12, 'fmt ');
view.setUint32(16, 16, true);
view.setUint16(20, 1, true);
view.setUint16(22, 1, true);
view.setUint32(24, 16000, true);
view.setUint32(28, 32000, true);
view.setUint16(32, 2, true);
view.setUint16(34, 16, true);
text(36, 'data');
view.setUint32(40, samples * 2, true);
return new Blob([buffer], { type: 'audio/wav' });
}

async function test() {
testing = true;
testResult = null;
Expand All @@ -70,18 +46,17 @@
const preset = $derived(sttPresetFor($sttConfig.preset));
const ready = $derived(sttReady($sttConfig));
const PRESETS = STT_PRESETS.map((p) => ({ value: p.preset, label: p.preset === 'custom' ? 'Self-hosted' : p.label, title: p.label }));
/** a host for the NavRow's value @param {string} url */
const hostOf = (url) => {
try {
return new URL(url).host;
} catch {
return url || 'not set';
}
};
</script>

<div id="ai-stt-settings" class="contents" data-keywords={keywords.join(' ')}>
{#if subpage}
<!-- 59 T4: Tools & connections ▸ Voice typing ▸ Set up — the provider picker sits above the form -->
<Section variant="card" label="Voice typing" badge="This device">
<SettingRow id="row-stt-preset" label="Voice typing provider" description="Speech to text for the assistant’s mic, in the AI window and in VR." wide>
<Segmented id="ai-stt-preset" label="Voice typing provider" options={PRESETS} value={$sttConfig.preset} onchange={(v) => { applySttPreset(v); testResult = null; }} />
<span id="ai-stt-status"><Badge tone={ready ? 'ok' : 'warn'} text={ready ? 'Ready' : $sttConfig.enabled === false ? 'Switched off' : preset.needsKey ? 'Needs an API key' : 'Needs a base URL'} /></span>
</SettingRow>
</Section>
<Section variant="card" label="Voice typing server" badge="This device">
<SettingRow id="row-stt-base" label="Base URL" description="The OpenAI-compatible endpoint, ending in /v1." wide>
<input id="ai-stt-base" class="settings-text" placeholder="https://…/v1" autocomplete="off" value={$sttConfig.baseUrl} onchange={(e) => setSttConfig({ baseUrl: value(e) })} />
Expand All @@ -105,12 +80,6 @@
{/snippet}
</SettingRow>
</Section>
{:else}
<SettingRow id="row-stt-preset" label="Voice typing provider" description="Speech to text for the assistant’s mic, in the AI window and in VR." wide>
<Segmented id="ai-stt-preset" label="Voice typing provider" options={PRESETS} value={$sttConfig.preset} onchange={(v) => { applySttPreset(v); testResult = null; }} />
<span id="ai-stt-status"><Badge tone={ready ? 'ok' : 'warn'} text={ready ? 'Ready' : preset.needsKey ? 'Needs an API key' : 'Needs a base URL'} /></span>
</SettingRow>
<NavRow id="ai-stt-server" label="Voice typing server" value={hostOf($sttConfig.baseUrl)} onclick={() => nav.openSub('ai:stt', 'Voice typing server', 'ai')} />
{/if}
</div>

Expand Down
14 changes: 5 additions & 9 deletions src/components/menu/Controls.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,12 @@
import Icon from '../ui/Icon.svelte';
import { createAttachmentKey, fromAction } from 'svelte/attachments';
// 41 G1: MobileAddButton / AiHudButton (and VoiceChat's mic button) became roster entries
import { aiAssistantHidden, settingsOpen, settingsSection, viewportMenuOpener } from '../../stores/appStore.js';
import { aiAssistantHidden, viewportMenuOpener } from '../../stores/appStore.js';
// 42 I9: the top-right group's and the touch tools' buttons join the roster
import { notesDrawerOpen, multiSelectMode, touchTools } from '../../stores/appStore.js';
import { undo, redo } from '$lib/history';
import { toolbarSlots, TOP_DEFAULT, TOUCH_DEFAULT, TOP_NATIVE, TOUCH_NATIVE } from '$lib/toolbarSlots';
import { aiReady } from '$lib/ai/providers';
import { explainMissing } from '$lib/tools/toolLinks.js'; // 59 T5
import { canvasCenter } from '$lib/canvasRect';
import { micActive, pttActive, toggleMic } from '$lib/voiceChat';
import { phoneShell } from '$lib/ui/phoneShell.js';
Expand Down Expand Up @@ -1372,14 +1372,10 @@
return id === SPACER ? 'Play' : (BUTTONS[id]?.label ?? BUTTONS[id]?.title ?? id);
}

/** the AI button press — AiHudButton's own path (I3): unconfigured points at Settings ▸ AI */
/** the AI button press — AiHudButton's own path (I3); 59 T5: unconfigured says what is missing
* and opens its row in Settings ▸ Tools & connections */
function toggleAiAssistant() {
if (!aiReady()) {
showQualityToast('Enable an AI provider in Settings to use the assistant');
settingsSection.set('ai');
settingsOpen.set(true);
return;
}
if (explainMissing('assistant')) return;
aiAssistantHidden.set($aiAssistantHidden === '' ? 'hidden' : '');
}

Expand Down
7 changes: 4 additions & 3 deletions src/components/menu/Settings.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
import ImportSettingsSection from './ImportSettingsSection.svelte'; // 46 P2
import PrivacySettings from './settings/PrivacySettings.svelte'; // 48 O1
import VRSettingsSection from './VRSettingsSection.svelte'; // the whole VR page
import AiSettings from './settings/AiSettings.svelte';
import AiSettings from './settings/AiSettings.svelte'; // 59 T4: the Tools & connections page (+ its provider forms)
import ConnectionSettings from './settings/ConnectionSettings.svelte';
import { settingsOpen, settingsSection, settingsSearchSeed, hidePanels, restorePanels, showToast } from '../../stores/appStore.js';
import { cloudPluginInfo } from '$lib/cloudHooks';
Expand Down Expand Up @@ -115,7 +115,8 @@
$: if ($settingsOpen) {
stepPanelsAside();
shortcutsExpanded = $settingsSection === 'shortcuts';
aiExpanded = $settingsSection === 'ai';
// 59 T4: Settings ▸ AI became Tools & connections ('ai' stays a deep-link alias)
aiExpanded = $settingsSection === 'ai' || String($settingsSection ?? '').startsWith('tools');
sceneExpanded = $settingsSection === 'scene';
connectionExpanded = $settingsSection === 'connection';
vrExpanded = $settingsSection === 'vr';
Expand Down Expand Up @@ -567,7 +568,7 @@
<VRSettingsSection />
</AccordionItem>
<AccordionItem bind:open={aiExpanded}>
{#snippet header()}AI{/snippet}
{#snippet header()}Tools & connections{/snippet}
<AiSettings />
</AccordionItem>
<AccordionItem bind:open={importExpanded}>
Expand Down
Loading
Loading