Skip to content

[feat] 57-chars: humanoid map + Apply clip, VRM characters, rig fix-ups, Download…, licence warnings, Avatars source - #427

Merged
AlexZ005 merged 12 commits into
feat/57from
feat/57-chars
Oct 23, 2026
Merged

AlexZ005 merged 12 commits into
feat/57from
feat/57-chars

Conversation

@AlexZ005

Copy link
Copy Markdown
Collaborator

Roadmap 57, lane 57-chars: M2, M4, M5 (Avatars source), M6, M7, M9. Packs half: theprototype-app/packs#18.

M2 humanoid map + retarget

  • Every rig's bones are mapped onto VRM 1.0's humanoid bone set (src/lib/humanoid/humanoidMap.js, a leaf). Detection works from bone names (Mixamo, KayKit, Meshy, VRoid/UniRig, Unreal, Rigify/Quaternius UAL), then from the hierarchy (spine order), then from the rig's shape (numbered UniRig bones). A VRM brings its own map.
  • retarget.js: a rest-pose and facing corrected retarget; the hips' travel is scaled to the character's size.
  • Apply clip… from a dialog, Inspector ▸ Character, the object menu, and the Animation window's "Apply to…". Every peer computes the fitted clip for itself, so the model's bytes never change and a late joiner computes the same clip. Removing an applied clip undoes cleanly.
  • The map can be fixed in Skeleton ▸ Humanoid map. A fix is a replicated config (userData.character); the file is untouched.

M4 VRM characters

  • @pixiv/three-vrm 3.5.5 (exact version) loads as a lazy chunk. It reads VRM 0.x and 1.0, including a .glb that carries the VRM extension. Spring bones and expressions work, there's a Set Expression flow node, and the licence meta shows in the Inspector and becomes the model's provenance licence. VRM 0.x models are turned to face +Z.

M6 rig fix-ups (lazy)

  • Rename, move in the rest pose, add, delete and mirror bones. Weights: automatic (heat diffusion in a worker), smooth, normalize, plus a weight brush and a test pose.
  • Each operation rewrites the GLB, sends it to peers once (objectfile replace:true) and undoes as one step.

M7 provenance + licence warnings

  • provenance.js records where every asset came from. Licence warnings show in Publish, Export, Community gallery and Download; they warn and never block. Exports and download zips include a CREDITS.txt.

M9 Download…

  • Layouts: (a) one GLB with the clips embedded (default), (b) model + one .tpanim per clip, (c) model only.
  • A single clip downloads as .tpanim, .glb, .vrma or .bvh (lazy BVH writer).
  • FBX/USD are listed but switched off, with the reason shown. Several selected objects download as one zip.

M5 Asset Browser ▸ Avatars

  • Reads the Open Source Avatars registry, pinned to a commit via jsDelivr. CC0 collections are listed by default and CC-BY ones only on request; a CC-BY avatar keeps its credit. Production sends no CSP header, so no cloud change is needed.

Proof

  • Unit: vitest 207 files / 2,532 tests pass. New tests cover humanoidMap, retarget, provenance, characterDownload (including a BVH round trip), rigFixes and osaRegistry.

  • svelte-check ratchet held at 299/39.

  • e2e (from a snapshot, port 5422):

    Suite Result
    character-apply ALL PASS, 22 (three peers incl. a late joiner, undo)
    character-vrm ALL PASS, 26 (two peers)
    character-download ALL PASS, 28
    character-skeleton ALL PASS, 24 (two peers)
    character-avatars-source ALL PASS, 8
  • Production build: three-vrm, springs, rig fix-ups, BVH writer, retarget and VRMA writer all stay off the boot graph (boot is 368 chunks / 8,076,062 bytes).

For the 57-clips union

  • This branch doesn't contain 57-clips' M1 yet. The planned merge (QUESTIONS-57-chars Q5):
    • register this detector through their setHumanoidDetector
    • wire setClipAssetResolver to their decodeTpanim
    • use retargetClip inside their bindClip
    • fold the duplicate rest-pose readers into one
    • write .tpanim through their encodeTpanim
  • The character config moved to userData.character because they use userData.humanoid as a plain bone map.

🤖 Generated with Claude Code

AlexZ005 and others added 12 commits October 23, 2026 04:52
…chy, shape) + rest-corrected retarget core

- humanoid/humanoidMap.js (leaf): the VRM 1.0 bone table, a name reader for Mixamo / KayKit /
  Meshy / VRoid (UniRig) / Unreal-Quaternius conventions, the spine order read off the hips->head
  path (Meshy writes it top-down), a structural fallback for numbered UniRig bones, overrides
- humanoid/retarget.js: SkeletonUtils.retargetClip's method (world rotations sampled at a fixed
  fps, hips travel scaled by size) with the rest-pose correction and a facing correction, on
  plain snapshots so every peer computes the same clip locally
- unit: 10 map + 6 retarget cases (A-pose target, -Z facing, 2x size, .bones[] tracks)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- rigs WITHOUT clips (a VRM, a Meshy / UniRig character, the KayKit avatars) ride the raw-bytes
  objectfile path like animated ones (fileHandler / importOptimize hasSkin)
- animatedImports: rest pose captured at registration, RUNTIME clips (computed per peer, never in
  the bytes), registration listeners; humanoid config / provenance / VRM expressions ride the bytes
- humanoid/humanoidRig.js (lazy): map = file's (VRM) or detected + user fixes; Apply clip from a
  scene model or the built-in KayKit motions; config replicates (objectParameters 'humanoid'),
  saves, undoes; every peer + late joiner retargets for itself
- Apply clip dialog, Inspector > Character (rig, applied clips, licence, VRM meta + expression),
  object menu Character > Apply clip..., Animation window 'Apply to...'
- vrm/vrmRuntime.js (lazy chunk, @pixiv/three-vrm 3.5.5 MIT): VRM 0.x + 1.0 (.vrm or a .glb with
  the extension), spring bones + expressions ticked after the mixers, own humanoid map, licence meta
  -> provenance; Set Expression flow node (deterministic on the trigger stamp)
- provenance.js (leaf): {tool, model, version, licence, prompt?, date} records, licence flags
  (credit / NC / no-redistribution / ND), warnings, credits text, VRM meta -> licence
- unit: provenance (5); e2e: character-apply, character-vrm (fixtures built in-test)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…lication

A clip is its own small file (fork F1). Humanoid clips are valid VRM Animation
(VRMC_vrm_animation 1.0), written in VRM's normalized space so three-vrm reads them back
unchanged and one file plays on any mapped humanoid; other clips are a plain GLB of the
source skeleton + one animation, bound by node name. Both carry extras.theprototype
{format, kind, skeleton {sig, humanoid}, fps, frames, loop, rootMotion, provenance}.

- src/lib/tpanim/: glb.js (hand-written deterministic GLB container), format.js
  (encode/decode, three clip <-> TpClip, toHumanoid / humanoidFrames / bindHumanoid,
  signatures, fit), humanoidBones.js (VRM 1.0 bone set), humanoidMap.js (the detector
  seam 57-chars M2 registers into; Mixamo/VRM-name fallback), clipAssets.js (lazy binder:
  hash -> decoded -> bound action, hudImages-style pull + retry, scene-asset source),
  convert.js (lazy: .vrma / .bvh / animation-only .glb + .fbx -> .tpanim on import),
  clipOps.js (resample, in place, seam, loop, trim/split/join — used by M3/M8).
- animatedImports: userData.clipRefs [{hash,name,speed?,loop?,mode?}] ride the
  objectfile message, save entries and undo; one objectParameters 'clipRefs' message + one
  'clipRefs' undo entry per change; the model bytes never change. Rest pose captured at
  register; crossfade-ready transport (at/from/fade) and local pose drivers.
- Rigged models with no clips import as animated imports so clips can be assigned.
- Explorer: 'anim' kind (Animation clips), conversion on import, drop a clip on a rigged
  model to assign + play it.
- Tests: unit tpanimFormat (round trip, determinism, humanoid bind on other rigs, VRMA
  read by three-vrm's own loader incl. a foreign non-identity-rest VRMA), tpanimOps,
  wireValidate clipRefs; e2e tpanim-clips (host + peer + late joiner, real KayKit assets).
- devDependency @pixiv/three-vrm-animation (unit test only).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… writer; M7 warnings in Export/Download/Publish

- download/glbEdit.js (leaf): edit a GLB's own JSON + BIN (append / strip animations) so the model
  and a VRM's extensions stay byte-for-byte what was imported
- download/clipFiles.js: a clip as its own file in the 57 M1 .tpanim shape (skeleton + one animation
  + extras.theprototype; humanoid = VRMC_vrm_animation), layout (a) clips embedded in the original bytes
- download/bvhWriter.js (lazy): world rotations folded into ZXY channels, offsets from rest, root
  offset zero - round trip through three's BVHLoader puts every joint where the rig does
- download/characterDownload.js + DownloadDialog: (a) one GLB (default), (b) model + .tpanim zip,
  (c) model only, one clip as .tpanim / .glb / .vrma / .bvh, FBX / USD listed off with the reason,
  several objects as one zip with CREDITS.txt; Object > Download..., Inspector > Character
- M7: LicenceWarnings in Publish / Export / Community gallery / Download (warn, never block);
  sceneLicences.js counts applied clips' licences; exports write CREDITS.txt
- unit: characterDownload (8); e2e: character-download

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the Skeleton view

- rig/rigOps.js: rename / move in the rest pose / add / delete a bone, mirror left <-> right,
  write skin weights - each edits the GLB itself and KEEPS THE BIND (newIBM = inv(newWorld) *
  oldWorld * oldIBM), so every skinned vertex stays where it was
- rig/weightsCore.js (leaf) + weightsWorker.js: automatic weights = bone-heat seeding (nearest bone)
  diffused over the welded SURFACE (never across air), smooth, normalize (<= 4), the brush
- rig/rigFixes.js: one operation = one new file, replaced on every peer (objectfile replace:true)
  and undoable (history 'rigbytes'); a brush stroke paints locally and is written on lift; test pose
  local; skeleton helper at the scene root
- SkeletonWindow (floating, lazy): Humanoid map (per-bone select, detected vs fixed, reset; a
  replicated config), Bones, Weights + brush; Inspector > Character > Skeleton..., object menu
- animatedImports: glTF primitive stamps per skinned mesh, replaceAnimatedBytes
- unit: rigFixes (10: bind kept for every op, surface heat); e2e: character-skeleton

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nitized .L/.R) and skip UE5 leaf tips + finger metacarpals

- the UAL 1 Standard rig is Rigify (DEF-upper_arm.L -> 'DEF-upper_armL' once three drops the '.'),
  UAL 2 is Unreal-named with _leaf_ tip bones that were read as finger roots
- unit: a Rigify case (raw + sanitized) and UE5 leaf / metacarpal bones

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… positions; converter tests

- toHumanoid: a rig whose REST hips sit at the floor (a BVH keeps the height in the motion)
  stands at its first frame — VRMA scales the hips by the rest height, and zero means nothing
- convert (BVH): centimetres detected from the first frame's root height, not the rest pose
- tpToThreeClip(tp, root): each node name resolves against THAT model (GLTFLoader sanitizes
  'Door Left' -> 'Door_Left'; an object named in the app keeps its name); clipFit matches both
- nodePositionsAt(tp, t): the file's own skeleton posed at t (thumbnails, model-less previews)
- unit: tpanimConvert (BVH -> humanoid in metres, an animation GLB split per animation, .vrma
  keeps its humanoid, a model GLB is left alone), nodePositionsAt

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rce Avatars registry)

- avatars/osaRegistry.js: the registry at a PINNED commit through the CDN and the CC0 list cache;
  CC0 collections by default, CC-BY only on request; VRM + public + https rows only
- Add to scene places the VRM as a character with a provenance step naming the collection's licence
  and author (fileHandler importFile opts.provenance), so a CC-BY avatar's credit reaches the M7
  warnings and CREDITS.txt
- unit: osaRegistry (2); e2e: character-avatars-source (routed registry + VRM fixture)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rData.humanoid is 57-clips' map key)

- 57-clips M1 reads root.userData.humanoid as a plain {vrmBone: node} map; this lane's config
  ({map fixes, applied clips}) moves to userData.character, and its wire parameter to 'character',
  so the two never read each other's shape at the union

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… by raw node names

The Explorer's clip cards get a 2D stick-figure thumbnail (lazy, no WebGL context per card);
double-clicking a clip plays it on the selected character; bindClip passes the model so
node names that sanitize differently still bind; the unsupported-file toast names clips.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…poke; character e2e measure what they claim

- SkeletonWindow: the helpers effect depended on `info` (derived from sceneRevision) while
  refreshHelpers pokes the scene, so it re-ran until svelte stopped it and the window never
  rendered; it is keyed on a string of the bones, their rest places and the live root id now
- the Bones fields (rename, rest) follow the picked bone and the rig, not every scene poke,
  so a test-pose drag no longer wipes a name being typed
- e2e: FULL_RESOURCE_TIMING init script (the 250-entry buffer fills during a dev boot, so every
  "lazy chunk fetched" check read false and every "not loaded at boot" check passed vacuously)
- character-vrm: a two-bone spring chain is ONE three-vrm joint; the swing is sampled while the
  body moves (the damped spring settles before a 600 ms sample)
- character-download: the (b) zip carries CREDITS.txt, crediting the applied clip's source
- character-skeleton: prints the window's DOM + page errors when the map does not show

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… two halves

- merge origin/feat/57-clips-m1 (PR #425): conflicts in animatedImports (rootDataFields carries
  character/provenance/vrmExpressions AND clipRefs; both register hooks), commandsHandler (the
  'character' / 'expression' / 'clipRefs' parameters side by side, M3's stamp + crossfade kept),
  fileHandler (both lanes routed clipless rigs to the bytes path), wireValidate, package-lock
- a humanoid clip asset binds through the M2 retarget: clipAssets loads humanoidRig before the
  first humanoid bind and calls retargetTpClip (rest pose + facing corrected, hips scaled, the
  character's own map incl. the user's fixes); bindHumanoid stays the fallback
- humanoidRig registers its detector through setHumanoidDetector (a VRM's file map, else the
  detected map with the user's fixes)
- tpSnapshot (retarget.js, a leaf): a decoded .tpanim's skeleton as a retarget snapshot
- Apply clip… lists the Explorer's clip files; applying one assigns it as a clip asset on each
  target (one mechanism per kind of source; non-humanoid targets are refused with the reason)
- Download: .tpanim / .vrma are written by 57 M1's encodeTpanim(toHumanoid(tpFromThreeClip(...)))
  so every clip file the app writes has one shape; the single-clip .glb keeps the rig's skeleton
- unit tpanimRetarget: a clip through encode -> decode -> retarget comes back onto its own rig as
  the same pose (arms resting turned 35 degrees)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AlexZ005

Copy link
Copy Markdown
Collaborator Author

5e5e9d4 — merged 57-clips M1 (#425) and joined the two halves. This PR now carries #425 too; merging it first is fine, as is merging this one alone.

  • Their humanoid clip assets (.tpanim) now bind through this lane's retarget: bindClip → new retargetTpClip (corrects for rest pose and facing; bindHumanoid is kept as the fallback). The character's own map, including the user's fixes, is registered through setHumanoidDetector.
  • Apply clip… lists the Explorer's .tpanim files under Clip files and applies them as their clip assets (assignClipAsset, one history batch). Rigs that aren't humanoid are refused.
  • Download: .tpanim and .vrma are now written by encodeTpanim(toHumanoid(...)), so every clip file the app writes has one shape. The single-clip .glb keeps the rig's own skeleton.
  • Merge trap: both lanes added hasSkin() to fileHandler, and git merged them into a duplicate declaration. Only npm run build caught it; I dropped one copy.
  • Left as is: restRowsOf and restPoseOf both still exist (folding them is cleanup).

Checks on 5e5e9d4:

  • Unit: 2,562 of 2,563 passed. The one failure is a tankSurfaces timeout under load average about 30; it passes 12/12 alone. New: tpanimRetarget (2 tests).
  • svelte-check ratchet: 299/39.
  • Build: OK, 369 chunks / 8,089,493 bytes, +1 chunk / +13.4 KB, all from M1. The VRM, retarget and encoder code all stay lazy.
  • e2e: all pass, 146 checks. character-clip-assets 13 (new: two peers, a Mixamo Kick as a humanoid .tpanim played on the KayKit knight, undo on both), tpanim-clips 25, character-apply 22, character-download 28, character-vrm 26, character-skeleton 24, character-avatars-source 8.

@AlexZ005
AlexZ005 merged commit 28a68d1 into feat/57 Oct 23, 2026
8 of 9 checks passed
@AlexZ005
AlexZ005 deleted the feat/57-chars branch October 23, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant