Repository navigation
[feat] 57-chars: humanoid map + Apply clip, VRM characters, rig fix-ups, Download…, licence warnings, Avatars source - #427
Merged
Merged
Conversation
…chy, shape) + rest-corrected retarget core - humanoid/humanoidMap.js (leaf): the VRM 1.0 bone table, a name reader for Mixamo / KayKit / Meshy / VRoid (UniRig) / Unreal-Quaternius conventions, the spine order read off the hips->head path (Meshy writes it top-down), a structural fallback for numbered UniRig bones, overrides - humanoid/retarget.js: SkeletonUtils.retargetClip's method (world rotations sampled at a fixed fps, hips travel scaled by size) with the rest-pose correction and a facing correction, on plain snapshots so every peer computes the same clip locally - unit: 10 map + 6 retarget cases (A-pose target, -Z facing, 2x size, .bones[] tracks) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- rigs WITHOUT clips (a VRM, a Meshy / UniRig character, the KayKit avatars) ride the raw-bytes
objectfile path like animated ones (fileHandler / importOptimize hasSkin)
- animatedImports: rest pose captured at registration, RUNTIME clips (computed per peer, never in
the bytes), registration listeners; humanoid config / provenance / VRM expressions ride the bytes
- humanoid/humanoidRig.js (lazy): map = file's (VRM) or detected + user fixes; Apply clip from a
scene model or the built-in KayKit motions; config replicates (objectParameters 'humanoid'),
saves, undoes; every peer + late joiner retargets for itself
- Apply clip dialog, Inspector > Character (rig, applied clips, licence, VRM meta + expression),
object menu Character > Apply clip..., Animation window 'Apply to...'
- vrm/vrmRuntime.js (lazy chunk, @pixiv/three-vrm 3.5.5 MIT): VRM 0.x + 1.0 (.vrm or a .glb with
the extension), spring bones + expressions ticked after the mixers, own humanoid map, licence meta
-> provenance; Set Expression flow node (deterministic on the trigger stamp)
- provenance.js (leaf): {tool, model, version, licence, prompt?, date} records, licence flags
(credit / NC / no-redistribution / ND), warnings, credits text, VRM meta -> licence
- unit: provenance (5); e2e: character-apply, character-vrm (fixtures built in-test)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…lication
A clip is its own small file (fork F1). Humanoid clips are valid VRM Animation
(VRMC_vrm_animation 1.0), written in VRM's normalized space so three-vrm reads them back
unchanged and one file plays on any mapped humanoid; other clips are a plain GLB of the
source skeleton + one animation, bound by node name. Both carry extras.theprototype
{format, kind, skeleton {sig, humanoid}, fps, frames, loop, rootMotion, provenance}.
- src/lib/tpanim/: glb.js (hand-written deterministic GLB container), format.js
(encode/decode, three clip <-> TpClip, toHumanoid / humanoidFrames / bindHumanoid,
signatures, fit), humanoidBones.js (VRM 1.0 bone set), humanoidMap.js (the detector
seam 57-chars M2 registers into; Mixamo/VRM-name fallback), clipAssets.js (lazy binder:
hash -> decoded -> bound action, hudImages-style pull + retry, scene-asset source),
convert.js (lazy: .vrma / .bvh / animation-only .glb + .fbx -> .tpanim on import),
clipOps.js (resample, in place, seam, loop, trim/split/join — used by M3/M8).
- animatedImports: userData.clipRefs [{hash,name,speed?,loop?,mode?}] ride the
objectfile message, save entries and undo; one objectParameters 'clipRefs' message + one
'clipRefs' undo entry per change; the model bytes never change. Rest pose captured at
register; crossfade-ready transport (at/from/fade) and local pose drivers.
- Rigged models with no clips import as animated imports so clips can be assigned.
- Explorer: 'anim' kind (Animation clips), conversion on import, drop a clip on a rigged
model to assign + play it.
- Tests: unit tpanimFormat (round trip, determinism, humanoid bind on other rigs, VRMA
read by three-vrm's own loader incl. a foreign non-identity-rest VRMA), tpanimOps,
wireValidate clipRefs; e2e tpanim-clips (host + peer + late joiner, real KayKit assets).
- devDependency @pixiv/three-vrm-animation (unit test only).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… writer; M7 warnings in Export/Download/Publish - download/glbEdit.js (leaf): edit a GLB's own JSON + BIN (append / strip animations) so the model and a VRM's extensions stay byte-for-byte what was imported - download/clipFiles.js: a clip as its own file in the 57 M1 .tpanim shape (skeleton + one animation + extras.theprototype; humanoid = VRMC_vrm_animation), layout (a) clips embedded in the original bytes - download/bvhWriter.js (lazy): world rotations folded into ZXY channels, offsets from rest, root offset zero - round trip through three's BVHLoader puts every joint where the rig does - download/characterDownload.js + DownloadDialog: (a) one GLB (default), (b) model + .tpanim zip, (c) model only, one clip as .tpanim / .glb / .vrma / .bvh, FBX / USD listed off with the reason, several objects as one zip with CREDITS.txt; Object > Download..., Inspector > Character - M7: LicenceWarnings in Publish / Export / Community gallery / Download (warn, never block); sceneLicences.js counts applied clips' licences; exports write CREDITS.txt - unit: characterDownload (8); e2e: character-download Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the Skeleton view - rig/rigOps.js: rename / move in the rest pose / add / delete a bone, mirror left <-> right, write skin weights - each edits the GLB itself and KEEPS THE BIND (newIBM = inv(newWorld) * oldWorld * oldIBM), so every skinned vertex stays where it was - rig/weightsCore.js (leaf) + weightsWorker.js: automatic weights = bone-heat seeding (nearest bone) diffused over the welded SURFACE (never across air), smooth, normalize (<= 4), the brush - rig/rigFixes.js: one operation = one new file, replaced on every peer (objectfile replace:true) and undoable (history 'rigbytes'); a brush stroke paints locally and is written on lift; test pose local; skeleton helper at the scene root - SkeletonWindow (floating, lazy): Humanoid map (per-bone select, detected vs fixed, reset; a replicated config), Bones, Weights + brush; Inspector > Character > Skeleton..., object menu - animatedImports: glTF primitive stamps per skinned mesh, replaceAnimatedBytes - unit: rigFixes (10: bind kept for every op, surface heat); e2e: character-skeleton Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nitized .L/.R) and skip UE5 leaf tips + finger metacarpals - the UAL 1 Standard rig is Rigify (DEF-upper_arm.L -> 'DEF-upper_armL' once three drops the '.'), UAL 2 is Unreal-named with _leaf_ tip bones that were read as finger roots - unit: a Rigify case (raw + sanitized) and UE5 leaf / metacarpal bones Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… positions; converter tests - toHumanoid: a rig whose REST hips sit at the floor (a BVH keeps the height in the motion) stands at its first frame — VRMA scales the hips by the rest height, and zero means nothing - convert (BVH): centimetres detected from the first frame's root height, not the rest pose - tpToThreeClip(tp, root): each node name resolves against THAT model (GLTFLoader sanitizes 'Door Left' -> 'Door_Left'; an object named in the app keeps its name); clipFit matches both - nodePositionsAt(tp, t): the file's own skeleton posed at t (thumbnails, model-less previews) - unit: tpanimConvert (BVH -> humanoid in metres, an animation GLB split per animation, .vrma keeps its humanoid, a model GLB is left alone), nodePositionsAt Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rce Avatars registry) - avatars/osaRegistry.js: the registry at a PINNED commit through the CDN and the CC0 list cache; CC0 collections by default, CC-BY only on request; VRM + public + https rows only - Add to scene places the VRM as a character with a provenance step naming the collection's licence and author (fileHandler importFile opts.provenance), so a CC-BY avatar's credit reaches the M7 warnings and CREDITS.txt - unit: osaRegistry (2); e2e: character-avatars-source (routed registry + VRM fixture) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rData.humanoid is 57-clips' map key)
- 57-clips M1 reads root.userData.humanoid as a plain {vrmBone: node} map; this lane's config
({map fixes, applied clips}) moves to userData.character, and its wire parameter to 'character',
so the two never read each other's shape at the union
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… by raw node names The Explorer's clip cards get a 2D stick-figure thumbnail (lazy, no WebGL context per card); double-clicking a clip plays it on the selected character; bindClip passes the model so node names that sanitize differently still bind; the unsupported-file toast names clips. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…poke; character e2e measure what they claim - SkeletonWindow: the helpers effect depended on `info` (derived from sceneRevision) while refreshHelpers pokes the scene, so it re-ran until svelte stopped it and the window never rendered; it is keyed on a string of the bones, their rest places and the live root id now - the Bones fields (rename, rest) follow the picked bone and the rig, not every scene poke, so a test-pose drag no longer wipes a name being typed - e2e: FULL_RESOURCE_TIMING init script (the 250-entry buffer fills during a dev boot, so every "lazy chunk fetched" check read false and every "not loaded at boot" check passed vacuously) - character-vrm: a two-bone spring chain is ONE three-vrm joint; the swing is sampled while the body moves (the damped spring settles before a 600 ms sample) - character-download: the (b) zip carries CREDITS.txt, crediting the applied clip's source - character-skeleton: prints the window's DOM + page errors when the map does not show Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… two halves - merge origin/feat/57-clips-m1 (PR #425): conflicts in animatedImports (rootDataFields carries character/provenance/vrmExpressions AND clipRefs; both register hooks), commandsHandler (the 'character' / 'expression' / 'clipRefs' parameters side by side, M3's stamp + crossfade kept), fileHandler (both lanes routed clipless rigs to the bytes path), wireValidate, package-lock - a humanoid clip asset binds through the M2 retarget: clipAssets loads humanoidRig before the first humanoid bind and calls retargetTpClip (rest pose + facing corrected, hips scaled, the character's own map incl. the user's fixes); bindHumanoid stays the fallback - humanoidRig registers its detector through setHumanoidDetector (a VRM's file map, else the detected map with the user's fixes) - tpSnapshot (retarget.js, a leaf): a decoded .tpanim's skeleton as a retarget snapshot - Apply clip… lists the Explorer's clip files; applying one assigns it as a clip asset on each target (one mechanism per kind of source; non-humanoid targets are refused with the reason) - Download: .tpanim / .vrma are written by 57 M1's encodeTpanim(toHumanoid(tpFromThreeClip(...))) so every clip file the app writes has one shape; the single-clip .glb keeps the rig's skeleton - unit tpanimRetarget: a clip through encode -> decode -> retarget comes back onto its own rig as the same pose (arms resting turned 35 degrees) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
5e5e9d4 — merged 57-clips M1 (#425) and joined the two halves. This PR now carries #425 too; merging it first is fine, as is merging this one alone.
Checks on 5e5e9d4:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Roadmap 57, lane 57-chars: M2, M4, M5 (Avatars source), M6, M7, M9. Packs half: theprototype-app/packs#18.
M2 humanoid map + retarget
src/lib/humanoid/humanoidMap.js, a leaf). Detection works from bone names (Mixamo, KayKit, Meshy, VRoid/UniRig, Unreal, Rigify/Quaternius UAL), then from the hierarchy (spine order), then from the rig's shape (numbered UniRig bones). A VRM brings its own map.retarget.js: a rest-pose and facing corrected retarget; the hips' travel is scaled to the character's size.userData.character); the file is untouched.M4 VRM characters
@pixiv/three-vrm3.5.5 (exact version) loads as a lazy chunk. It reads VRM 0.x and 1.0, including a.glbthat carries the VRM extension. Spring bones and expressions work, there's a Set Expression flow node, and the licence meta shows in the Inspector and becomes the model's provenance licence. VRM 0.x models are turned to face +Z.M6 rig fix-ups (lazy)
objectfilereplace:true) and undoes as one step.M7 provenance + licence warnings
provenance.jsrecords where every asset came from. Licence warnings show in Publish, Export, Community gallery and Download; they warn and never block. Exports and download zips include aCREDITS.txt.M9 Download…
.tpanimper clip, (c) model only..tpanim,.glb,.vrmaor.bvh(lazy BVH writer).M5 Asset Browser ▸ Avatars
Proof
Unit: vitest 207 files / 2,532 tests pass. New tests cover humanoidMap, retarget, provenance, characterDownload (including a BVH round trip), rigFixes and osaRegistry.
svelte-check ratchet held at 299/39.
e2e (from a snapshot, port 5422):
character-applycharacter-vrmcharacter-downloadcharacter-skeletoncharacter-avatars-sourceProduction build: three-vrm, springs, rig fix-ups, BVH writer, retarget and VRMA writer all stay off the boot graph (boot is 368 chunks / 8,076,062 bytes).
For the 57-clips union
setHumanoidDetectorsetClipAssetResolverto theirdecodeTpanimretargetClipinside theirbindClip.tpanimthrough theirencodeTpanimuserData.characterbecause they useuserData.humanoidas a plain bone map.🤖 Generated with Claude Code