Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills
1 change: 1 addition & 0 deletions .claude/skills
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ generated/
!.opencode/agents/**
!.opencode/skills/
!.opencode/skills/**
# Skill entry points for Claude Code and Codex: symlinks to .opencode/skills
!.claude/
.claude/*
!.claude/skills
!.agents/
.agents/*
!.agents/skills
.opencode/node_modules/
.opencode/package.json
.opencode/package-lock.json
Expand Down Expand Up @@ -103,3 +110,6 @@ samples/Weather/**/*.lua
!samples/Weather/config/*.lua
Error
/samples/Book/Images

# Blender auto-backups next to committed .blend files
*.blend1
174 changes: 174 additions & 0 deletions .opencode/skills/orca-3d-scene/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
---
name: orca-3d-scene
description: Set up, drive and debug an ORCA project that renders a 3D scene (SceneKit): package.lua, Screen > Viewport3D > Scene > Camera, Mesh/Material/Shader XML, custom uniforms as property types, blend and draw order, AnimationClip players scrubbed from input values, prefabs, running the app and verifying frames. Use for any 3D app, HMI display or cluster project in ORCA, whether exported from Blender or written by hand.
---

# ORCA 3D Scene

`apps/cluster-demo/` is the worked example of everything here. Generated
assets come from `$orca-blender-pipeline`; this skill covers what the engine
expects and how to run it. Facts below were verified against the source in
`plugins/SceneKit/`, `source/renderer/` and `source/core/components/`.

## Project layout

```
apps/<project>/
package.lua manifest (package.xml is NOT read any more)
Screens/<S>.xml Screen > Viewport3D > Scene > Camera + content
Scenes/<S>.xml Node3D prefab with the content (generated)
Meshes/<M>.xml <Mesh Source="<Project>/Meshes/<M>.mesh"/> (+ .mesh, gitignored)
Shaders/<G>.xml <Shader> with VertexShader / FragmentShader text
Materials/<Mat>.xml <Material Shader=... BlendMode=... Uniform.X=.../>
Animations/<A>.xml <AnimationClip> + Animations.lua index
```

`package.lua` essentials (see `apps/cluster-demo/package.lua`):

```lua
Name = "ClusterDemo" -- asset path prefix
StartupScreen = "ClusterDemo/Screens/Cluster"
WindowWidth, WindowHeight = 1920, 720
ScreenLibrary = { IsExternal = true }
SystemMessages = { { Message = "KeyDown", Key = "q", Command = "return" }, ... } -- field is Message, not Name
EnginePlugins = { { Name = "orca.UIKit" }, { Name = "orca.SceneKit" } } -- SceneKit must be listed
PropertyTypes = { { Name = "Level", Category = "Uniform", DataType = "Float" }, ... }
```

Asset paths are `<Name>/<relative path>` without extension; `.xml` is tried
by the loader. Run with `build/bin/orca apps/<project>`.

## Scene XML

```xml
<Screen Name="Cluster" Width="1920" Height="720" ClearColor="#000000">
<Viewport3D Name="Viewport" Width="1920" Height="720">
<Scene Name="Scene" Camera="DisplayCamera">
<Camera Name="DisplayCamera" Fov="12.19" FovType="Yfov" ZNear="1" ZFar="20"
LayoutTransform="0 0 7.2 0 0 0 1 1 1"/>
<PrefabView3D Name="ClusterView" Prefab="ClusterDemo/Scenes/Cluster"/>
</Scene>
</Viewport3D>
</Screen>
```

- Coordinate system: right-handed, Y up, camera looks down its local -Z.
- `LayoutTransform="tx ty tz rx ry rz sx sy sz"`; rotation in degrees,
applied X then Y then Z. Atoms `LayoutTransformTranslation/Rotation/Scale`
are what animations target. `RenderTransform*` is an extra transform on top.
- The camera must be a **direct child of the Scene** (lookup is by name among
direct children). `Fov` is the full angle; use `FovType="Yfov"` with the
vertical FOV. `ProjectionType` is ignored (always perspective).
- `Viewport3D` renders its first direct `Scene` child and always draws to the
full window; make it full-size.
- `Model3D Mesh="<P>/Meshes/<M>" Material="<P>/Materials/<Mat>"`. `<Mesh>`'s
own `Material` is ignored. `#Quad` and `#Plane` are built-in mesh sources
for smoke tests.
- `PrefabView3D Prefab="..."` loads a file as its child (async, after Start).

## Shaders and materials

```xml
<Shader Name="IC_Flat" Uniform.Tint="#FFFFFF" Uniform.Intensity="1">
<VertexShader>
in vec3 a_position; in vec2 a_texcoord0; out vec2 v_texcoord0;
uniform mat4 u_modelViewProjectionTransform;
void main() { v_texcoord0 = a_texcoord0; gl_Position = u_modelViewProjectionTransform * vec4(a_position, 1.0); }
</VertexShader>
<FragmentShader Out="FragColor">
in vec2 v_texcoord0;
void main() { FragColor = vec4(Tint.rgb, Intensity); }
</FragmentShader>
</Shader>
<Material Name="IC_White" Shader="ClusterDemo/Shaders/IC_Flat" BlendMode="Opaque" Uniform.Tint="#F2F4F7" Uniform.Intensity="1"/>
```

- The engine prepends `#version 330 core` (`300 es` on QNX/WebGL), declares
`out lowp vec4 <Out>;`, and emits `uniform <type> <name>;` for **every
property set on the Shader element**. Declare custom uniforms there
(`Uniform.Name="default"`) so colours get the sRGB->linear conversion; do not
redeclare them in GLSL. Text may be plain or CDATA; escape `<` as `&lt;`
in plain text.
- Attributes: `a_position`, `a_normal`, `a_tangent`, `a_binormal`, `a_color`,
`a_texcoord0..4`. Builtins: `u_modelViewProjectionTransform`,
`u_modelTransform`, `u_viewTransform`, `u_projectionTransform`,
`u_normalTransform`, `u_opacity`, `u_time`, `u_viewport`,
`u_cameraPosition`, `u_texture`, `u_color`.
- Custom uniforms are attached properties `Category.Name`; the uniform name
is the part after the dot. Each name needs a `PropertyTypes` entry in
`package.lua` (`DataType` `Float` or `Color`; vectors also work now but
keep to these two). Values set on the `Model3D` override the material's,
which is how per-node animation works.
- Built-in material colours (`Diffuse`, `Emissive`, ...) are also available
as uniforms by name. `BlendIntensity` is forced to the node opacity.
- `BlendMode` on the Material: `Opaque`, `Alpha`, `Additive`
(`SRC_ALPHA, ONE`), `PremultipliedAlpha`; `AlphaAutomatic` inherits the
Shader's `BlendMode`. A `Material.BlendMode` attribute on the `Model3D`
overrides both. Shader `DepthTestFunction` / `DepthWriteEnabled` apply too.
- Draw order is tree order with depth test `LessOrEqual` and depth writes on
by default: order transparent siblings back to front. Culling is off.
- Every `Model3D` loads its own Material and compiles its own Shader (no
cache); keep the material count reasonable.

## Animation and binding

```xml
<AnimationClip Name="Gauge_Rev" Mode="PlayOnce" StartTime="0" StopTime="1">
<AnimationCurve Path="../../IC_Root/IC_Panel_Rev/Rev_Needle" Property="LayoutTransformRotation">
<AnimationCurve.Keyframes>
<Keyframe Time="0" Value="0 225 0 0" TangentMode="1"/>
<Keyframe Time="1" Value="0 -45 0 0" TangentMode="1"/>
</AnimationCurve.Keyframes>
</AnimationCurve>
</AnimationClip>
...
<Node3D Name="Animations">
<AnimationPlayer Name="Gauge_Rev" Clip="ClusterDemo/Animations/Gauge_Rev"/>
</Node3D>
```

- An `<AnimationPlayer>` element is **an object of its own**; curve `Path`s
are relative to it (`..` allowed). `Keyframe` fields are `Time`, `Value`
(vec4), `TangentMode` (`1` = linear; default bezier uses `InSlope`,
`OutSlope`, `InWeight`, `OutWeight`).
- `Property` is the short name of any property on the target: transform
atoms, `Opacity`, or a uniform such as `Level` if the node declares it
(`Uniform.Level="0.5"` on the Model3D).
- **Scrubbing:** setting `CurrentTime` on a player that is not playing
applies the clip at that time. Bind an input to a gauge with
`player.CurrentTime = Duration * (value - Min) / (Max - Min)` using the
entries in `Animations.lua`. Another clip can do it too: curves with
`Property="CurrentTime"` targeting the players
(`apps/cluster-demo/Demo/FillAll.xml`).
- Playback: `AutoplayEnabled="true"`, `Mode="Loop|PingPong"` on the clip,
`Speed`, `Play/Stop/Pause/Resume` messages. Time advances once per painted
frame.

## Run and verify

```bash
build/bin/orca apps/<project> 2>&1 | tee /tmp/orca.log
```

Errors to grep for: `shader compilation failed`, `program link failed`,
`model not found`, `is not a MESH file`, `Could not set field`, `Unknown
property`, `Unknown element type`, `Failed to load prefab`.

There is no screenshot command. On macOS capture only the orca window:

```bash
WID=$(swift -e 'import CoreGraphics; for w in CGWindowListCopyWindowInfo(.optionOnScreenOnly, kCGNullWindowID) as! [[String: Any]] where (w["kCGWindowOwnerName"] as? String) == "orca" { print(w["kCGWindowNumber"]!) }' | head -1)
screencapture -x -o -l "$WID" frame.png
```

Take two captures a second apart and `cmp` them to prove animation runs.
`-test=<file.lua>` runs headless Lua against the engine without a window
(see `tests/test_animations.lua` for clips driven from Lua).

## Engine limits to design around

- `TextBlock3D` renders white unless `Color` is set, lays out at 512 px, and
cannot use custom shaders; baked text meshes are the alternative.
- No orthographic cameras; `Viewport3D` ignores its own rect.
- Vertex colours load (`a_color`, UINT8 normalised) when the FBX had them.
- Rebuild after editing `.cgen`: `make modules && make`.
4 changes: 4 additions & 0 deletions .opencode/skills/orca-3d-scene/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "ORCA 3D Scene"
short_description: "Set up, drive and debug ORCA 3D scene projects"
default_prompt: "Use $orca-3d-scene to create or debug an ORCA project with a SceneKit scene: package.lua, Viewport3D/Scene/Camera XML, shaders and materials, animation players bound to input values, and verifying the running app."
153 changes: 153 additions & 0 deletions .opencode/skills/orca-blender-pipeline/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
---
name: orca-blender-pipeline
description: Build 3D display content (instrument clusters, HMI scenes, dashboards) in Blender so that tools/blender-export.py turns it into a runnable ORCA project: shader node groups that become GLSL, UV-driven materials, value-driven "fill-up" animations, and the export, regeneration and verification loop. Use when authoring or changing a Blender scene for ORCA, exporting one, or debugging why an export looks or behaves differently in the engine.
---

# ORCA Blender Pipeline

The pipeline is: Blender scene -> `tools/blender-export.py` (inside Blender)
-> FBX + `orca-fbx` for meshes, XML for everything else -> `build/bin/orca`.
`apps/cluster-demo/` is the reference: `blender/build_cluster.py` builds the
whole scene procedurally and documents every convention below in code.

Read `$orca-3d-scene` for the engine side (project layout, XML, binding,
running). This skill is about what the Blender file must look like.

## Setup

```bash
brew install lz4 freetype jpeg libpng # engine deps on macOS
export PKG_CONFIG_PATH=/opt/homebrew/opt/jpeg/lib/pkgconfig:$PKG_CONFIG_PATH
make # engine
make orca-fbx # needs /Applications/Autodesk/FBX SDK/2020.2
```

Blender 4.x/5.x. The exporter runs inside Blender (it uses `bpy`); the
Blender MCP server or `blender -b file.blend --python ...` both work.

## Scene conventions

- **Axes and units.** Author Z up with the camera on -Y looking +Y. The
exporter rotates everything into ORCA's Y-up, camera-looks-down--Z space, so
never pre-rotate for the engine. Units are arbitrary but keep them sane
(the cluster uses 1 unit = 10 cm); the camera frustum defines the display.
- **Camera = display.** One camera, perspective, `sensor_fit='HORIZONTAL'`,
and `scene.render.resolution_x/y` set to the target display. The exporter
derives the vertical FOV from these; the window size comes from the same
resolution.
- **Hierarchy.** Empties are fine as groups; every object's local transform is
exported as-is (`LayoutTransform`). Put a pivot where something rotates or
scales at runtime (needle origin on the dial centre, bar origin on its left
end).
- **Layering.** The engine draws siblings in tree order with depth writes on.
The exporter sorts siblings back-to-front by camera distance, so give
stacked transparent elements distinct depths (small Z offsets per layer) and
keep glows behind the things they glow around.
- **Panels that face the eye.** Place side panels on a cylinder around the
camera and yaw them towards it; nothing in the engine requires this, it is
what makes a wide display read correctly.
- **Text.** Bake text to mesh in the build script (`meshes.new_from_object`)
so the export is complete; keep the string, font and size in custom
properties (`orca_text*`) for a later `TextBlock3D` replacement. Fonts: the
cluster uses system DIN; any TTF works.
- **Collections.** Put everything exportable in one collection and export it
with `--collection`; keep runtime-driven objects in a sub-collection so they
are easy to find.

## Materials are shader node groups

A material is **one group node named `Shader` plus a blend mode**. The group
is the fragment shader: it reads UV0 and its inputs (the uniforms) and
outputs `Color` and `Alpha`. The material wraps that in Emission plus
Transparent (alpha) or Add Shader (additive) so EEVEE previews it faithfully.

Rules the exporter depends on:

- The group needs a Text datablock named `<group>.frag` holding the GLSL
fragment body (`in vec2 v_texcoord0;` plus `void main()` writing
`FragColor`). ORCA adds the `#version`, the `out` declaration and the
`uniform` lines itself, so do not write `#version`, do not declare
`FragColor`, and any `uniform` lines you keep for readability are stripped on
export. An optional `<group>.vert` overrides the default vertex stage.
- Keep the node graph to Math, Mix, Separate XYZ, Vector Math and Texture
Coordinate so group and GLSL stay equivalent; write both at the same time.
- Inputs: floats and colours only (ORCA passes other uniform types badly).
Name inputs so they do not shadow node properties (`Intensity`, `Tint`, not
`Opacity`, `Color`). Colour inputs are vec4 in GLSL; use `.rgb`.
- Colours are authored as display sRGB and converted to linear by the engine
(`pow 2.2`) before your shader runs, and the framebuffer is sRGB. Blender
with the `Standard` view transform shows the same result, so author colours
in the material's node inputs and preview with `Standard`.
- UV0 carries the shader coordinate: U along an element, V across it; for
rings U is the angle and V the radius. Build geometry with UVs that mean
something (sweep fraction, segment index, distance along a stroke).
- `material["orca_blend"]` is `opaque`, `alpha` or `additive`.
- No lights, no textures needed; if an Image Texture node is present its
image is copied to `Images/` and set as the material `Texture` (untested
path).

The six groups in the cluster (`IC_Flat`, `IC_Gradient`, `IC_Sweep`,
`IC_Segments`, `IC_SpunMetal`, `IC_RadialGlow`) cover flat colour, ramps and
glows, gauge fills, segmented bars, fake brushed metal and soft blobs; start
from them.

## Animations are value-driven clips

Each gauge gets one action that fills it from empty (frame 0) to full (last
frame) with **linear** keys. The runtime never plays it: it sets the player's
`CurrentTime`, so `time / duration` is the reading. Author with
`scene.render.fps = 100` and frames 0..100 so the clip is exactly 1 s long and
time == normalised value.

- One action may hold several slots: the needle's rotation (object) and the
sweep's `Level` (the material node tree's `nodes["Shader"].inputs[...]`)
live in the same `Gauge_Rev` action.
- Put the signal on the action: `action["orca_signal"]`, `orca_min`,
`orca_max`. The exporter writes them to `Animations.lua`.
- Supported channels: object `location`, `rotation_euler`, `scale` and shader
input `default_value`. Non-linear keys are baked per frame.
- Animated shader inputs are exported per node (each Model3D gets its own
`Uniform.<Name>`), because every node loads its own material instance.

## Export

```python
import runpy
runpy.run_path("tools/blender-export.py")["export"](
"apps/<project>", collection="<Collection>")
```

or headless (untested): `blender -b scene.blend --python tools/blender-export.py -- apps/<project> --collection <Collection>`.

Outputs: `Meshes/` (binary `.mesh` are gitignored; the `<Mesh>` XML next to
them is committed), `Shaders/`, `Materials/`, `Animations/` + `Animations.lua`,
`Scenes/<Screen>.xml` (prefab, regenerated every time), and once only
`Screens/<Screen>.xml` and `package.lua` (app-owned afterwards; the export
result prints the camera element to copy if it changed, and lists property
types missing from `package.lua`).

Save the `.blend` and the build script under `apps/<project>/blender/`.

## Verify

1. `build/bin/orca apps/<project>` and read stderr: `shader compilation
failed`, `model not found`, `Could not set field`, `Unknown element` are the
usual export mistakes.
2. Compare against Blender's render through the display camera. Expect the
same layout and colours; a washed-out image means colours were exported
linear instead of sRGB or a uniform was not declared on the Shader.
3. For motion, capture the window twice and compare (see `$orca-3d-scene`).

## Common mistakes

- Blown-out or missing glow: wrong `orca_blend`, or two layers at the same
depth drawn in the wrong order.
- Mirrored or sideways scene: FBX exported with other axis settings than the
exporter uses (`-Z` forward, `Y` up, bake space transform).
- A uniform renders black/zero: its name is missing from `PropertyTypes` in
`package.lua` (the export result lists these), or it collides with an
engine property name.
- Gauge pose does not change: the clip's curve `Path` does not resolve; paths
are relative to the player object (`../../<node path>` from
`Animations/<clip>`), and `Keyframe` fields are `Time`, `Value`,
`TangentMode` (capitalised).
4 changes: 4 additions & 0 deletions .opencode/skills/orca-blender-pipeline/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "ORCA Blender Pipeline"
short_description: "Author Blender scenes that export to ORCA"
default_prompt: "Use $orca-blender-pipeline to build or change a Blender scene (shader node groups, UV-driven materials, fill-up animations) and export it with tools/blender-export.py into an ORCA project."
9 changes: 7 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
## Agents and skills

Agents are classic implementation roles under `.opencode/agents/`; skills are
task playbooks under `.opencode/skills/`. Choose the task skill first, then use
an agent when delegation or specialist ownership is useful.
task playbooks under `.opencode/skills/`. The same skills are reachable as
`.claude/skills/` (Claude Code) and `.agents/skills/` (Codex); both are
symlinks, so edit only `.opencode/skills/`. Choose the task skill first, then
use an agent when delegation or specialist ownership is useful.

| Typical task | Skill | Agent |
|---|---|---|
Expand All @@ -14,6 +16,8 @@ an agent when delegation or specialist ownership is useful.
| Write Lua or MoonScript programs | `orca-lua-authoring` | `lua-developer` |
| Write UI/package XML | `orca-xml-authoring` | `xml-ui-developer` |
| Define images, textures, shaders, font families, or scenes | `orca-xml-assets-scenes` | `renderer-developer` |
| Build or debug a 3D app: SceneKit scene XML, shaders/materials, animation players bound to inputs, running it | `orca-3d-scene` | `renderer-developer` |
| Author a Blender scene for ORCA and export it (`tools/blender-export.py`, `orca-fbx`) | `orca-blender-pipeline` | `renderer-developer` |
| Bind Lua to XML with controllers, events, or data bindings | `orca-controller-binding` | `lua-developer` or `xml-ui-developer` |
| Add or update tests | `orca-testing` | `test-writer` |
| Apply C or CSS conventions | `orca-c-style` or `orca-css` | relevant implementation agent |
Expand Down Expand Up @@ -68,6 +72,7 @@ After `.cgen` changes, run `make modules`.
| Lua API | generated Lua reference, then `docs/LUA_API.md` |
| UI / layout / XML / style | `docs/UI_SYSTEM.md` |
| Text rendering / fonts / glyphs | `docs/renderer-internals.md` → *Text Rendering Pipeline* |
| 3D scene / Blender export | `apps/cluster-demo/` (reference project), `docs/animations.md` |
| C style | `docs/CODING_STYLE.md` |

## Working rules
Expand Down
Loading
Loading