Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions NOTICE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ OpenMagia would not exist without them.
- Role: the generation engine. Prompt-to-video/audio, first/last-frame
conditioning, and ordered reference (Ref2VA) inference on Metal.

### FastVideo — optional distilled video backend
- Upstream: <https://github.com/hao-ai-lab/FastVideo>
- Code license: Apache License 2.0 — Hao AI Lab and contributors.
- Role: optional native Apple Silicon MLX inference. Uses FastH3 Trim for
text-to-video and FastH3 Omni for image-reference video, with a separate
managed runtime and checkpoints under `addons/fastvideo/`.
- FastH3 weights retain the MiniMax H3 Community License; LightVAE is
Apache-2.0. Model weights and upstream source are downloaded separately,
not distributed in this repository. See `docs/FASTVIDEO_INTEGRATION.md`.

### llama.cpp — local prompt-formatting runtime
- Upstream: <https://github.com/ggml-org/llama.cpp>
- License: MIT License — Copyright (c) 2023–2026 The ggml authors
Expand Down
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,10 @@ The editor launches on macOS, Linux, and Windows, but OpenMagia only advertises

Apple Silicon systems can install [H3MLX](https://github.com/robzombai/h3mlx) from **Settings → Models → Add-ons** or with `./install.sh --with-h3mlx`. OpenMagia builds it in the ignored `addons/h3mlx` directory, reuses the existing MiniMax H3 weights, applies its tested video-decoder compatibility fix automatically, and validates the repaired pipeline before completing installation. No manual source editing is required. H3MLX-specific generation presets, progress parsing, queue recovery, backend selection, updates, and complete removal are handled by OpenMagia.

### FastVideo video engine (optional)

Install FastVideo from **Settings → Models → Available**, then select **Use for video**. It uses FastH3 Trim for text and FastH3 Omni for image references in a separate MLX runtime. Apple Silicon, at least 36 GB memory, and approximately 170 GB disk space are required. Opening-frame anchors and clips under five seconds show a direct action to switch to H3. [Setup, safety review and benchmark method](docs/FASTVIDEO_INTEGRATION.md). [Measured five-second comparisons and sample videos](benchmarks/2026-10-26-fastvideo/README.md).

### Music generation (optional)

Songs come from [YuE 2](https://github.com/multimodal-art-projection/YuE) (m-a-p/YuE2-3B). Install it with `./install.sh --with-yue` or from **Settings → Music**.
Expand Down Expand Up @@ -142,10 +146,24 @@ Read [CONTRIBUTING.md](CONTRIBUTING.md), follow the [Code of Conduct](CODE_OF_CO

## Built with

[MiniMax H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) · [YuE 2](https://github.com/multimodal-art-projection/YuE) by m-a-p · [h3.c](https://github.com/antirez/h3.c) by Salvatore Sanfilippo · [FFmpeg](https://ffmpeg.org/) · [llama.cpp](https://github.com/ggml-org/llama.cpp) · [Qwen2.5 7B Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF)
[MiniMax H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) · [YuE 2](https://github.com/multimodal-art-projection/YuE) by m-a-p · [h3.c](https://github.com/antirez/h3.c) by Salvatore Sanfilippo · [FastVideo](https://github.com/hao-ai-lab/FastVideo) by Hao AI Lab · [FFmpeg](https://ffmpeg.org/) · [llama.cpp](https://github.com/ggml-org/llama.cpp) · [Qwen2.5 7B Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF)

The Character Creator approach was inspired by [PoopMan333's H3 Character Sheet Generator](https://huggingface.co/PoopMan333/H3_Character_Sheet_Generator). OpenMagia is independent and is not affiliated with or endorsed by MiniMax or the projects above. Full attribution and component licenses are in [NOTICE.md](NOTICE.md).

## License

Copyright © 2026 David Vera. OpenMagia is released under the [GNU Affero General Public License v3.0 only](LICENSE). Modified network-accessible versions must make their corresponding source available under the same license. Third-party components retain their own licenses.

### MiniMax H3 Image (Apple Silicon)

Install **MiniMax H3 Images** under **Settings → Models → Available** after
reviewing the [model license](https://huggingface.co/MiniMaxAI/MiniMax-H3-Image/blob/main/LICENSE).
Allow roughly 52 GB of disk space. The module uses the [H3 Image MLX runtime](https://github.com/Anionex/h3-image-mlx)
and its documented INT8 model pack in `addons/h3-image`.

Choose **Use for images** after installation, or switch models in **Generate →
Image**. The selection is saved independently of the video engine. H3 Image
supports fixed six-step text generation and image editing with up to nine
ordered still-image references in OpenMagia. Refer to them as `<Picture 1>`,
`<Picture 2>`, and so on. Choose an aspect ratio, seed and output preset
from 512 px preview through 4096 px on the longest edge.
205 changes: 160 additions & 45 deletions app.js

Large diffs are not rendered by default.

20 changes: 20 additions & 0 deletions benchmarks/2026-10-26-fastvideo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# OpenMagia video benchmark — 26 October 2026

Apple M5 Max, 128 GB unified memory. 512×512, approximately five seconds, seed 20261026.

| Backend | Input | Wall seconds | Output seconds | Passes |
|---|---|---:|---:|---:|
| h3.c | text | 310.605 | 5.18 | 20 |
| H3MLX | text | 262.144 | 5.20 | 20 |
| FastVideo | text | 132.413 | 5.17 | 8 |
| h3.c | image | 326.590 | 5.18 | 20 |
| H3MLX | image | 283.883 | 5.20 | 20 |
| FastVideo | image | 175.620 | 5.17 | 8 |

Single-run measurements of different app presets and distilled models, not equal computational work. The reference image comes from the original h3.c text output, which may favor that model’s appearance. The first exploratory native timings were discarded and rerun without downloads. OS file and shader caches may be warm. Run order: FastVideo text/image, h3.c text/image, then H3MLX text/image. Timings include process startup, conditioning, denoising, decoding and media registration. FastVideo uses Trim INT6 for text and Omni INT6 for references, with M5 NAX attention and reference pixels capped to the output area. No temporal interpolation or spatial upscaling was enabled. FastVideo memory figures are peak MLX allocations, not total RAM; native peak memory was not measured.

Prompt:

> A small red tin toy robot stands on a pale wooden table against a plain teal backdrop. The robot slowly raises its right arm, waves twice, then lowers it. One continuous locked-off medium shot, soft studio light, clear natural motion, no text, no cuts. Quiet mechanical clicks, no speech or music.

Open [comparison.html](comparison.html) locally to play the six videos. [CSV](results.csv) · [JSON](results.json).
3 changes: 3 additions & 0 deletions benchmarks/2026-10-26-fastvideo/comparison.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<!doctype html><html lang="en"><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>OpenMagia video benchmark</title><style>
body{max-width:1320px;margin:40px auto;padding:0 24px;background:#101012;color:#ececf0;font:16px/1.6 system-ui}h1{font-size:36px;line-height:1.1}h2{margin-top:36px}p{color:#bcbcc5}.grid{display:grid;grid-template-columns:repeat(3,1fr);gap:20px}article{padding:16px;background:#1c1c21;border-radius:12px}h3{margin:0 0 14px}h3 span{float:right;color:#cbc4ee;font-weight:500}video{width:100%;border-radius:8px}article p{margin:8px 0 0;font-size:14px}.reference{width:180px;border-radius:10px}button{padding:10px 16px;border:1px solid #555;border-radius:8px;background:#292532;color:white;font:inherit;cursor:pointer}details{margin-top:28px}pre{white-space:pre-wrap;color:#bcbcc5}a{color:#cbc4ee}@media(max-width:800px){.grid{grid-template-columns:1fr}h1{font-size:28px}}</style>
<h1>OpenMagia video benchmark</h1><p>Apple M5 Max · 128 GB · 26 October 2026<br>512 × 512 · approximately five seconds · seed 20261026</p><button onclick="document.querySelectorAll('video').forEach(v=>{v.currentTime=0;v.muted=true;v.play()})">Play all muted</button><h2>Text to video</h2><div class="grid"><article><h3>h3.c <span>5m 11s</span></h3><video controls muted preload="metadata" src="h3-metal-text.mp4"></video><p>20 passes · 5.18s output</p></article><article><h3>H3MLX <span>4m 22s</span></h3><video controls muted preload="metadata" src="h3mlx-text.mp4"></video><p>20 passes · 5.20s output</p></article><article><h3>FastVideo <span>2m 12s</span></h3><video controls muted preload="metadata" src="fastvideo-text.mp4"></video><p>8 passes · 5.17s output</p></article></div><h2>One image + text</h2><div class="grid"><article><h3>h3.c <span>5m 27s</span></h3><video controls muted preload="metadata" src="h3-metal-image.mp4"></video><p>20 passes · 5.18s output</p></article><article><h3>H3MLX <span>4m 44s</span></h3><video controls muted preload="metadata" src="h3mlx-image.mp4"></video><p>20 passes · 5.20s output</p></article><article><h3>FastVideo <span>2m 56s</span></h3><video controls muted preload="metadata" src="fastvideo-image.mp4"></video><p>8 passes · 5.17s output</p></article></div><h2>Common reference</h2><img class="reference" src="reference.png" alt="Red toy robot benchmark reference"><details><summary>Prompt and method</summary><pre>A small red tin toy robot stands on a pale wooden table against a plain teal backdrop. The robot slowly raises its right arm, waves twice, then lowers it. One continuous locked-off medium shot, soft studio light, clear natural motion, no text, no cuts. Quiet mechanical clicks, no speech or music.</pre><p>Single-run measurements of different app presets and distilled models, not equal computational work. The reference image comes from the original h3.c text output, which may favor that model’s appearance. The first exploratory native timings were discarded and rerun without downloads. OS file and shader caches may be warm. Run order: FastVideo text/image, h3.c text/image, then H3MLX text/image. Timings include process startup, conditioning, denoising, decoding and media registration. FastVideo uses Trim INT6 for text and Omni INT6 for references, with M5 NAX attention and reference pixels capped to the output area. No temporal interpolation or spatial upscaling was enabled. FastVideo memory figures are peak MLX allocations, not total RAM; native peak memory was not measured.</p></details><p><a href="results.csv">CSV</a> · <a href="results.json">JSON and file hashes</a></p></html>
1 change: 1 addition & 0 deletions benchmarks/2026-10-26-fastvideo/dependency-audit.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"dependencies": [{"name": "annotated-doc", "version": "0.0.5", "vulns": []}, {"name": "anyio", "version": "4.15.1", "vulns": []}, {"name": "av", "version": "19.0.2", "vulns": []}, {"name": "certifi", "version": "2026.10.23", "vulns": []}, {"name": "click", "version": "8.5.0", "vulns": []}, {"name": "filelock", "version": "4.2.4", "vulns": []}, {"name": "fsspec", "version": "2026.7.0", "vulns": []}, {"name": "h11", "version": "0.16.0", "vulns": []}, {"name": "hf-xet", "version": "1.7.0", "vulns": []}, {"name": "httpcore", "version": "1.0.9", "vulns": []}, {"name": "httpx", "version": "0.28.1", "vulns": []}, {"name": "huggingface-hub", "version": "1.33.0", "vulns": []}, {"name": "idna", "version": "3.20", "vulns": []}, {"name": "markdown-it-py", "version": "4.2.0", "vulns": []}, {"name": "mdurl", "version": "0.1.2", "vulns": []}, {"name": "mlx", "version": "0.32.4", "vulns": []}, {"name": "mlx-metal", "version": "0.32.4", "vulns": []}, {"name": "numpy", "version": "2.5.4", "vulns": []}, {"name": "opencv-python", "version": "5.0.0.93", "vulns": []}, {"name": "packaging", "version": "26.3", "vulns": []}, {"name": "pillow", "version": "13.0.0", "vulns": []}, {"name": "pygments", "version": "2.21.0", "vulns": []}, {"name": "pyyaml", "version": "6.0.3", "vulns": []}, {"name": "regex", "version": "2026.9.29", "vulns": []}, {"name": "rich", "version": "15.0.0", "vulns": []}, {"name": "safetensors", "version": "0.8.0", "vulns": []}, {"name": "scipy", "version": "1.18.1", "vulns": []}, {"name": "shellingham", "version": "1.5.4", "vulns": []}, {"name": "tokenizers", "version": "0.22.2", "vulns": []}, {"name": "tqdm", "version": "4.70.1", "vulns": []}, {"name": "transformers", "version": "5.15.1", "vulns": []}, {"name": "typer", "version": "0.27.3", "vulns": []}, {"name": "typing-extensions", "version": "4.16.0", "vulns": []}], "fixes": []}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added benchmarks/2026-10-26-fastvideo/h3mlx-image.mp4
Binary file not shown.
Binary file added benchmarks/2026-10-26-fastvideo/h3mlx-text.mp4
Binary file not shown.
Binary file added benchmarks/2026-10-26-fastvideo/reference.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 7 additions & 0 deletions benchmarks/2026-10-26-fastvideo/results.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
backend,mode,wall_seconds,duration_seconds,width,height,audio,requested_frames,passes,layers,reuse,seed,output,model,attention,peak_mlx_gib,sha256
h3.c,text,310.605,5.18,512,512,True,120,20,45,2,20261026,h3-metal-text.mp4,MiniMax-H3 FL2VA,native Metal,,b2f961b031fd9b2ae37172e066b39e31150ef1481020b4ae8918b20713667070
h3.c,image,326.59,5.18,512,512,True,120,20,45,2,20261026,h3-metal-image.mp4,MiniMax-H3 Ref2VA,native Metal,,9bca7768b99565d13daf04115fe640ca0c612832121c63c9a0f56b569b0ef412
H3MLX,text,262.144,5.2,512,512,True,120,20,35,2,20261026,h3mlx-text.mp4,MiniMax-H3 FL2VA,native Metal,,5b3460e0da369114c25efd35341c67683dd1dbb6c1434c0f2b2d2ddc92cf60ed
H3MLX,image,283.883,5.2,512,512,True,120,20,35,2,20261026,h3mlx-image.mp4,MiniMax-H3 Ref2VA,native Metal,,6e3ffba8216d87c83d3dbc331cab4bdea0c3d22088bda815891ab24648e7fc51
FastVideo,text,132.413,5.17,512,512,True,120,8,,,20261026,fastvideo-text.mp4,FastVideo/FastVideo-FastH3-Trim-8-Step-MLX-INT6,nax,17.622,da21fca6e952ecc5cd110e0572aaa3bcb94d8f116e21423429f9b2c4a2415497
FastVideo,image,175.62,5.17,512,512,True,120,8,,,20261026,fastvideo-image.mp4,FastVideo/FastVideo-FastH3-Omni-8-Step-V1,nax,20.424,eabd1b4c13e7c672af1bc5fdc2b811ae9609f73bbfe33a9f1fa56b5cfa99e070
151 changes: 151 additions & 0 deletions benchmarks/2026-10-26-fastvideo/results.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
{
"source_revisions": {
"h3.c": "8974cc055ea9c02fcd14cc27dfda3e1027c05153",
"H3MLX": "db3cae141c1cced70355f1d49f7268cf60d60db4",
"FastVideo": "a0e6cf1f5086c9af735d73e6c0b80dbc1d5817b3"
},
"fastvideo_attention_provenance": {
"kernel": "h3-true-sparse-nax-v1",
"relaxed_precision": true,
"math_mode": "safe",
"key_rows": 32,
"source_sha256": "eaa1ea090045600d622ca2a975c1483996ddaa7d22403b19615329cecde28ffd",
"expanded_header_sha256": "e7cd1bbe30d9dddd7998fea7a4d74c03f142335b47ab960c072242f269171663",
"installed_header_sha256": {
"mlx/backend/metal/kernels/steel/attn/nax.h": "fb10bcae44095e9fed5c81213d03527dd5fe5ac11be11ec7784bd357c6f317ec",
"mlx/backend/metal/kernels/steel/defines.h": "b03cea6a7d5cfe814e6838d8536e2aacdcafd2744cb408e954fda336ce21759a",
"mlx/backend/metal/kernels/steel/utils/integral_constant.h": "886eccaac4f8f00eab5a10551fdb5315df7ab7462aba13d69bb95765c1a4735e",
"mlx/backend/metal/kernels/steel/utils/type_traits.h": "4766c5a3809d3cf7740e63acc5703ef76b6b4e85d8baf16602c9d517a2fd426c"
}
},
"hardware": "Apple M5 Max, 128 GB unified memory",
"date": "2026-10-26",
"os": "macOS 27.0.1",
"run_order": [
"FastVideo text",
"FastVideo image",
"h3.c text",
"h3.c image",
"H3MLX text",
"H3MLX image"
],
"method": "One serial process-per-job generation per case. All final timings exclude concurrent model downloads.",
"prompt": "A small red tin toy robot stands on a pale wooden table against a plain teal backdrop. The robot slowly raises its right arm, waves twice, then lowers it. One continuous locked-off medium shot, soft studio light, clear natural motion, no text, no cuts. Quiet mechanical clicks, no speech or music.",
"reference_sha256": "ec61159c46d1b9987410cc47b24cd48211e43a6859bc88b3ba7d3746edca2f0d",
"results": [
{
"backend": "h3.c",
"mode": "text",
"wall_seconds": 310.605,
"duration_seconds": 5.18,
"width": 512,
"height": 512,
"audio": true,
"requested_frames": 120,
"passes": 20,
"layers": 45,
"reuse": 2,
"seed": 20261026,
"output": "h3-metal-text.mp4",
"model": "MiniMax-H3 FL2VA",
"attention": "native Metal",
"peak_mlx_gib": null,
"sha256": "b2f961b031fd9b2ae37172e066b39e31150ef1481020b4ae8918b20713667070"
},
{
"backend": "h3.c",
"mode": "image",
"wall_seconds": 326.59,
"duration_seconds": 5.18,
"width": 512,
"height": 512,
"audio": true,
"requested_frames": 120,
"passes": 20,
"layers": 45,
"reuse": 2,
"seed": 20261026,
"output": "h3-metal-image.mp4",
"model": "MiniMax-H3 Ref2VA",
"attention": "native Metal",
"peak_mlx_gib": null,
"sha256": "9bca7768b99565d13daf04115fe640ca0c612832121c63c9a0f56b569b0ef412"
},
{
"backend": "H3MLX",
"mode": "text",
"wall_seconds": 262.144,
"duration_seconds": 5.2,
"width": 512,
"height": 512,
"audio": true,
"requested_frames": 120,
"passes": 20,
"layers": 35,
"reuse": 2,
"seed": 20261026,
"output": "h3mlx-text.mp4",
"model": "MiniMax-H3 FL2VA",
"attention": "native Metal",
"peak_mlx_gib": null,
"sha256": "5b3460e0da369114c25efd35341c67683dd1dbb6c1434c0f2b2d2ddc92cf60ed"
},
{
"backend": "H3MLX",
"mode": "image",
"wall_seconds": 283.883,
"duration_seconds": 5.2,
"width": 512,
"height": 512,
"audio": true,
"requested_frames": 120,
"passes": 20,
"layers": 35,
"reuse": 2,
"seed": 20261026,
"output": "h3mlx-image.mp4",
"model": "MiniMax-H3 Ref2VA",
"attention": "native Metal",
"peak_mlx_gib": null,
"sha256": "6e3ffba8216d87c83d3dbc331cab4bdea0c3d22088bda815891ab24648e7fc51"
},
{
"backend": "FastVideo",
"mode": "text",
"wall_seconds": 132.413,
"duration_seconds": 5.17,
"width": 512,
"height": 512,
"audio": true,
"requested_frames": 120,
"passes": 8,
"layers": null,
"reuse": null,
"seed": 20261026,
"output": "fastvideo-text.mp4",
"model": "FastVideo/FastVideo-FastH3-Trim-8-Step-MLX-INT6",
"attention": "nax",
"peak_mlx_gib": 17.622,
"sha256": "da21fca6e952ecc5cd110e0572aaa3bcb94d8f116e21423429f9b2c4a2415497"
},
{
"backend": "FastVideo",
"mode": "image",
"wall_seconds": 175.62,
"duration_seconds": 5.17,
"width": 512,
"height": 512,
"audio": true,
"requested_frames": 120,
"passes": 8,
"layers": null,
"reuse": null,
"seed": 20261026,
"output": "fastvideo-image.mp4",
"model": "FastVideo/FastVideo-FastH3-Omni-8-Step-V1",
"attention": "nax",
"peak_mlx_gib": 20.424,
"sha256": "eabd1b4c13e7c672af1bc5fdc2b811ae9609f73bbfe33a9f1fa56b5cfa99e070"
}
]
}
Loading
Loading