Skip to content
View bertholomus's full-sized avatar

Block or report bertholomus

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
bertholomus/README.md

BertholomusAI

Big open models on hardware you can own. We serve DeepSeek, GLM, Mistral, Beam, Ling, Qwen and others on NVIDIA DGX Spark (GB10) clusters and on single RTX PRO 6000 cards. We write our own engine families, make our own quants and publish every recipe with its measurements.

Each recipe pins its exact revisions and hashes, and ships its gate results, its limits and its credits.

Latest (November 2026)

  • Qwen3.8-Flash-Next NVFP4 on one RTX PRO 6000 (v1.1): GPTQ-calibrated NVFP4 experts on the stock TensorFold 1.2.9 release, no fork and no patches. The full 262K context; 159 tok/s on one stream, 445 tok/s total with 8 agents.
  • spark-attic (v0.1.7): about 2 GiB more memory for CUDA on a headless GB10. The driver reserves memory for a display that a headless node never drives. atticd lends it to the inference engine, so the KV cache grows with no other change. Replies stay identical and speed stays within noise in vLLM, SGLang and TensorFold. Upstream: TensorFold #2762, #2763.
  • GLM-5.3 (full) EXLR8 on four DGX Sparks (v2): the gr8 engine and exlr8 weights by KindlingAI (Matt Mastracci, mmastrac, and Chuck, chuck-ads / adapt-ai-systems). Our recipe adds a concurrency dial: 4 chats at once at 95 tok/s total (about 24 each), or one chat with up to 864K tokens (with spark-attic on). Our busy guard is merged upstream in gr8 #516: a request that would evict a live session gets a 429 at once, so agents do not go quiet.
  • DeepSeek-V4.1-Flash-1017 on 4× GB10 (v0.3.7): 32 users at once, 290 tok/s total, first token in 0.42 s (was 15.9 s). One stream: 138 tok/s on code.
  • DeepSeek-V4.1-Flash-1017 on 2× GB10 (v0.7.5): 32 users at once, 249 tok/s total (was 198). One stream: 114 tok/s on code. 1M-token prompts.

Recipes

Model Hardware Highlights Recipe Weights Support
GLM-5.3 (full) EXLR8 4× GB10 KindlingAI's gr8 + exlr8; concurrency dial, busy guard, optional spark-attic, SSD KV store GitHub KindlingAI Active
Qwen3.8-Flash-Next 1× RTX PRO 6000 NVFP4 (GPTQ experts) on the stock TensorFold release; 262K context; 445 tok/s total with 8 agents GitHub NVFP4 Maintained
Mistral Large 4 (1T) 4× GB10 Our mistral_large4 TensorFold family; the full 1M window on one stream; Mistral's draft head: ~30 tok/s on code GitHub 2.5 bpw Maintained
Beam-1.0 (501B) 2× GB10 Our TensorFold beam family; 868K-token window; 6–36 % faster than our vLLM lane on six of seven decode rows GitHub 3 bpw Maintained
Ling-3.1-flash-1019 (560B) 2× GB10 Our TensorFold bailing_v3 family; native 262K window; MTP drafting GitHub 3.03 bpw Maintained
Ripple-65B-Preview 1× RTX PRO 6000 NVFP4 + calibrated FP8 attention; 218 tok/s one stream on code GitHub NVFP4/FP8 Maintained
DeepSeek-V4.1-Flash-1017 4× GB10 Our TensorFold (Zig) family, exact 4-way split; 1M tokens a request; vision, tool calls, structured output, DSpark drafting GitHub 4.0 bpw Maintenance
DeepSeek-V4.1-Flash-1017 2× GB10 The same family on two nodes; 1M window, vision, DSpark drafting GitHub 2.5 bpw Maintenance
GLM-5.3 (full) 4× GB10 Our TensorFold branch; 1M context through context parallelism; KL 0.109 vs BF16 GitHub 3.0 bpw Frozen

Engines and upstream work

Tools

Support

Models and engines move fast, so each repository has one support level:

  • Active: spark-attic, the GLM-5.3 EXLR8 recipe and the TensorFold fork. We triage issues and review pull requests every week, and we update them as the engines move.
  • Maintained: we publish a new verified tag when an engine or model change needs one, and we answer issues.
  • Maintenance: stable as pinned. We answer issues and review bug-fix pull requests. No new features.
  • Frozen: the verified snapshot as pinned. No new support.
  • Archived: read-only. The README links to the replacement.

A bug report helps most with your hardware, driver version, recipe tag or engine commit, and the logs.

Earlier recipes (archived)

How we publish

  • Every number is our own measurement on the stated hardware, or is marked as someone else's.
  • We credit the model, the weights, the engine and the recipe each build stands on.
  • We never present a third-party quant as ours.
  • We pin exact revisions and hashes, and we list what is not measured yet.

bertholomus.com · Hugging Face · X: @bertholomusai

Pinned Loading

  1. spark-attic spark-attic Public

    More memory for CUDA on an NVIDIA GB10 (DGX Spark, ASUS Ascent GX10): lend the unused display carveout to vLLM, SGLang and TensorFold as KV cache

    Python 45 3

  2. glm-5.3-exlr8-4x-spark glm-5.3-exlr8-4x-spark Public

    GLM-5.3 (full) EXLR8 on four DGX Sparks with gr8: bit275 (~2.75 bpw) manifest, concurrency dial CONC 1/2/4/8, busy guard (BUSY=1, v2), optional SSD KV store and spark-attic

    Shell 10

  3. deepseek-v4.1-tensorfold-tp2-2xgb10 deepseek-v4.1-tensorfold-tp2-2xgb10 Public

    BertholomusAI recipe: DeepSeek-V4.1-Flash-1017 on 2x NVIDIA DGX Spark (GB10), our own TensorFold TP2 engine. Weights: bertholomus EXL3 2.5bpw quant (MIT). v0.7.5.

    Python 73 7

  4. deepseek-v4.1-tensorfold-tp4-4xgb10 deepseek-v4.1-tensorfold-tp4-4xgb10 Public

    DeepSeek-V4.1-Flash on 4x DGX Spark (GB10): exact 4-node TensorFold (Zig) recipe, vision, 1M context

    Python 14 1