Repository navigation

perf: worker processes, compiled TabICL, GPU preprocessing, int8 KV, requirement + host admission (0.12.0) - #40

Merged

amrit110 merged 39 commits into

mainfrom

feat/memory-optimizations

Oct 18, 2026

Merged

perf: worker processes, compiled TabICL, GPU preprocessing, int8 KV, requirement + host admission (0.12.0)#40
amrit110 merged 39 commits into
mainfrom
feat/memory-optimizations

Commits on Oct 18, 2026
- feat(tabicl): compressed ICL KV cache (fp16/int8) + memory benchmarks
  
  amrit110
  and
  claude
  committed
  f64a122
- perf(tabicl): int8 ICL KV cache by default (A100 study: identical labels/metrics on 7 datasets, ~0.52x resident)
  
  amrit110
  and
  claude
  committed
  896550a
- feat(memory): admit fits by resident + working floor, not unconstrained peak
  
  amrit110
  and
  claude
  committed
  996bdd8
- feat(memory): extrapolate uncalibrated shapes as resident (linear in rows) + floor
  
  amrit110
  and
  claude
  committed
  8167b5a
- fix(tabicl): release pinned host memory after calls; host-memory admission guard
  
  amrit110
  and
  claude
  committed
  fa6bdca
- fix(tabicl): give tabicl a disk offload dir; leave host headroom for its staging
  
  amrit110
  and
  claude
  committed
  8809596
- fix: review findings on #40
  
  amrit110
  and
  claude
  committed
  4a503ac
- fix(engine): count each device's baseline memory (weights, CUDA context) in admission
  
  amrit110
  and
  claude
  committed
  cb037e3
- fix(multidevice): contexts moving between tiers stay routable
  
  amrit110
  and
  claude
  committed
  fd2227e
- bench: multi-GPU scaling benchmark (threads vs capped threads vs processes)
  
  amrit110
  and
  claude
  committed
  2019511
- feat(memory): per-model requirement calibrations; width-aware resident slope
  
  amrit110
  and
  claude
  committed
  5a34568
- bench: predict/fit overlap on one GPU (head-of-line study)
  
  amrit110
  and
  claude
  committed
  f2c2671
- bench(pressure): fresh process per shape; --max-slowdown for models that degrade instead of failing
  
  amrit110
  and
  claude
  committed
  6691b19
- feat(serve): one worker process per GPU behind the multi-device router
  
  amrit110
  and
  claude
  committed
  59b103a
- feat(memory): TabPFN-3.5 requirement calibration (A100); width-dependent resident extrapolation
  
  amrit110
  and
  claude
  committed
  31561eb
- fix(tabicl): one active-context cache per thread on the shared backbone
  
  amrit110
  and
  claude
  committed
  d2316ec
- feat(engine): light predicts run beside a fit on the same device (head-of-line)
  
  amrit110
  and
  claude
  committed
  7c106bf
- fix: review findings on the worker-process and scheduling commits
  
  amrit110
  and
  claude
  committed
  da94ff7
- perf(tabicl): prepare a fit's CPU half before taking the device
  
  amrit110
  and
  claude
  committed
  6d88c44
- fix(workers): close worker shards at interpreter exit (no spurious death reports)
  
  amrit110
  and
  claude
  committed
  04d3751
- perf(tabicl): cache ssmax log(n) tensors (one GPU sync per length, not per call)
  
  amrit110
  and
  claude
  committed
  dbdaf41
- feat(memory): extend the TabPFN-3.5 requirement grid to large tables
  
  amrit110
  and
  claude
  committed
  78ef4f2
- docs: roadmap and changelog for worker processes, scheduling, profiling, TabPFN calibration
  
  amrit110
  and
  claude
  committed
  41c7b44
- docs(readme): worker processes and predicts beside fits
  
  amrit110
  and
  claude
  committed
  4d3382f
- feat(memory): admit host memory by estimated bytes, not large-fit slots
  
  amrit110
  and
  claude
  committed
  44471ff
- fix(memory): FIFO node-wide host admission that survives worker death
  
  amrit110
  and
  claude
  committed
  613a2ac
- fix(serve): export HostMemoryUnavailableError (the app failed to start)
  
  amrit110
  and
  claude
  committed
  ae441e8
- perf(tabicl): torch.compile the attention blocks (1.7-2.2x GPU half)
  
  amrit110
  and
  claude
  committed
  e7ae9c1
- docs: torch.compile results in roadmap, changelog and README; bench_compile covers TabPFN
  
  amrit110
  and
  claude
  committed
  2de5ff2
- fix: review of the host guard and compile changes
  
  amrit110
  and
  claude
  committed
  950e5ca
- feat(multidevice): pluggable placement policy and residency events (#39)
  
  amrit110
  and
  claude
  committed
  e039171
- perf(tabicl): fit Yeo-Johnson power transforms on the GPU, all columns at once
  
  amrit110
  and
  claude
  committed
  68b7163
- fix: review of placement/events and the GPU Yeo-Johnson
  
  amrit110
  and
  claude
  committed
  064d2b8
- bench: memory-pressure search with compiled blocks and fast preprocessing
  
  amrit110
  and
  claude
  committed
  d1d3607
- fix(memory): compiled TabICL is admitted against its own requirement grid
  
  amrit110
  and
  claude
  committed
  025053c
- bench+docs: predict throughput is host-bound (~21/s per GPU, GPU 20% busy); roadmap status for #33, #35, #37
  
  amrit110
  and
  claude
  committed
  4e7a950
- fix: final pre-release review of 0.12.0
  
  amrit110
  and
  claude
  committed
  454a85a
- fix(engine): make room against the device's real free memory too
  
  amrit110
  and
  claude
  committed
  b13c9dd
- test: import the shared multidevice helpers the way CI's pytest path allows; ruff format
  
  amrit110
  and
  claude
  committed
  b417dd8

Commits

Commits on Oct 18, 2026