Skip to content

generate: clamp a stage's budget to the room left in the context instead of refusing the request - #25

Open
timoncool wants to merge 1 commit into
ServeurpersoCom:masterfrom
timoncool:semantic-budget-kv
Open

timoncool wants to merge 1 commit into
ServeurpersoCom:masterfrom
timoncool:semantic-budget-kv

Shrink a stage's budget to the room the KV cache leaves instead of re…

426716a
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs