All pull requests

Pull requests

• Tgi router latency metrics

  #10·pathorn opened Dec 15, 2023·

  ·

• Add support for Mistral

  #9·pathorn opened Dec 7, 2023·

  ·
  1

• Update flash attention to version v2.3.6

  #8·pathorn opened Dec 1, 2023·

  ·

• Expose raw logit weights to the API if request_logits=true

  #7·pathorn opened Nov 28, 2023·

  ·
  4

• Print stats about the KV Cache

  #6·NikolaBorisov opened Oct 10, 2023·

  ·

• Update deps

  #5·NikolaBorisov opened Oct 7, 2023·

  ·