Issues

• Llama-API info about rate limits and status monitoring

  c0sogi/llama-api#30 · Filter by author AndreiBlindu opened on Mar 25, 2025

• FastAPI + llamapi issue

  c0sogi/llama-api#29 · Filter by author Samraw003 opened on May 15, 2024

• Stopped working after enabling CUDA

  c0sogi/llama-api#28 · Filter by author alexellis opened on Jan 12, 2024

• High RAM and CPU usage

  c0sogi/llama-api#27 · Filter by author delta-whiplash opened on Dec 21, 2023

• Usage of embedding through langchain

  c0sogi/llama-api#26 · Filter by author jordandroid opened on Dec 11, 2023

• Support min_p sampler

  c0sogi/llama-api#25 · Filter by author atisharma opened on Nov 15, 2023

• How can I use a specific prompt template?

  c0sogi/llama-api#24 · Filter by author Dougie777 opened on Nov 7, 2023

• how to run this api in cpu only mode

  c0sogi/llama-api#23 · Filter by author delta-whiplash opened on Nov 7, 2023
  1

• Zephyr7b gives gobbly gook output but Mistral7b works fine.

  c0sogi/llama-api#22 · Filter by author Dougie777 opened on Nov 1, 2023

• exllama GPU split

  c0sogi/llama-api#21 · Filter by author atisharma opened on Oct 27, 2023
  1

• exllamav2

  c0sogi/llama-api#20 · Filter by author ehartford opened on Oct 20, 2023
  2

• Any way to define embeddings model in model_definitions.py?

  c0sogi/llama-api#19 · Filter by author morgendigital opened on Oct 9, 2023
  1

• Long generations dont return data but server says 200 OK. Swagger screen just says LOADING forever.

  c0sogi/llama-api#18 · Filter by author Dougie777 opened on Sep 17, 2023
  5

• Is there a way to use this on google Colab and have the url be public?

  c0sogi/llama-api#11 · Filter by author ashercn97 opened on Sep 10, 2023
  1

• Proxy to openAI

  c0sogi/llama-api#9 · Filter by author kreolsky opened on Sep 8, 2023
  2

• Using with LangChain instead openai API

  c0sogi/llama-api#8 · Filter by author kreolsky opened on Sep 3, 2023
  1