Topic: "llm-serving"
France-Travail/happy_vllm
A REST API for vLLM, production ready
Language: Python - Size: 859 KB - Last synced at: 4 days ago - Pushed at: 13 days ago - Stars: 20 - Forks: 2
France-Travail/benchmark_llm_serving
A library to benchmark LLMs via their API exposure
Language: Python - Size: 8.04 MB - Last synced at: 4 days ago - Pushed at: 5 months ago - Stars: 6 - Forks: 0
Related Topics