🚀 An open-source Cost & Resource Optimization Platform for LLMs. Be Frugal! 💰
Numexa is an AI-driven cost and resource optimization tool designed to enhance operational efficiency. It achieves this by leveraging contextual insights derived from usage metrics. Numexa employs cutting-edge techniques such as intelligent caching and data retrieval, harnessing the power of vector databases to streamline operations. Explore how Numexa can revolutionize your resource management and cost-saving endeavors.
-
📝 Model agnostic functionality records unlimited requests from various providers like OpenAI, Cohere, Anthropic and more.
-
📋 Model management
-
🔔 Alerting & Notification with predefined policies, like error rate, threshold, cost, etc.
-
💾 Caching, Custom Rate Limits, and Retries,
-
📊 Track costs and latencies by users, applications, and endpoints
-
🔜 (Coming soon) Intellegient caching and data retrieval
-
🔜 (Coming soon) Cost and resource optimization
Before you begin, ensure you have the following installed on your system:
- Git: Installation Guide
- Docker: Installation Guide
- Docker Compose: Installation Guide
- Make: Installation Guide
-
Clone the Repository:
git clone <repository_url> cd <repository_directory>
-
Build and Start the Services: Run the following commands to build and start the project services
make all docker compose -f docker-compose.dev.yaml up -d
-
Verify Services: After running the above commands, your project services should be up and running. You can verify this by checking the logs
Join our #Discord or drop email at [email protected]