airllm
AirLLM 2.8T LLM inference with single 4GB GPU
REPOWATCH / EXPLORE
Search the full Trending archive by name, purpose, tag or language.
SEARCH RESULTS
AirLLM 2.8T LLM inference with single 4GB GPU
Official inference framework for 1-bit LLMs
Tensors and Dynamic neural networks in Python with strong GPU acceleration
An Open Source Machine Learning Framework for Everyone
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
DeepGEMM: clean and efficient BLAS kernel library on GPU
AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI
Explore the project overview and official repository.
Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels
📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.