Local models, inference and training
Compare model runtimes, low-memory inference, training, gateways and routing by hardware and serving needs.
New projects join this topic as the archive updates. Browse the latest recorded projects first.
Choose by your task
43 projects
free-claude-code
Alishahryar1/free-claude-code
Use Claude Code, Codex, Antigravity, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)
Explore free-claude-code →kimi-k3-in-c
FareedKhan-dev/kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
Explore kimi-k3-in-c →colibri
JustVugg/colibri
A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
Explore colibri →heretic
p-e-w/heretic
Training-free language model modification: Remove censorship, suppress slop, reduce sycophancy, and more
Explore heretic →FreeToken
FlashML-org/FreeToken
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
Explore FreeToken →ComfyUI
Comfy-Org/ComfyUI
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.
Explore ComfyUI →vllm
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Explore vllm →tokenspeed
lightseekorg/tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
Explore tokenspeed →pytorch
pytorch/pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
Explore pytorch →transformers
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Explore transformers →tensorflow
tensorflow/tensorflow
An Open Source Machine Learning Framework for Everyone
Explore tensorflow →litellm
BerriAI/litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Explore litellm →DeepGEMM
deepseek-ai/DeepGEMM
DeepGEMM: clean and efficient BLAS kernel library on GPU
Explore DeepGEMM →LongCat-Video
meituan-longcat/LongCat-Video
See the project overview and official repository for its purpose and setup.
Explore LongCat-Video →tilelang
tile-ai/tilelang
Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels
Explore tilelang →PageIndex
VectifyAI/PageIndex
📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG
Explore PageIndex →Model-Optimizer
NVIDIA/Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
Explore Model-Optimizer →stable-diffusion.cpp
leejet/stable-diffusion.cpp
Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++
Explore stable-diffusion.cpp →higgsfield
higgsfield-ai/higgsfield
Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters
Explore higgsfield →oh-my-hermes
rlaope/oh-my-hermes
All in one plugin for Hermes Agent ⚚ the coding intelligence, a long-term memory system and model optimized workflow packages
Explore oh-my-hermes →llmfit
AlexsJones/llmfit
Hundreds of models & providers. One command to find what runs on your hardware.
Explore llmfit →OmniRoute
diegosouzapw/OmniRoute
Never stop coding. Free MIT AI gateway: one endpoint, 352 providers (150+ free), 1200+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 550+ contributors
Explore OmniRoute →magnitude
magnitudedev/magnitude
Open source inference server that runs the best local models for your hardware, plugged into the agent you already use. Works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.
Explore magnitude →timesfm
google-research/timesfm
TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.
Explore timesfm →miles
radixark/miles
Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.
Explore miles →freellmapi
tashfeenahmed/freellmapi
7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.
Explore freellmapi →router
workweave/router
Model router for agentic systems. Routes every prompt to the right model in <50ms. Cut costs 40-70% with just an endpoint change.
Explore router →marin
marin-community/marin
Open-source framework for the research and development of foundation models.
Explore marin →sub2api
Wei-Shaw/sub2api
See the project overview and official repository for its purpose and setup.
Explore sub2api →omlx
jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Explore omlx →unsloth
unslothai/unsloth
Local UI to run and train LLMs and diffusion models, including Qwen3.8, Kimi K3, MiniMax-H3, Gemma 4, DeepSeek-V4, FLUX and more.
Explore unsloth →Soup
MakazhanAlpamys/Soup
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
Explore Soup →LTX-2
Lightricks/LTX-2
Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.
Explore LTX-2 →Switchyard
NVIDIA-NeMo/Switchyard
Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Explore Switchyard →Questions before choosing
Does downloading a model mean my machine can run it?
Check format, quantization, RAM or VRAM, hardware backends and context length. Distinguish local execution from cloud forwarding; measure latency and resource use on your task.
Projects are grouped using their saved purpose descriptions. Topic membership is a discovery aid, not an installation or performance test. Check each project’s overview and official documentation for requirements, sources and versions.
Explore other topics →