GRepoWatch
Daily at 07:00 UTC+8Subscribe中文
← All topics

Local models, inference and training

Compare model runtimes, low-memory inference, training, gateways and routing by hardware and serving needs.

New projects join this topic as the archive updates. Browse the latest recorded projects first.

Choose by your task

43 projects

Last recorded ·

TensorFold

ashhart/TensorFold

LLM Inference Engine for Metal, CUDA and Vulkan.

Explore TensorFold →

Last recorded ·

free-claude-code

Alishahryar1/free-claude-code

Use Claude Code, Codex, Antigravity, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

Explore free-claude-code →

Last recorded ·

llama.cpp

ggml-org/llama.cpp

LLM inference in C/C++

Explore llama.cpp →

Last recorded ·

rails

rails/rails

Ruby on Rails

Explore rails →

Last recorded ·

kimi-k3-in-c

FareedKhan-dev/kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

Explore kimi-k3-in-c →

Last recorded ·

colibri

JustVugg/colibri

A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.

Explore colibri →

Last recorded ·

heretic

p-e-w/heretic

Training-free language model modification: Remove censorship, suppress slop, reduce sycophancy, and more

Explore heretic →

Last recorded ·

FreeToken

FlashML-org/FreeToken

FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.

Explore FreeToken →

Last recorded ·

nanochat

karpathy/nanochat

The best ChatGPT that $100 can buy.

Explore nanochat →

Last recorded ·

ComfyUI

Comfy-Org/ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.

Explore ComfyUI →

Last recorded ·

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Explore vllm →

Last recorded ·

tokenspeed

lightseekorg/tokenspeed

TokenSpeed is a speed-of-light LLM inference engine.

Explore tokenspeed →

Last recorded ·

airllm

lyogavin/airllm

AirLLM 2.8T LLM inference with single 4GB GPU

Explore airllm →

Last recorded ·

BitNet

microsoft/BitNet

Official inference framework for 1-bit LLMs

Explore BitNet →

Last recorded ·

pytorch

pytorch/pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Explore pytorch →

Last recorded ·

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Explore transformers →

Last recorded ·

tensorflow

tensorflow/tensorflow

An Open Source Machine Learning Framework for Everyone

Explore tensorflow →

Last recorded ·

litellm

BerriAI/litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

Explore litellm →

Last recorded ·

DeepGEMM

deepseek-ai/DeepGEMM

DeepGEMM: clean and efficient BLAS kernel library on GPU

Explore DeepGEMM →

Last recorded ·

pi

earendil-works/pi

AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI

Explore pi →

Last recorded ·

LongCat-Video

meituan-longcat/LongCat-Video

See the project overview and official repository for its purpose and setup.

Explore LongCat-Video →

Last recorded ·

tilelang

tile-ai/tilelang

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

Explore tilelang →

Last recorded ·

PageIndex

VectifyAI/PageIndex

📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG

Explore PageIndex →

Last recorded ·

Model-Optimizer

NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Explore Model-Optimizer →

Last recorded ·

stable-diffusion.cpp

leejet/stable-diffusion.cpp

Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++

Explore stable-diffusion.cpp →

Last recorded ·

higgsfield

higgsfield-ai/higgsfield

Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters

Explore higgsfield →

Last recorded ·

oh-my-hermes

rlaope/oh-my-hermes

All in one plugin for Hermes Agent ⚚ the coding intelligence, a long-term memory system and model optimized workflow packages

Explore oh-my-hermes →

Last recorded ·

llmfit

AlexsJones/llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

Explore llmfit →

Last recorded ·

OmniRoute

diegosouzapw/OmniRoute

Never stop coding. Free MIT AI gateway: one endpoint, 352 providers (150+ free), 1200+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 550+ contributors

Explore OmniRoute →

Last recorded ·

magnitude

magnitudedev/magnitude

Open source inference server that runs the best local models for your hardware, plugged into the agent you already use. Works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.

Explore magnitude →

Last recorded ·

timesfm

google-research/timesfm

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

Explore timesfm →

Last recorded ·

miles

radixark/miles

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

Explore miles →

Last recorded ·

sie

superlinked/sie

Self-host open-source models for agents. One cluster, one API.

Explore sie →

Last recorded ·

minimind

jingyaogong/minimind

🧠 Train a 64M-parameter LLM from scratch in just 2h!

Explore minimind →

Last recorded ·

freellmapi

tashfeenahmed/freellmapi

7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.

Explore freellmapi →

Last recorded ·

router

workweave/router

Model router for agentic systems. Routes every prompt to the right model in <50ms. Cut costs 40-70% with just an endpoint change.

Explore router →

Last recorded ·

marin

marin-community/marin

Open-source framework for the research and development of foundation models.

Explore marin →

Last recorded ·

sub2api

Wei-Shaw/sub2api

See the project overview and official repository for its purpose and setup.

Explore sub2api →

Last recorded ·

omlx

jundot/omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Explore omlx →

Last recorded ·

unsloth

unslothai/unsloth

Local UI to run and train LLMs and diffusion models, including Qwen3.8, Kimi K3, MiniMax-H3, Gemma 4, DeepSeek-V4, FLUX and more.

Explore unsloth →

Last recorded ·

Soup

MakazhanAlpamys/Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

Explore Soup →

Last recorded ·

LTX-2

Lightricks/LTX-2

Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.

Explore LTX-2 →

Last recorded ·

Switchyard

NVIDIA-NeMo/Switchyard

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Explore Switchyard →

Questions before choosing

Does downloading a model mean my machine can run it?

Check format, quantization, RAM or VRAM, hardware backends and context length. Distinguish local execution from cloud forwarding; measure latency and resource use on your task.

Projects are grouped using their saved purpose descriptions. Topic membership is a discovery aid, not an installation or performance test. Check each project’s overview and official documentation for requirements, sources and versions.

Explore other topics →