GRepoWatch
Daily at 07:00 UTC+8Subscribe中文
← Back to today’s ranking

NVIDIA /

Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Visit GitHub repository ↗
Last recorded · 2026-09-27#3

Rank in that snapshot

+301Daily Stars4,904Total Stars

Project overview

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Recorded Trending signals

NVIDIA/Model-Optimizer was recorded 2 times between 2026-09-25 and 2026-09-27. Its best recorded rank is #3. The latest saved snapshot reports 301 daily Stars and 4,904 total Stars. These are archive observations, not quality ratings or continuous activity measurements.

Official project documentation

The following is an excerpt from the project’s official README, fetched from GitHub. It may change independently of this archive. RepoWatch has not installed or functionally tested this project.

NVIDIA Model Optimizer

Documentation | Roadmap | Announcement Blogs


NVIDIA Model Optimizer (referred to as Model Optimizer, or ModelOpt) is a library comprising state-of-the-art model optimization techniques including quantization, pruning, Neural Architecture Search (NAS), distillation, speculative decoding and sparsity to accelerate models.

[Input] Model Optimizer currently supports inputs of a Hugging Face, PyTorch or ONNX model.

[Optimize] Model Optimizer provides Python APIs for users to easily compose the above model optimization techniques and export an optimized quantized checkpoint. Model Optimizer is also integrated with NVIDIA Megatron-Bridge, Megatron-LM and Hugging Face Accelerate for training required inference optimization techniques.

[Export for deployment] Seamlessly integrated within the NVIDIA AI software ecosystem, the quantized checkpoint generated from Model Optimizer is ready for deployment in downstream inference frameworks like SGLang, TensorRT-LLM, TensorRT, or vLLM. The unified Hugging Face export API now supports both transformers and diffusers models.

Latest News

Previous News

Install

To install stable release packages for Model Optimizer with pip from PyPI:

pip install -U nvidia-modelopt[all]

Model Optimizer will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.

To install from source in editable mode with all development dependencies or to use the latest features, run:

# Clone the Model Optimizer repository
git clone git@github.com:NVIDIA/Model-Optimizer.git
cd Model-Optimizer

pip install -e .[dev]

You can also directly use NVIDIA container images, which have Model Optimizer pre-installed:

  • nvcr.io/nvidia/pytorch:<version>-py3
  • nvcr.io/nvidia/nemo:<version>
  • nvcr.io/nvidia/tensorrt-llm/release:<version>

Before pulling and using the container images, please review their respective license terms. Make sure to upgrade Model Optimizer to the latest version as described above. Visit our installation guide for more fine-grained control on installed dependencies or for alternative docker images and environment variables to setup.

Techniques

TechniqueDescriptionGetting startedExamples
Post Training QuantizationCompress model size by 2x-4x, speeding up inference while preserving model quality![Start here][HF LLMs / VLMs] [Megatron-Bridge LLMs / VLMs] [Diffusers] [ONNX] [Windows]
Quantization Aware Training / DistillationRefine accuracy of quantized models even further with a few training steps![Start here][Hugging Face] [Megatron-Bridge]
PruningReduce your model parameters or memory footprint and accelerate inference by removing unnecessary weights![Start here][General] [Megatron-Bridge]
DistillationReduce deployment model size by teaching small models to behave like larger models![Start here][Hugging Face] [Megatron-Bridge] [Megatron-LM]
Speculative DecodingTrain draft modules to predict extra tokens during inference![Start here][Hugging Face] [Megatron-LM]
SparsityEfficiently compress your model by storing only its non-zero parameter values and their locations[Start here][Hugging Face]

Pre-Quantized Checkpoints

Resources

Model Support Matrix

Model TypeSupport Matrix
LLM / VLM QuantizationView Support Matrix
Diffusers QuantizationView Support Matrix
ONNX QuantizationView Support Matrix
Windows QuantizationView Support Matrix
Quantization Aware TrainingView Support Matrix
PruningView Support Matrix
DistillationView Support Matrix
Speculative DecodingView Support Matrix

Deprecation Policy

Model Optimizer follows a structured approach to managing deprecated features:

  • Communication: Deprecation notices are documented in the Changelog. Deprecated items include source code statements indicating deprecation timing, with runtime warnings issued upon use.
  • Migration Period: Since Model Optimizer is still pre-1.0, we provide a 1-release (~1-month) migration period after deprecation. During this window, deprecated features continue functioning while issuing warnings.
  • Scope: The policy addresses both complete deprecations (entire APIs removed) and partial ones (specific parameters removed while methods remain).
  • Removal: Following the migration period, deprecated elements are removed in alignment with semantic versioning standards, potentially including breaking changes in minor version updates while Model Optimizer remains in 0.x.

Citation

If you use NVIDIA Model Optimizer in your research, please cite it as follows:

@misc{nvidia-modelopt,
  author       = {{NVIDIA Corporation}},
  title        = {{NVIDIA Model Optimizer}},
  howpublished = {\url{https://github.com/NVIDIA/Model-Optimizer}},
  year         = {2024--2026},
  note         = {GitHub repository}
}

Contributing

Model Optimizer is now open source! We welcome any feedback, feature requests and PRs. Please read our Contributing guidelines for details on how to contribute to this project.

AI Agents

ModelOpt's agent skills can be installed from this repository and used in any workspace.

Claude Code

claude plugin marketplace add https://github.com/NVIDIA/Model-Optimizer.git
claude plugin install modelopt@modelopt

Codex

codex plugin marketplace add https://github.com/NVIDIA/Model-Optimizer.git

Then open /plugins, select the modelopt marketplace, and install modelopt. Contributors can also use the skills directly from a checkout. See the agent tooling notes.

Top Contributors

Happy optimizing!