LATEST RELEASE: You are currently on the main branch which tracks under-development progress towards the next release. The latest release of the Triton Model Analyzer is 1.6.0 and is available on branch r21.07.
Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models. These reports will help the user better understand the trade-offs in different configurations and choose a configuration that maximizes the performance of Triton Inference Server.
- Installation
- Quick Start
- Model Analyzer CLI
- Launch Modes
- Model Analyzer Metrics
- Configuring Model Analyzer
- Model Config Search
- Checkpointing
- Model Analyzer Reports
- Deployment with Kubernetes
We appreciate any feedback, questions or bug reporting regarding this project. When help with code is needed, follow the process outlined in the Stack Overflow (https://stackoverflow.com/help/mcve) document. Ensure posted examples are:
-
minimal – use as little code as possible that still produces the same problem
-
complete – provide all parts needed to reproduce the problem. Check if you can strip external dependency and still show the problem. The less time we spend on reproducing problems the more time we have to fix it
-
verifiable – test the code you're about to provide to make sure it reproduces the problem. Remove all other problems that are not related to your request/question.