Automodel
Nemo AutoModel is a Pytorch DTensor‑native SPMD open-source training library under NVIDIA NeMo Framework, designed to streamline and scale training and finetuning for LLMs, VLMs, diffusion models, and retrieval models. Designed for flexibility, reproducibility, and scale, NeMo AutoModel enables both small-scale experiments and massive multi-GPU, multi-node deployments for fast experimentation in research and production environments. What you can expect: Hackable with a modular design that allows easy integration, customization, and quick research prototypes. Minimal ceremony: YAML-driven recipes; override any field using CLI. High performance and flexibility with custom kernels and DTensor support. Seamless integration with Hugging Face for day-0 model support, ease of use, and wide range of supported models. Efficient resource management using Kubernetes and Slurm, enabling scalable and flexible deployment across configurations. Documentation with step-by-step guides and runnable examples. One program, any scale: The same training script runs on 1 GPU or 1000+ by changing the mesh. PyTorch Distributed native: Partition model/optimizer states with DeviceMesh + placements (Shard, Replicate). SPMD first: Parallelism is configuration. No model rewrites when scaling up or changing strategy. Decoupled concerns: Model code stays pure PyTorch; parallel strategy lives in config. Composability: Mix tensor, sequence, and data parallel by editing placements. Portability: Fewer bespoke abstractions; easier to reason about failure modes and restarts. Feature Roadmap Getting Started LLM Pre-training Supervised Fine-Tuning (SFT) Parameter-Efficient Fine-Tuning (PEFT) VLM Supervised Fine-Tuning (SFT) Parameter-Efficient Fine-Tuning (PEFT) Supported Models Performance Interoperability Contributing License TL;DR: SPMD turns “how to parallelize” into a runtime layout choice, not a code fork.
View Automodel on GitHub
Nemo AutoModel is a Pytorch DTensor‑native SPMD open-source training library under NVIDIA NeMo Framework, designed to streamline and scale training and finetuning for LLMs, VLMs, diffusion models, and retrieval models. Designed for flexibility, reproducibility, and scale, NeMo AutoModel enables both small-scale experiments and massive multi-GPU, multi-node deployments for fast experimentation in research and production environments.
What you can expect: Hackable with a modular design that allows easy integration, customization, and quick research prototypes. Minimal ceremony: YAML-driven recipes; override any field using CLI. High performance and flexibility with custom kernels and DTensor support. Seamless integration with Hugging Face for day-0 model support, ease of use, and wide range of supported models. Efficient resource management using Kubernetes and Slurm, enabling scalable and flexible deployment across configurations. Documentation with step-by-step guides and runnable examples. One program, any scale: The same training script runs on 1 GPU or 1000+ by changing the mesh. PyTorch Distributed native: Partition model/optimizer states with DeviceMesh + placements (Shard, Replicate). SPMD first: Parallelism is configuration. No model rewrites when scaling up or changing strategy. Decoupled concerns: Model code stays pure PyTorch; parallel strategy lives in config. Composability: Mix tensor, sequence, and data parallel by editing placements. Portability: Fewer bespoke abstractions; easier to reason about failure modes and restarts. Feature Roadmap Getting Started LLM Pre-training Supervised Fine-Tuning (SFT) Parameter-Efficient Fine-Tuning (PEFT) VLM Supervised Fine-Tuning (SFT) Parameter-Efficient Fine-Tuning (PEFT) Supported Models Performance Interoperability Contributing License
TL;DR: SPMD turns “how to parallelize” into a runtime layout choice, not a code fork.
Automodel at a glance
| Stars | 985 |
|---|---|
| Forks | 320 |
| Language | Python |
| License | Apache-2.0 |
| Last update | 2026-09-29 |
| Contributors | 135 |
How to install Automodel
bash uv venv
Where Automodel is listed
Navid Moazzez
AI business strategist & AI OS builder
Navid Moazzez helps creators and founders master AI and build their own AI Operating System (AI OS) to automate their business and life.
Navid.me is reader-supported. When you buy through links on this site, I may earn an affiliate commission. Learn more.
GitHub repos like this
More repo topics
More free tools
X
Bluesky
LinkedIn
Threads
Mastodon
Instagram
TikTok
Pinterest
YouTube
Amazon KDP
Apple Books
Amazon
Substack
Spotify
Apple Podcasts
Related MCP servers & CLIs
Free AI newsletterThe most actionable AI newsletter for founders
Every week, get proven AI strategies, curated tools, and step-by-step systems to grow your audience, create better content, and build a profitable creator business.
No fluff, no filler, no BS. Just five minutes each week that might level up your online business and life.
P.S. Sign up now to get free access to my ultimate AI tools guide for creators.
