NVIDIA NIM (NVIDIA Inference Microservices) is a family of containerized inference microservices for serving AI models through documented APIs; the “NVIDIA NIM API” is the interface those services expose, OpenAI-compatible for LLMs. You can first evaluate selected models through NVIDIA-hosted endpoints in the API Catalog, then deploy a supported NIM container on your own NVIDIA-accelerated infrastructure. Current NIM for LLMs exposes native OpenAI-compatible endpoints through its inference backend, but compatibility is scoped to the model, endpoint and NIM release; it is not a promise that every NIM in the broader vision, speech, retrieval or scientific catalog implements every OpenAI feature.
What is NVIDIA NIM API?
NVIDIA Inference Microservices, or NIM, packages the components required to serve an AI model behind an API. Depending on the NIM, that package can include a model or model manifest, a supported inference engine, runtime configuration, request routing and operational endpoints. The objective is to remove much of the manual work involved in selecting an engine, matching it to supported hardware and exposing a stable service.
There are two closely related experiences that people often call the “NIM API.” The first is a NVIDIA-hosted endpoint in the NVIDIA API Catalog, intended for trying a model and building a prototype. The second is the API exposed by a NIM container you operate on supported NVIDIA infrastructure. Hosted access is a service; self-hosted NIM is deployable inference software. For the deployment choice, read the hosted vs self-hosted NVIDIA NIM deployment comparison.
NIM is broader than chat models. NVIDIA’s catalog groups microservices around large language models, retrieval, visual and multimodal models, speech, healthcare, route optimization and climate simulation. Each model page and API reference is therefore more authoritative than any fixed list in an overview article.
How NVIDIA NIM works: hosted API vs self-hosted containers
| Question | NVIDIA-hosted API Catalog endpoint | Self-hosted NIM container |
|---|---|---|
| Who runs inference? | NVIDIA operates the live endpoint. | Your team runs the container on supported NVIDIA-accelerated infrastructure. |
| Best fit | Model discovery, evaluation and application prototyping. | Controlled development or production deployment, subject to the applicable NIM offering and license. |
| Infrastructure work | No local GPU deployment is required for the hosted call. | You manage GPU capacity, model storage, networking, updates and orchestration. |
| Data path | Requests go to an NVIDIA-hosted service. | Requests stay in the environment and network boundary you operate. |
| Source of truth | The live model page and its API reference in the catalog. | The NIM release documentation, support matrix and model profile for your container version. |
NVIDIA describes the API Catalog endpoints as a way to explore models and prototype applications. Production support, security lifecycle and commercial use depend on the chosen NIM offering and NVIDIA AI Enterprise terms. Check the current owner pages for access, cost and limits rather than assuming prototype conditions carry into production.
Which models and API endpoints does NVIDIA NIM support?
The catalog changes continuously. As of the verification date, NVIDIA’s LLM support material includes NVIDIA Nemotron models and model families from publishers such as Meta, OpenAI, Qwen, DeepSeek, MiniMax and Z.ai. The wider catalog also contains retrieval, vision, multimodal, speech and industry-specific microservices. Availability as a hosted endpoint, downloadable NIM and certified deployment profile can differ for the same model.
For self-hosted LLMs, the current documentation distinguishes model-specific NIMs, with a curated manifest and validated profiles, from model-free NIM, which can serve a compatible model selected at runtime. Model-free does not mean universal: the model architecture must still be supported by the inference backend, and the hardware must satisfy the relevant requirements.
| API area | Typical interface | What to verify |
|---|---|---|
| LLM chat and generation | /v1/chat/completions, /v1/completions and, in current NIM LLM releases, /v1/responses. | Model ID, supported parameters, streaming and tool behavior for that release. |
| LLM discovery and operations | /v1/models, health, readiness, metadata and metrics endpoints. | Hosted and self-hosted management surfaces are not necessarily identical. |
| Retrieval | Embedding and reranking APIs exposed by the selected retrieval NIM. | Input format, maximum payload and model-specific scoring behavior. |
| Vision, speech and scientific NIMs | Industry-standard or NVIDIA-documented APIs for that domain. | Do not assume an OpenAI chat schema; use the model page’s reference. |
Use the live API Catalog to discover hosted and downloadable models. Before self-hosting, use the current NIM LLM support matrix to verify the exact model, profile and hardware combination.
Is NVIDIA NIM API OpenAI-compatible?
Yes for the documented NIM LLM interface, with an important qualifier. Current NVIDIA documentation says NIM LLM exposes an OpenAI-compatible inference API backed by vLLM. Its API reference lists familiar routes including chat completions, completions, responses and model discovery. That makes it possible to reuse many OpenAI-style clients and request shapes.
“OpenAI-compatible” is not the same as “identical to every OpenAI API feature.” Optional parameters, response fields, tool-calling behavior, structured output, multimodal input and endpoint coverage can vary by model and NIM release. NVIDIA’s current 2.x architecture intentionally exposes more upstream model and vLLM behavior instead of hiding every difference behind an emulation layer.
Compatibility checklist before migration
- Confirm that the selected model exposes the route your application uses.
- Check the model ID returned by /v1/models or specified by the hosted catalog page.
- Test streaming, tools, structured output and token accounting instead of assuming parity.
- Separate the endpoint URL and credentials from application logic so hosted and self-hosted environments can be switched cleanly.
- Read the API reference for the exact NIM version deployed in production.
Can you try NVIDIA NIM API for free?
Yes, for prototyping. Hosted NIM endpoints in the API Catalog are free for NVIDIA Developer Program members. You sign in at build.nvidia.com, generate one nvapi- API key and use it with the hosted models through the OpenAI-compatible base URL https://integrate.api.nvidia.com/v1. Usage is capped by model-specific rate limits rather than credits, and production use requires NVIDIA AI Enterprise. The Developer Program also covers development and testing of eligible self-hosted NIM.
The full walkthrough, covering account verification, key generation, a first call and fixes for 403 and 429 errors, lives in one place: how to get a free NVIDIA NIM API key. For quotas, licensing and production cost, check NVIDIA NIM pricing and rate limits.
NVIDIA NIM architecture and deployment options
What happens inside a current NIM LLM container?
NVIDIA’s current NIM LLM architecture packages an orchestration layer around vLLM. A proxy exposes inference and management routes, reports liveness immediately, reports readiness after the model has loaded, and forwards supported requests to the inference backend. At startup, NIM resolves configuration, selects a model profile, obtains model artifacts when needed and starts serving after the backend is ready.
The container also exposes operational surfaces such as health, metadata and Prometheus-compatible metrics. This is one of the practical differences between a raw model script and a deployable inference service: an orchestrator can distinguish “container is alive” from “model is ready,” monitor latency and GPU utilization, and replace an unhealthy instance.
Where can NIM be deployed?
| Deployment pattern | Use it for | What you operate |
|---|---|---|
| Single container | Evaluation, development and a controlled single-node service. | Supported host, NVIDIA driver/container stack, model cache and service endpoint. |
| Kubernetes | Replicas, scheduling, rolling operations and production orchestration. | GPU-capable cluster plus Helm, NIM Operator, KServe, OpenShift or another supported path. |
| Cloud or data center | Deploying near existing applications and governed data. | Capacity, networking, observability and the applicable NIM license/support lifecycle. |
| Hosted catalog endpoint | Testing the model and API before infrastructure work. | Your client application only; NVIDIA operates the endpoint. |
Do not select hardware from a generic memory estimate. NVIDIA publishes release-specific support matrices that map models, precision, tensor-parallel profiles and verified GPU SKUs. A larger context window or model can also change runtime memory needs, so the current support matrix and sizing tests are the correct decision inputs.
NVIDIA NIM use cases: who should use it?
NIM is a strong fit when a team wants a documented API during development but also needs the option to operate inference inside its own infrastructure. Common use cases in NVIDIA’s documentation include chat and virtual assistants, summarization, content generation, translation and sentiment analysis. The broader catalog extends to retrieval, image and video workflows, multimodal understanding, speech and scientific or industry-specific workloads.
Choose NIM when:
- you want to evaluate a model through a hosted endpoint and later deploy a related NIM under your control;
- your platform team already operates supported NVIDIA GPUs or Kubernetes GPU nodes;
- data location, network boundaries and operational observability matter;
- validated profiles and an enterprise support lifecycle are more valuable than assembling an inference stack by hand;
- your application benefits from an OpenAI-style LLM API but needs self-hosted deployment.
Choose another approach when:
- you need a completely hands-off production API and do not want to operate GPU infrastructure;
- the required model, architecture or hardware is not covered by the current NIM support path;
- your primary task is training rather than inference serving;
- you need a vendor-neutral runtime across non-NVIDIA accelerators;
- the simplest managed endpoint is more important than deployment portability.
NVIDIA NIM vs other inference platforms
The useful comparison is not “which provider has the biggest catalog this month?” It is who owns the runtime and how much operational control you need.
| Approach | Main advantage | Main trade-off | Best decision signal |
|---|---|---|---|
| NVIDIA NIM | Hosted evaluation plus a supported path to a self-hosted container and enterprise lifecycle. | You operate supported NVIDIA infrastructure for self-hosted deployment. | You need infrastructure control without building the entire serving layer from scratch. |
| Managed inference API | Fastest operational start; the provider manages serving capacity. | Less control over runtime, data path and deployment portability. | You want an API service, not an inference platform to operate. |
| Direct open-source engine | Maximum control over the engine and upstream release cadence. | Your team owns packaging, validation, security updates and operational integration. | You have inference specialists and need custom behavior beyond a packaged NIM. |
| Cloud-native model service | Close integration with one cloud’s identity, networking and monitoring. | Portability and available models depend on that cloud service. | Your application and governance are already standardized on one cloud. |
For a provider-by-provider decision, compare NIM with other inference APIs. For adjacent decisions, see NVIDIA NIM alternatives, LLM inference cost comparison and the vLLM vs TensorRT-LLM vs SGLang comparison. Keep those comparison and cost intents separate from this overview so frequently changing prices and rate limits do not obscure NIM’s architecture.
Which NVIDIA NIM guide do you need?
Stay on this page for hosted vs self-hosted architecture, model discovery, compatibility and use cases.
You are reading the NIM overviewUse the dedicated setup owner for account requirements, key generation and a first authenticated call.
Get a free NVIDIA NIM API keyUse the pricing owner for current quotas, rate limits, licensing and 429 troubleshooting.
Check NVIDIA NIM pricing and rate limitsFAQ
Update log
- Clarified that this page explains the NIM platform and API, and made the hand-off to the dedicated NVIDIA NIM API key guide more prominent (key setup, verification and error fixes live there, not here).
- Made this page the NIM overview hub and added natural routes to the key, pricing, alternatives, inference-cost and self-hosted engine owner pages after a current NVIDIA documentation review.
- Refocused the page on NIM architecture, model discovery, OpenAI compatibility and deployment choices; removed the duplicated key tutorial, volatile credit/rate tables, GPU sandbox detail and pricing examples; added routing to the dedicated owner pages.
- Previous broad NVIDIA Build and NIM guide.
Sources and further reading
Variable platform facts were verified only against current NVIDIA-owned documentation and catalog pages on September 6, 2026. Model availability, support profiles, access conditions and endpoints can change; follow the linked version-specific references before deployment.
- NVIDIA – NVIDIA NIM overview and use-case catalog.
- NVIDIA – API Catalog Quickstart Guide.
- NVIDIA – Run NIM anywhere: hosted and downloadable access conditions.
- NVIDIA – General NIM FAQ.
- NVIDIA – NVIDIA API Catalog.
- NVIDIA – NVIDIA NIM for Large Language Models overview.
- NVIDIA – Get started with NIM LLM.
- NVIDIA – NIM LLM API Reference.
- NVIDIA – NIM LLM Architecture.
- NVIDIA – Support Matrix for Certified NIMs.
- NVIDIA – Model-Free NIM.
- NVIDIA – NIM LLM Kubernetes deployment options.

