NVIDIA V100: what this GPU still does very well for AI, and its limits

By Mouhamed BANKOLE, CEO at ITNET Technologies · Updated November 6, 2026 · Reading time: 9 min

You are looking for a GPU to run an AI model, and the NVIDIA V100 keeps showing up in rental offers, at a much lower price than recent cards. The question is simple: is this GPU, released in 2017, still good enough for your projects, or will you hit software that no longer supports it? This article covers its specifications, the PCIe and SXM2 differences, what still runs on it in 2026, its limits and its rental cost. It ends with a decision grid and a getting-started procedure.

In short

• The NVIDIA V100 is a Volta-generation data center GPU, with 16 or 32 GB of HBM2 memory and 640 Tensor Cores.
• It is still useful for inference on mid-sized AI models, image generation and scientific computing, thanks to its fast memory.
• Its main limit is software: CUDA 13 and vLLM now require a newer generation, while Ollama still supports it.
• Its strength is price: hourly rental is often low for 32 GB of memory.
• Pick it to test, prototype or serve an internal use case, and move to a newer GPU for a high-traffic service.

Contents

1. What is the NVIDIA V100?
2. What are the V100's technical specifications?
3. What is the difference between a V100 PCIe and a V100 SXM2?
4. What can you do with a V100 in AI today?
5. Which AI software still runs on a V100?
6. What are the V100's limits compared with recent GPUs?
7. How much does it cost to rent a V100?
8. V100 or a newer GPU: how do you choose?
9. How do you get started with a rented V100?

What is the NVIDIA V100?

The NVIDIA V100, long called the Tesla V100, is a data center accelerator based on the Volta architecture, designed for AI and high-performance computing. Its original datasheet names it Tesla V100, while NVIDIA's current page simply says V100; it ships with Tensor Cores, units built for the matrix math of deep learning.

Key points from its ID card:

• a server card, with no video output, passively cooled by the chassis airflow;
• two form factors: a standard PCIe card and an SXM2 module linked by NVLink;
• two memory sizes: 16 GB or 32 GB of HBM2;
• compute capability 7.0, the value that decides compatibility with AI software.

It is no longer NVIDIA's top of the range, but its large, fast memory keeps it in GPU rental catalogs.

What are the V100's technical specifications?

The V100 packs 5,120 CUDA cores, 640 Tensor Cores, up to 32 GB of HBM2 and 900 GB/s of memory bandwidth, whatever the form factor. The figures below come from NVIDIA's Tesla V100 datasheet.

Three points really matter for AI:

• memory: it sets the size of the model you can load;
• bandwidth: in text generation, it often limits speed more than raw compute;
• Tensor Cores: they speed up mixed-precision math, the usual mode for inference.

What is the difference between a V100 PCIe and a V100 SXM2?

The V100 SXM2 is a module mounted on a server motherboard and linked to other GPUs through NVLink, while the V100 PCIe is a plug-in card that goes through the PCIe bus. According to NVIDIA's datasheet, the SXM2 offers 300 GB/s of interconnect versus 32 GB/s for PCIe, plus slightly more compute.

In practice:

• with a single GPU, the gap is small: same memory, same bandwidth, about 11 to 12% more compute on the SXM2 based on the NVIDIA datasheet;
• with several GPUs, NVLink makes the difference: a model split across two cards exchanges data much faster;
• when renting, you do not pick the physical form factor, but it helps to know which one is offered, especially for a two-GPU setup.

What can you do with a V100 in AI today?

A 32 GB V100 is suited to inference on mid-sized language models, image generation, light fine-tuning and scientific computing. It is no longer suited to training large models from scratch.

Use cases where it is still relevant:

• internal assistant or chatbot: a quantized 7B to 14B model runs comfortably in memory;
• prototyping an AI application before choosing the final infrastructure;
• image generation with Stable Diffusion or ComfyUI;
• audio transcription and batch document processing;
• light fine-tuning (LoRA) of a small model;
• scientific computing in double precision, its original strength.

To estimate the memory you need, a simple rule: in FP16, each parameter takes 2 bytes. A 7B model therefore needs about 14 GB for its weights alone, and its 4-bit quantized version roughly a quarter of that. Add headroom for context and concurrent requests. Our guide to hosting an AI application explains this sizing in detail.

Which AI software still runs on a V100?

Ollama and most tools built on llama.cpp or PyTorch still run on a V100, but vLLM and CUDA 13 now require compute capability 7.5 or higher, which the V100 does not reach. Rental listings often leave this out, yet it decides whether your project will run.

What this means for you:

• for an internal assistant, Ollama, possibly with Open WebUI, is the natural choice on a V100: our Ollama guide covers installation and choosing a model for your VRAM;
• for a high-throughput service that relied on vLLM, plan for a newer GPU or an API;
• pin your versions: CUDA 12.9 and driver 580, rather than chasing the latest release.

Pro tip: before renting, check the minimum compute capability in each tool's documentation. Two minutes of reading saves a day lost to an installation error.

What are the V100's limits compared with recent GPUs?

The V100's limits are shrinking software support, lower performance per watt than recent generations, and no support for the newest numeric formats. It is still capable, but it is no longer the target of optimizations.

• Compatibility: as shown above, CUDA 13 and vLLM no longer support it, according to the NVIDIA developer blog and the vLLM documentation.
• Optimizations: many recent libraries target architectures from compute capability 7.5 or 8.0 first.
• Power draw: 250 to 300 W per card according to the NVIDIA datasheet, for less compute than current cards.
• Memory: 32 GB at most, which means quantization or several GPUs for large models.

How much does it cost to rent a V100?

A V100 usually rents for between a few tens of cents and a few dollars per hour, depending on the provider. The GetDeploying comparison, updated November 6, 2026, reports a median on-demand price of $1.02 per GPU per hour, with wide gaps between small providers and the big clouds.

At Wayhost, the GPU rental page listed the following offers, as observed on November 6, 2026 (prices excluding VAT):

Other points noted on the same Wayhost page, on November 6, 2026:

• hourly or monthly billing, no commitment, prepaid;
• hosting in France, in Cergy, with root access via SSH key;
• an OpenAI-compatible AI API, billed per token, for those who do not want to manage a server.

V100 or a newer GPU: how do you choose?

Choose a V100 when memory and price matter more than speed, and a newer GPU when you need vLLM, high throughput or the latest optimizations. The grid below sums up the most common cases.

If you do not need a GPU at all, for example to host only the interface or the application that calls an API, a VPS is enough: our article What is a VPS? explains how it works, and our best VPS comparison helps you pick the right setup. For a steady workload and data that must stay on a dedicated machine, compare with a physical server; our article on the dedicated server explains the benefits, and our server hosting guide compares the different options.

How do you get started with a rented V100?

Start by checking that your software supports compute capability 7.0, then install a compatible driver and CUDA, and test a model with Ollama. The procedure takes only a few steps.

1. List your tools and check their compatibility in the software table above.
2. Connect over SSH with your key.
3. Check the GPU with the nvidia-smi command, which shows the model, memory and driver version.
4. Stay on a 580-branch driver and CUDA 12.9 if you compile code.
5. Install Ollama and load a quantized model sized for the memory, following our guide to running AI locally with Ollama.
6. Measure generation speed and memory use, then adjust the model size.
7. Secure access: never expose the AI engine's API directly to the internet.

Frequently asked questions

Is the V100 still good for AI?

Yes, for inference on mid-sized models, image generation and prototyping. Its 32 GB of memory and its bandwidth are still useful. It is, however, outdated for training large models and is no longer supported by vLLM.

What is the difference between the Tesla V100 and the V100?

They are the same GPU. NVIDIA's original datasheet says Tesla V100, while the maker's current page simply says V100. Both names refer to the same Volta chip.

How much memory does a V100 have?

A V100 comes with 16 GB or 32 GB of HBM2 memory. The 32 GB version is the more interesting one for AI, because it fits larger models. Bandwidth is identical on both versions.

Does Ollama work on a V100?

Yes, Ollama supports NVIDIA GPUs from compute capability 5.0, and the V100 is on its list. It is the simplest tool to run a language model on this GPU. All you need is a recent driver and a model of the right size.

Can you use vLLM on a V100?

Not with the current stable release, which requires compute capability 7.5 or higher. The V100 is at 7.0. For a high-throughput service, choose a newer GPU or an OpenAI-compatible API.

What model size fits on a 32 GB V100?

A 7B to 14B model fits easily in a quantized version, with headroom for context. Larger models need heavier quantization or two GPUs. Always measure the memory actually used with nvidia-smi.

Is it better to buy or rent a V100?

For a one-off project or a prototype, hourly rental avoids the upfront cost and hosting the card yourself. Buying only makes sense for continuous use, with a suitable server and enough cooling. Renting also lets you switch GPUs as your needs change.

Key takeaways

• The V100 is a Volta data center GPU: 640 Tensor Cores, 16 or 32 GB of HBM2, 900 GB/s.
• The SXM2 stands out with NVLink, mostly useful with several GPUs.
• Ollama runs on a V100, stable vLLM and CUDA 13 do not: check your tools before renting.
• It suits inference, images and prototyping, not training large models.
• Its price-to-memory ratio makes it a good entry point into self-hosted AI.

To test a model on an NVIDIA V100 32 GB hosted in France, discover Wayhost GPU rental.

Article written by Mouhamed BANKOLE, CEO at ITNET Technologies.