LLM inference on NVIDIA DGX Spark and DGX Station GB300 Benchmark Series

How fast a local LLM runs depends on the hardware, the model and the number of requests it processes at the same time. To measure this, I benchmarked 20 current open-weight models on an NVIDIA DGX Station GB300 and 14 of them on a DGX Spark. This series presents the results, one machine and one topic per article.

What I measured

Every model ran in its own vLLM container and was benchmarked with NVIDIA’s AIPerf client. All requests had the same size: a 1,024-token prompt (slightly more once wrapped in each model’s chat template) and exactly 512 output tokens, measured for five minutes at each load level. The load is the number of concurrent requests, from 1 up to 32 on the Spark and up to 256 on the Station. A concurrent request here means a slot that sends its next request as soon as the previous answer is complete. AIPerf counts only requests that complete within the measurement window; tokens of requests still running when it closes are left out. vLLM’s own token counter, which AIPerf records alongside, counts every token the server generated in that time. Where single requests take about a minute or longer, AIPerf’s total can therefore be lower than vLLM’s; how much lower depends on how many requests are still running when the window closes. Where the difference matters, the articles give both.

In total, that’s 394,956 measured requests, 147 million answer tokens (excluding reasoning; the 512 output tokens per request include it) and about 32 hours of measurement. The raw data, every configuration file and 2,071 charts are public:

• aiperf_dgx_spark_260917: DGX Spark, 14 models
• aiperf_dgx_station_260918: DGX Station, 20 models
• aiperf_dgx_station_260919: DGX Station, second run of the five largest models
• aiperf_tool: the harness I built to run the tests

The benchmark measures speed: tokens per second, in total and per request, and how long it takes until an answer starts. It doesn’t measure answer quality or how well a model suits a particular task, and it reflects the software available in September 2026, without speculative decoding.

The articles

DGX Spark Benchmarking

The first article in the series covers how fast popular LLMs run on the DGX Spark. All 14 models I ran on it appear in one table, from small Gemma models to a 120B Nemotron, with a closer look at how the number of active parameters and the precision of the weights relate to speed on this machine.

The second article covers the five largest models I ran on the DGX Station, with weights between 164 and 352 GB. One fits entirely in the GPU’s memory, the others had part of their weights moved to the CPU’s memory. The article shows how each behaves from 1 to 128 concurrent requests.

The third article covers the 15 small and mid-size models on the DGX Station and how far they scale, up to 256 concurrent requests, including where throughput levels off.

The fourth article takes eight versions of three Qwen models (Qwen3.8 and Qwen3.6), in several precisions (BF16, FP8, NVFP4) from different publishers, and compares how they perform on the DGX Station.

The fifth article compares two runs of the same five large models on the DGX Station with different vLLM settings, and what changed in the results.

What this series doesn’t do

The articles don’t compare the two machines with each other, and they don’t recommend models. They describe what I measured, with enough context to read the numbers. Whether a result fits a particular application is a question only that application can answer. If you are evaluating one of these machines for a specific model or application, we would be glad to hear about your requirements. We can benchmark your model with your own prompts and answer lengths and help you assess what the results mean for your use case.

All measurements in the series come from the published data. If you find an error or an omission, please contact us.