Apple Silicon vs NVIDIA for running LLMs at home: memory against bandwidth

The trade in one paragraph

Generating text is limited by memory: how much there is, which decides what fits, and how fast it is, which decides how many tokens per second are possible. An Apple Silicon Mac shares one large pool of memory between CPU and GPU, up to 512 GB on the M5 Ultra and M3 Ultra. An NVIDIA GeForce card has at most 32 GB (RTX 5090), but its memory is faster: 1,792 GB/s, against 1,200 GB/s on the fastest Mac. So for a model that fits on the card, the card has the higher ceiling, and for a model that does not, the Mac can run it at all.

Every speed below is the ceiling from the Can I run it? calculator: memory bandwidth divided by the bytes read per token, for one conversation at an 8,192-token context. Real software stays below it on both platforms. Nothing here is a benchmark.

Every Apple chip at its largest memory

Each Mac is shown at the largest memory Apple offers for that chip, with the GPU's default share of it (see the limit section below). Ceilings in tokens per second:

Bandwidth within a chip family depends on the version: the M4 Max with a 32-core GPU has 410 GB/s and only 36 GB, while the 40-core version has 546 GB/s and up to 128 GB. The Pro chips have far less bandwidth than the Max chips of the same generation.

The same models on NVIDIA and the other unified-memory machines

For the graphics cards, "offload" assumes 32 GB of DDR5-5600 system RAM. A desktop with more RAM runs the bigger models too, slowly. With 128 GB of DDR5-5600 next to an RTX 4090, the ceilings are 3.9 tokens per second for Llama 3.3 70B (37 of its 80 layers in RAM), 43.6 for gpt-oss-120b, which as a mixture-of-experts model reads only a few experts per token, and 7 for Qwen3 235B-A22B. llama.cpp's option to keep only the expert weights in RAM (--n-cpu-moe) can do better than the whole-layer split the calculator assumes.

What the tables say

• For models that fit on the card, the RTX 5090 has the highest ceiling of any machine here. For Llama 3.1 8B it is 316 tokens per second. The M5 Ultra reaches 212, more than the RTX 4090's 178; an M4 Max with a 40-core GPU reaches 96.
• At 70B, the Mac wins unless you buy a workstation card. Llama 3.3 70B at Q4_K_M does not fit on any GeForce card, and offloading leaves an RTX 4090 at 3.9 tokens per second. A 128 GB M4 Max holds it all, with a ceiling of 12.1, and an M5 Max 13.6. The 96 GB RTX PRO 6000 holds it at 39.6, or two 24 GB GeForce cards (see the GPU tiers guide).
• Above 128 GB, few machines hold the model at all. Qwen3 235B-A22B at Q4_K_M needs 136.42 GiB at 8k. Only these hold it entirely in GPU-usable memory: Apple M2 Ultra, Apple M3 Ultra, Apple M5 Ultra, AMD Ryzen AI Max+ PRO 495 (Radeon 8065S).
• The DGX Spark and Ryzen AI Max sit between. Both have 128 GB, like an M4 Max or M5 Max, at about half the bandwidth (273 and 256 GB/s), so their ceilings are half as high or less on the same models.

Prompt processing is a different question

Reading a long prompt before the first token appears (prefill) is limited by the GPU's compute, not by memory bandwidth, so the ceilings above say nothing about it. Long documents, big codebases and retrieval-augmented prompts are where it matters. This site has not measured prefill on Macs or NVIDIA cards yet and makes no claim about it; measured benchmarks are planned, with scripts and raw results.

Software

• NVIDIA cards use CUDA, which nearly every inference engine supports: llama.cpp, Ollama, LM Studio, vLLM, SGLang and TensorRT-LLM among them. FP8 and NVFP4 checkpoints need recent NVIDIA tensor cores to run at full speed.
• Macs use Metal through llama.cpp (and the tools built on it, such as Ollama and LM Studio) and through Apple's own MLX framework. vLLM-style serving engines target CUDA first. For one person chatting with a model, llama.cpp and MLX cover most needs.

Power

Apple publishes a maximum power figure for each desktop Mac, for the whole computer at the wall while running a compute-intensive test: 140 W for the Mac mini (2024) with M4 Pro, 200 W for the Mac Studio (M5 Max, 2026) with M5 Max (18-core CPU, 32-core GPU), 270 W for the Mac Studio (2025) with M3 Ultra, 385 W for the Mac Studio (M5 Ultra, 2026) with M5 Ultra. NVIDIA's Total Graphics Power is for the card alone: 450 W for the RTX 4090 and 575 W for the RTX 5090, before the rest of the PC. Both are maximums, not what a machine draws while generating text. The electricity calculator turns either into a monthly cost; at its defaults, a Mac Studio with M3 Ultra comes to $3.01 a month and a PC with an RTX 4090 to $9.47, both upper bounds.

The GPU memory limit on Macs

macOS does not let the GPU use all of unified memory. Metal reports the limit as recommendedMaxWorkingSetSize, and llama.cpp prints it when it starts. Apple documents no formula, but its 2021 tech talk on Metal compute gives two examples, 21 GB of 32 GB and 48 GB of 64 GB, and the calculator follows them: two thirds of memory up to 32 GB, three quarters above. A 128 GB Mac therefore has about 96 GB for a model by default.

The limit can be raised with sudo sysctl iogpu.wired_limit_mb=<megabytes>, which Apple's MLX project documents: set it above the model's size and below the machine's memory. Community reports say the setting resets on reboot, and an MLX maintainer linked a kernel panic to a limit set too high, so leave macOS several gigabytes. The calculator lets you enter your own share and marks models that fit only after raising it.

Which to choose

• Models up to about 30B parameters, at the highest speed: the RTX 5090, with a ceiling of 82 tokens per second for Qwen3 32B at Q4_K_M. A 24 GB card such as the RTX 4090 holds the same model (21.67 GiB at 8k) with a ceiling of 46, against 38 on an M3 Ultra and 55 on an M5 Ultra.
• 70B dense models or 100B+ mixture-of-experts models on one machine: a Max or Ultra Mac with 96 to 128 GB or more, or a 128 GB DGX Spark or Ryzen AI Max PC at lower bandwidth.
• 200B-class models: an M2, M3 or M5 Ultra with 192 GB or more, or a desktop with a large GPU plus lots of system RAM, which runs them at a fraction of the speed.
• Training or fine-tuning: NVIDIA, for the software support. The fine-tuning cost estimator sizes the job and prices it on rented GPUs, which avoids buying hardware for a one-off run.

Also available as Markdown.