Apple Metal, Explained for People Who Build Mac Apps

Most Mac developers use Metal every day without writing a single line of it. Every SwiftUI animation, every Core Image filter, every Core ML model that lands on the GPU runs through Metal somewhere underneath. Lately I’ve been pulling more of my own work down to that layer: video filters, on-device models for clients who don’t want their data leaving the building, and a few experiments for live visuals. So here’s the version of Metal I wish someone had given me. What it is, what it changes about how you design a Mac app, what the public numbers actually say, and a benchmark you can run on your own machine instead of taking anyone’s word for it.

What Metal actually is

Metal is Apple’s low-level API for the GPU. It covers both graphics (drawing triangles, pixels and frames) and general-purpose compute (any maths you can split across thousands of threads). It arrived on iOS in 2014 and on the Mac in 2015, and replaced OpenGL and OpenCL, which Apple deprecated in 2018. It’s “low overhead” in a specific sense. You build pipeline states and command buffers up front, then hand the GPU batches of work, instead of making thousands of small driver calls per frame.

It’s really a small family of pieces:

• The Metal API. Devices, command queues, command buffers, encoders, buffers and textures. You describe work, commit it, and the GPU runs it asynchronously.
• Metal Shading Language (MSL). A C++-based language for the code that runs on the GPU: vertex and fragment shaders for graphics, and kernel functions for compute. If you’ve written GLSL or CUDA, it’ll feel familiar within an afternoon.
• Metal Performance Shaders (MPS). Apple’s library of hand-tuned GPU kernels for image processing (blurs, convolutions, histograms), linear algebra (matrix multiply) and neural-network building blocks. They’re tuned per GPU family, so for the common stuff you usually shouldn’t write your own.
• MPSGraph. A compute-graph framework on top of MPS. You describe a whole model or calculation as a graph and it compiles and optimises it for the GPU. PyTorch’s mps backend on the Mac is built on it, and Apple positions it as the way to bring Core ML models into Metal workloads.
• Metal 3 and Metal 4. Metal 3 (2022) brought MetalFX upscaling, mesh shaders, fast resource loading and offline shader compilation. Metal 4 (2025) reworks how you encode commands and compile shaders so both are cheaper, and makes machine learning a first-class citizen. You can encode inference networks at the command level or run them directly inside your shaders. MetalFX also gained frame interpolation and denoising.

Unified memory changes the maths

The part people underrate is Apple silicon’s unified memory. On a typical PC, the CPU has its RAM and the GPU has its own VRAM, so every texture or tensor you want on the GPU gets copied across a bus first. On M-series chips, the CPU, GPU and Neural Engine all share one pool of memory. A buffer created with .storageModeShared is the same bytes for both sides, with no upload step.

Two practical consequences:

1. Small, frequent jobs become worth sending to the GPU. The copy tax that used to kill “just blur this frame on the GPU” mostly disappears.
2. Big models fit. A Mac with 64 GB or more of unified memory can hold a model that would need a very expensive discrete card elsewhere. That’s why local LLMs on Macs became a thing at all.

What Metal does for Mac app design

This is where it stops being trivia and starts being design.

Smooth 120Hz UI and animation

MacBook Pros with ProMotion refresh at up to 120Hz, which gives you about 8.3 ms per frame. Core Animation and SwiftUI already render through Metal, so the job is mostly to stay out of their way. Keep heavy work off the main thread and avoid forcing the system to redraw layers on the CPU every frame. When you do need a custom look, SwiftUI’s shader modifiers (colorEffect, distortionEffect, layerEffect) let you drop a small MSL function onto a view. It runs on the GPU, not in your view’s body. That’s how you get a ripple, a glow or a live gradient that holds 120 fps instead of stuttering at 30.

Custom rendering

Charts with a million points, node editors, audio visualisers, maps and 3D previews. Anything that draws more than a few thousand things per frame belongs in an MTKView with your own pipeline, not in a stack of views. You draw exactly what changed, and the GPU does the rest.

Image and video processing

Core Image runs on Metal, and you can write your own Core Image kernels in MSL. For video, the pattern is to decode with the hardware media engine, wrap each frame as a Metal texture without copying it (CVMetalTextureCache), run your filters as compute kernels, then display or encode. On unified memory, that whole chain can stay on the GPU. For a 1080p or 4K pipeline, the difference between that and pushing frames through the CPU is the difference between “real time” and “export and wait”.

On-device AI

This is the big one for me. Clients in law, health and finance keep asking for AI that never leaves their machines, and on a Mac that’s realistic:

• Core ML runs a converted model and schedules it across the CPU, GPU and Neural Engine. You can force cpuOnly, cpuAndGPU or all to compare them.
• MPS and MPSGraph sit underneath when the GPU is used, and you can call them directly for custom layers.
• MLX, Apple’s open-source array framework, was designed around unified memory. It’s what I reach for when running small LLMs locally. mlx-lm makes it one command.
• llama.cpp has a Metal backend too, which is what a lot of local-model apps use under the hood.

Battery and thermals

Massively parallel work, like pixels, matrices and model layers, generally costs less energy on the GPU than on the CPU, because the GPU is built for exactly that shape of work and finishes sooner, then goes idle. Fans stay quieter and the battery lasts longer. The catch is the other direction. Waking the GPU for tiny jobs, or bouncing data back and forth every few microseconds, can cost more than it saves. Batch your work, keep it on the GPU, and measure with Instruments (the Metal System Trace and the Power Profiler) rather than guessing.

What the public benchmarks say

I only want to quote numbers that someone published with their method, so here are the ones I trust, with links. These are other people’s measurements, not ours.

What I take from that:

• On the same chip, the GPU wins big for parallel, heavy work. Convolutions, sorting and the big matrix multiplies inside transformers are the clearest cases.
• It doesn’t win everything. Tiny elementwise ops and gathers were slower on the GPU in the MLX paper, because launch overhead dominates. That’s the “don’t send small jobs” rule from the battery section, showing up in the data.
• The software stack matters as much as the silicon. On the same M2 Ultra, different runtimes ranged from single digits to around 230 tokens per second.
• Geekbench is a useful ladder between chips. It doesn’t tell you anything about your workload.

Our real-time benchmark

So I wrote one we can run ourselves. MetalBench is a small Swift command-line tool. It runs the same work on the CPU and through Metal and writes the results to results.json:

1. 1080p Gaussian blur, every frame. A 1920×1080 RGBA image, radius 8, two passes. CPU on one thread and on all cores, compared with our own MSL compute kernel and with MPSImageGaussianBlur. It reports ms per frame and fps.
2. Matrix multiply. 2048×2048 float32, Accelerate’s BLAS on the CPU (a strong baseline, not a strawman) against MPSMatrixMultiplication. It reports GFLOPS.
3. AI inference. A small transformer-style feed-forward model, run through Accelerate on the CPU and MPSGraph on the GPU. It reports inferences and tokens per second. You can also pass any Core ML model to compare cpuOnly, cpuAndGPU and all (Neural Engine), and a companion script runs a small LLM in MLX on mx.cpu against mx.gpu.

Machine: pending.

To run it yourself on any Apple silicon Mac:

cd metal-bench
swift build -c release
.build/release/MetalBench --out results.json
# optional: a small LLM on MLX, CPU vs GPU
pip install mlx mlx-lm && python3 scripts/mlx_llm_bench.py --out mlx_results.json

What it looks like (simulated)

How I’d use this in a real app

• Default to Apple’s frameworks. SwiftUI, Core Animation, Core Image and Core ML already sit on Metal. Most of the speed comes from not fighting them.
• Drop to MPS or MPSGraph when you have a clear hot path, like a filter chain, a custom layer or a big matrix job.
• Write MSL when nothing off the shelf fits, such as custom visual effects, simulations or unusual image maths.
• Keep data on the GPU for whole pipelines, batch small jobs, and trust Instruments over intuition.
• Measure on the machines your users actually have. An M1 Air and an M4 Max are different planets.

Metal isn’t magic. It’s a way to put the right work on the right silicon, and on Apple silicon the “right silicon” shares memory, which makes the GPU much easier to use well. When the real numbers from our runs land, I’ll publish them here, warts and all.