Open-weight model finder
Which open-weight LLMs can actually run on this machine?
Enter your GPU or Mac and get a ranked answer: which models fit, at which quantization, how fast they will decode, how they score on general and coding evaluations, and whether the licence lets you ship what they produce.
- 119
- models tracked
- 0
- cleared for commercial use
- 40
- publishers
- 2026-08-22
- data last refreshed
How the ranking is computed
Every number on this page is derived from published model shape and published hardware specifications, so you can re-derive any of them by hand. Nothing is fitted to private benchmarks and nothing is a vendor claim repeated verbatim.
The memory and speed model
Weights
bytes = parameters x bits_per_weight / 8Bits per weight is the effective figure including the per-block scales k-quants carry, so Q4_K_M is costed at 4.85 bits rather than 4.
KV cache
bytes = 2 x layers x kv_heads x head_dim x context x bytes_per_elementRead from each model's own config.json, so grouped-query and latent-attention models are costed correctly instead of by a generic per-billion rule. This term is why a model with a large parameter count can fit while a smaller one with many KV heads does not.
Runtime overhead
bytes = engine_floor + engine_share x weightsAllocator, activation and graph-capture buffers. The floor is a property of the engine - llama.cpp holds about 0.6 GB, Ollama more for its server and runner processes, vLLM around 2 GB before it preallocates its KV pool - with a surcharge on CUDA for the driver context.
Usable memory
budget = capacity x platform_shareDiscrete GPUs give about 92% of VRAM to the runtime. Apple Silicon caps GPU-wired memory near 75% of unified memory. CPU-only inference leaves several gigabytes to the operating system.
Decode throughput
tokens/s = min(bandwidth / (active_weight_bytes + 1/2 x kv_cache_bytes) x efficiency, ceiling)Local decoding is memory-bandwidth bound. Mixture-of-experts models read only their active experts, which is why a 120B-A12B model decodes near the speed of a 12B dense model while still needing the full 120B resident. Efficiency is the share of peak bandwidth the engine's decode loop reaches, with an extra penalty for the scattered reads MoE routing causes. The ceiling is the engine's single-stream limit: below roughly a gigabyte read per token, kernel launch and sampling bind instead of memory.
The ranking
The default order leads with measured capability, discounts it by the quality lost to quantization, and then applies throughput as a modifier rather than a driver. Past roughly thirty tokens per second more throughput stops changing how a model feels to use, so speed only moves the ranking meaningfully when a model is slow enough to be impractical. A model that only fits by streaming experts from system RAM is ranked below one that sits entirely in accelerator memory.
The general score is the Artificial Analysis Intelligence Index as published. The coding score is the mean of that model's available coding evaluations - SciCode, LiveCodeBench-Repo, Terminal-Bench Hard and LiveCodeBench - and every component is listed on each model's detail panel so you can see what the average is made of rather than trusting a single composite.
Full-precision weights are excluded from the automatic quantization search. Eight-bit is indistinguishable from native precision in practice, so serving BF16 locally costs double the memory and half the throughput for no measurable gain.
The inference engine
What the engine changes. Three things, all of which move the answer. Which quantization formats can be loaded at all - llama.cpp reads GGUF k-quants, vLLM reads AWQ, GPTQ, FP8 and INT8, ExLlamaV2 reads EXL2, MLX its own group-quantized format - so a Q5_K_M fit simply does not exist under vLLM. How much of peak bandwidth the decode loop sustains, which is where fused attention kernels earn their keep. And how much memory the process holds before a single weight is loaded.
Offload is an engine capability. A large MoE that overflows VRAM can still run if routed experts stay in system RAM and stream per token. llama.cpp and Ollama do this; vLLM and ExLlamaV2 expect the model resident. The same machine and the same model therefore reads "needs offload" under one runtime and "does not fit" under another - which is the honest answer, not an inconsistency. Throughput for an offloaded fit is the byte-weighted harmonic mean of accelerator bandwidth and the roughly 55 GB/s host path, so a model that spills a tenth of its weights stays near GPU speed while one that spills most of them runs at host speed.
These are ratios, not measurements. The efficiency factors are anchored on the 0.55 llmfit documents for llama.cpp and set relative to it from each engine's published design - fused kernels and paged attention above it, a wrapper process fractionally below. They rank runtimes against each other on one machine. They are not a benchmark of any specific build, and a tuned deployment will beat them.
What the browser cannot tell us
No web API reports VRAM, installed RAM or memory bandwidth. Automatic detection reads the WebGPU adapter vendor and architecture, the CPU core count and a heavily clamped memory hint, then proposes the closest preset and states its confidence. Confirm or correct the fields - a guess presented as a measurement would be worse than no detection at all.
Sources
Canonical repository, licence tag, gated-access flag, download counts, and the architecture parameters in each model's config.json - layer count, KV head count and head dimension - which the KV-cache term is computed from.
Intelligence Index, Agentic Index and the individual coding evaluations, plus parameter counts, active-parameter counts, context windows, licence names and the commercial-use classification.
The hardware-fit approach this page follows: score models on memory fit, throughput, quality and context, and estimate decode speed from memory bandwidth divided by bytes read per token.
The catalogue was last read from these sources on 2026-08-22. Licence summaries are orientation, not legal advice - every model links to its licence text, and the terms of custom vendor licences change without notice.
Frequently asked questions
How much VRAM do I need to run an open-weight LLM locally?
Roughly the parameter count times the bits per weight divided by eight, plus the KV cache and about a gigabyte of runtime overhead. A 27B model at Q4_K_M needs about 16 GiB of weights; at 32K context its KV cache can add anywhere from 1 to 30 GiB depending on how many KV heads the architecture uses. That second term is why parameter count alone is a poor predictor of whether a model fits.
Which quantization should I use?
Q4_K_M is the usual sweet spot: it retains most of the model's capability at under a third of the memory of native precision. Q6_K and Q8_0 are effectively lossless if you have the headroom. Below Q4 the degradation becomes visible, and a large model at 2-bit will often lose to a smaller model at 4-bit - which is why this tool lets you set a floor on quantization quality rather than always reaching for the biggest model that technically fits.
Which local LLM model is best for coding?
It depends on what fits your hardware, not a single universal answer. This page's Coding specialisation sorts the catalogue by the mean of each model's coding evaluations - SciCode, LiveCodeBench-Repo, Terminal-Bench Hard and LiveCodeBench - after applying your VRAM, quantization and context filters, so the top result is a model you can actually run, not just the highest scorer in the catalogue overall.
Can I run a coding LLM locally?
Yes, and coding is one of the more forgiving workloads for local models: agentic coding benefits from long context and low latency more than from having the single best score on a leaderboard. A quantized 30B-class model on a 24GB GPU handles most day-to-day coding tasks well under llama.cpp or vLLM; smaller quantized 7-14B models still cover autocomplete and single-file edits on 8-16GB of VRAM. Select the Coding specialisation above to filter the catalogue to what fits your machine.
Can I use open-weight models commercially?
Most, but not all. Apache 2.0 and MIT models can be deployed commercially with only attribution obligations. Community licences such as Meta's add scale thresholds and acceptable-use policies. Vendor licences from NVIDIA, Liquid AI, LG AI Research and others range from freely commercial to research-only. This page shows the licence family, whether commercial use is permitted, the obligations it creates and the restrictions it imposes, with a link to the licence text for every model.
Does a mixture-of-experts model need less memory?
No. An MoE needs all of its parameters resident in memory, because any token may route to any expert. What it saves is bandwidth: only the active experts are read per token, so a 120B model with 12B active decodes at roughly the speed of a 12B dense model. The memory requirement follows total parameters; the speed follows active parameters.
How accurate are the tokens-per-second estimates?
They are estimates from a bandwidth model, not measurements. Local decoding is memory-bandwidth bound, so bandwidth divided by bytes read per token, times a fixed efficiency factor, tracks real throughput within a reasonable margin across llama.cpp, vLLM and MLX. Treat them as a way to compare models on the same machine rather than as a benchmark result - actual throughput depends on your runtime, batch size, prompt length and thermal headroom.
Why run a model locally rather than call an API?
Because the data cannot leave. Teams working on classified, export-controlled or contractually confidential material often cannot send requirements, test procedures or source code to a hosted endpoint at all. Local open-weight models make the analysis possible without the data ever crossing an organisational boundary, and they make the deployment reproducible and auditable - the same weights produce the same behaviour, indefinitely.
How current is this catalogue?
It is regenerated daily from Hugging Face and Artificial Analysis rather than hand-maintained, so it reflects the state of those sources on the date shown next to the results. Open-weight releases move quickly; a model that tops the ranking for your hardware today may be displaced within weeks.
Run verification and validation on your own infrastructure
KomAInu is an on-premise AI solution for DO-178C, ISO 26262 and IEC 62304 verification. It is model-agnostic and runs on your own hardware with the open-weight models you choose. Requirements, test procedures and source code never leave your network.
Filter the catalogue by publisher jurisdiction when procurement, export-control or data-residency rules constrain which weights you can host. Europe, France and other regions stay filters and an optional sort, never a hidden bias in Recommended.