lllm3090.sweep#
Survey what is published, and price it against the card in this machine.
The curated list in data/models.yaml is small and hand-checked, and the
field moves in weeks. This is the tool that widens it: it asks HuggingFace what
GGUF checkpoints exist, derives each one’s KV cost from its own config.json,
and runs the result through the same lllm3090.catalog.fit() and
lllm3090.catalog.plan() the panel uses – so a candidate is judged by
exactly the arithmetic that will judge it once it is in the catalogue.
Nothing here downloads weights. A survey costs one config file per candidate, which is how a 20 GB mistake gets avoided for the price of a few kilobytes.
It never produces a speed. Tokens per second is a measurement on one card,
and the roofline that would derive one is calibrated between 20% and 35% of roof
for a resident MoE – wider than the error bar that would imply. Entries come
out of here with verified: false and no expected_tok_s; lllm3090
bench is what fills that in.
- lllm3090.sweep.QUANT_PREFERENCE = ('UD-IQ4_XS', 'UD-Q4_K_XL', 'UD-Q4_K_S', 'UD-Q4_K_M', 'MXFP4', 'IQ4_XS', 'Q4_K_M', 'Q4_K_S', 'Q4_0')#
Quantisations worth considering, best first.
The catalogue lives at roughly four bits per weight, which is what fits this class of card with context left over. Unsloth’s
UD-dynamic quants come first because two of them are already in the list and have been measured;MXFP4is here for gpt-oss, which ships in nothing else.
- lllm3090.sweep.F16 = 2#
Bytes per element in an f16 KV cache. The catalogue records the f16 figure and halves it at plan time, because the engine is run with q8 caches.
- class lllm3090.sweep.Candidate(repo: str, file: str, name: str, size_gb: float, kv_kib_per_token: float, max_ctx: int, kind: str, params: str, downloads: int = 0, mmproj: str | None = None, mmproj_gb: float = 0.0)[source]#
Bases:
objectA published checkpoint, priced but not downloaded.
- as_model() Model[source]#
The catalogue entry this would become, for pricing.
Deliberately built through the real
lllm3090.catalog.Modelrather than a parallel structure: if a candidate cannot be expressed as a catalogue entry, it cannot be added to the catalogue, and the sweep should find that out here rather than at paste time.
- class lllm3090.sweep.Priced(candidate: Candidate, fit: Fit, plan: Plan, status: str, note: str)[source]#
Bases:
objectA candidate with the verdict this card gives it.
- exception lllm3090.sweep.Unsupported[source]#
Bases:
ExceptionThis config cannot be priced, and guessing would be worse than skipping.
Raised rather than returning a fallback figure on purpose. A wrong KV cost does not fail loudly: it produces a plan the card cannot honour, an engine that loads and reports itself healthy, and a failure at the first request.
- lllm3090.sweep.kv_kib_per_token(cfg: dict[str, Any]) float[source]#
KV cache cost per token at f16, in KiB, from a model’s own config.
full_attention_layers x 2 (K, V) x kv_heads x head_dim x 2 bytes.The subtlety is which heads and which head dimension. Gemma-4 gives its full-attention layers a geometry of their own –
num_global_key_value_headsandglobal_head_dim– separate from the sliding layers’ figures, and reading the sliding layers’ numbers instead is wrong by 2x on the 26B and 4x on the 12B. Where a config draws that distinction, the global figures are the ones that cost per token, because the global layers are the only ones that do.Reproduces every figure in the shipped catalogue exactly; see
tests/test_sweep.py, which is what keeps that true.
- lllm3090.sweep.max_ctx(cfg: dict[str, Any]) int[source]#
The model’s own RoPE ceiling, past which output degrades rather than costs.
- lllm3090.sweep.kind(cfg: dict[str, Any]) str[source]#
moeordense, which is what decides whether size predicts speed.
- lllm3090.sweep.params(cfg: dict[str, Any]) str[source]#
The one-line architecture summary a catalogue entry carries.
- lllm3090.sweep.price(candidate: Candidate, profile: Profile | None = None, desktop: bool = True) Priced[source]#
Run one candidate through the catalogue’s own arithmetic.
- lllm3090.sweep.TODO_NOTE = 'TODO: why would someone pick this over the entry above it?'#
What a swept entry cannot know, and must not invent.
notesis prose about why someone would pick this model over its neighbours, and no arithmetic produces it. Emitting a plausible-sounding sentence would be worse than emitting nothing, because it would read exactly like the hand-written notes around it.
- lllm3090.sweep.to_yaml(results: list[Priced], profile: Profile | None = None) str[source]#
Catalogue entries for these candidates, ready to paste into models.yaml.
Every field that is arithmetic is filled in. Every field that is judgement – notes, tags – is left as a marker, and the speed is left absent entirely. A block from here parses back into
lllm3090.catalog.Modelunchanged, whichtests/test_sweep.pyasserts, so a paste that loads is a paste that the panel can already price.profilemust be the one the results were priced against. The plan in each note is a claim about a specific card, so naming a different one – the detected card, when--gpupriced for another – would attach a correct figure to the wrong hardware, which is worse than omitting it.
- lllm3090.sweep.survey(limit: int = 100, profile: Profile | None = None, desktop: bool = True, known: set[str] | None = None) tuple[list[Priced], list[Priced], list[tuple[str, str]]][source]#
Price the most-downloaded GGUF repositories against this card.
Returns what is worth adding, what was priced and rejected, and what could not be priced at all with the reason. The rejections are returned rather than dropped because they are the part worth writing down: two of the models most often recommended for a 24 GB card fit it and leave a window an agent cannot use, and a survey that printed only its successes would lose that both times it ran.