lllm3090.gguf#

Read what a GGUF says about itself, without loading it.

A GGUF carries its metadata and its full tensor list in a header, before any weights. That is enough to answer questions the catalogue cannot answer from a file size – currently one question: does this checkpoint carry a multi-token prediction head?

Everything here reads the header and stops. An 18 GB checkpoint is inspected in tens of milliseconds, because the tokenizer vocabulary is walked as offsets rather than materialised as strings.

Why this is derived rather than declared: the catalogue can say a repo ships MTP, but only the file on disk decides whether the engine can use it. Two builds of the same model differ, quantisers strip the head, and a partially downloaded file has no tensors at all. A flag passed to llama.cpp on the strength of a YAML field would produce an engine that fails at load; read from the file it cannot be wrong.

The header is not free, and the panel asks every two seconds. Reading all nine catalogued checkpoints takes about 0.65 s, which is why facts() memoises on the file’s identity rather than its name – see its docstring for what that costs and what invalidates it.

lllm3090.gguf.U8 = 0#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.I8 = 1#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.U16 = 2#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.I16 = 3#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.U32 = 4#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.I32 = 5#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.F32 = 6#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.BOOL = 7#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.STRING = 8#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.ARRAY = 9#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.U64 = 10#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.I64 = 11#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.F64 = 12#: GGUF metadata value types, by their on-disk tag.

lllm3090.gguf.MTP_MARKERS = ('nextn', 'mtp')#

Substrings that identify a multi-token prediction head in a tensor name.

Qwen calls it nextn and puts it in one extra block past the last real layer – blk.64.nextn.* on the 27B, blk.40.nextn.* on the 35B-A3B. DeepSeek uses the same word. mtp is accepted for anything that spells it the other way.

exception lllm3090.gguf.Malformed[source]#

Bases: Exception

This file is not a GGUF, or its header does not parse.

lllm3090.gguf.header(path: Path | str) → tuple[dict[str, Any], list[str]][source]#: A checkpoint’s metadata and tensor names, read from its header alone.

class lllm3090.gguf.Facts(mtp: bool, full_attention_layers: int | None)[source]#

Bases: NamedTuple

What the file itself says, as opposed to what models.yaml declares.

Both fields are exactly what has_mtp() and full_attention_layers() return, and both are derived from one read. They used to be two, which meant a caller wanting both parsed an 18 GB checkpoint’s header twice.

Create new instance of Facts(mtp, full_attention_layers)

mtp: bool#: A usable multi-token prediction head is present.

full_attention_layers: int | None#: How many layers keep a KV cache, or None where the question does not apply to this architecture.

lllm3090.gguf.facts(path: Path | str) → Facts[source]#

Everything derived from this checkpoint’s header, read at most once.

A failed read is not cached: the overwhelmingly likely reason is a file that is still arriving, and remembering “not a GGUF” about a checkpoint that is about to become one would need a second invalidation rule to undo. Re-reading a header that fails to parse is cheap – it fails in the first few bytes.

lllm3090.gguf.full_attention_layers(path: Path | str) → int | None[source]#

How many of this checkpoint’s layers keep a KV cache, or None.

The catalogue’s kv_kib_per_token is kv_heads x (key_length + value_length) x 2 bytes x this number, and on the hybrid models that carry an MTP head it reproduces the hand-entered figures exactly. So this is what lets a declared field be checked rather than trusted.

Two things make it less obvious than counting blocks:

full_attention_interval – these are hybrids, and only one layer in four keeps a cache. The rest are SSM, whose state is per-sequence rather than per-token and therefore costs nothing as context grows.
block_count includes the MTP head, which is why it reads 65 for a 64-layer model and 41 for a 40-layer one. Subtracting it is the whole reason the head can be priced as “one more of these”.

None for an architecture that is not shaped this way – Gemma-4 with its sliding-window pattern and per-layer head counts, gpt-oss with no interval at all. That is not a failure: none of them carries an MTP head, so nothing needs the number.

lllm3090.gguf.has_mtp(path: Path | str) → bool[source]#

Whether this checkpoint carries a usable multi-token prediction head.

Requires the tensors, not merely the metadata key. A conversion can announce nextn_predict_layers and still ship no head, and llama.cpp refuses to start with --spec-type draft-mtp against a checkpoint that has none – so the tensors are the only honest test.

Any failure to read the file is a False: this decides whether to add a flag to a working command line, and the cost of being wrong in that direction is an engine that will not start.