# 8. A second library, under the same ids

Date: 2026-08-08

## Status

Accepted

## Context

Project Gutenberg Australia holds books Project Gutenberg does not. The two
clear their books against different countries' copyright law — Australia's is
life+70 from the author's death, America's is 95 years from publication — so
the collections barely overlap. Matching all 4,384 Australian titles against
the 77,820 in the Gutenberg dump on title and author finds around 650 in both.
The rest is roughly four fifths of the smaller library, and it is not marginal:
it holds every Orwell novel, *Tender is the Night*, late Woolf and Chesterton,
and several thousand pulp titles nobody else has transcribed.

somnia identifies a book by one integer. That `gid` is the primary key of
`books`, the foreign key of `chapters` and `chunks`, the unique key of a live
queue row, a path segment in four HTTP routes, the thing a saved position is
saved against, and the number the page holds in a variable all night. Adding a
second library means either that integer stops being sufficient, or it does
not.

Three things about the Australian library are not true of the other one. There
is no CSV dump and no API — the catalog is one plain text file meant for a
person to read. A book's address cannot be computed from its id, the way
Gutenberg's always can. And it is finished: no new books since 31 December
2024.

## Decision

**One `gid` space, offset.** Australian ids are `900,000,000` plus the book's
address — two digits of directory and seven of filename. Gutenberg is at 78,000
after fifty years, so the ranges cannot meet. Both halves of the address go in
because the file stem alone is not unique: `1000621` is *The Last Lemurian*
under `ebooks11` and *A Mummer's Throne* under `ebooks10`.

**One `catalog` table**, holding both libraries, because the question the page
asks is "what can I listen to tonight" and that has one answer. Which library a
result came from is read back off its id rather than stored, so there is one
fact and not two that can drift.

**A new `catalog_urls` table** mapping gid to address, written at import. Not
derived: sixty-nine of the four thousand break even the loose naming rule, so a
computed URL would 404 at the front of the queue, hours after the press, and
read as "Gutenberg does not have this book".

**One parser.** Australia marks its books up the same way — a heading per
chapter, a `<p>` per paragraph — and `parse_book_html` handled them unaltered
the first time it was tried. Only what wraps them differs, which is one
function that cuts on the HTML comments fencing the site banner and footer.

## Consequences

Nothing downstream of the catalog learns that there are two libraries. The
queue, the player, the audio routes, the semantic index and every saved
position keep the column they already have, and no migration touches them.

`catalog_urls` is a new table, so `CREATE TABLE IF NOT EXISTS` really does reach
an existing database. A missing row means Gutenberg proper, which is exactly
what every database that predates this table already meant, so there is nothing
to backfill and no window in which search returns nothing.

The catalog is rebuilt from both lists at once. A library that is unreachable
costs the whole update rather than half of one, and `somnia catalog-update`
now makes two requests instead of one.

Nine-digit ids appear in the CLI, in URLs and in the page. They are ugly and
they are load-bearing: the id says which library the book is from.

**somnia does not check whether an Australian book is public domain where the
listener is, and cannot** — the index carries no author death dates. The
library is cleared for Australia only and says so itself. That check is the
reader's.