# HTTP API and CLI

Reference for scrap's HTTP API, command line and settings. Requests and responses are JSON unless noted.
Responses of text (JSON, the PWA's scripts, styles and pages) over 1 kB are gzipped for a client
that sends `Accept-Encoding: gzip`; stored files, previews, downloads, fonts and range requests
never are.
The service listens on `127.0.0.1`, or with `scrap serve --uds` only on a Unix socket (the
single-server install's default), and has no login; put Tailscale Serve (or an authenticating
proxy) in front. The author of a capture or edit is the `Tailscale-User-Login` header (or the header named by
`SCRAP_IDENTITY_HEADER`), else `tagged`; a header naming `claude` (MCP's author) counts as none.
With `SCRAP_OIDC_ISSUER` set it is instead the username (`preferred_username`) in the login proxy's token.
A text revision made over HTTP is always source `web`, never Claude's.

## HTTP

### Host and Origin

Every route but `/mcp` (which has its own, the same check) and `/healthz` checks the request's
`Host` and `Origin`, against the allow-list `/mcp` uses ([](mcp.md) has its rules), so one setting,
`SCRAP_PUBLIC_URL`, covers both. This stops another web page from reaching scrap through your browser:
a DNS rebinding page (its name as `Host`) and a form or `fetch` it sends from its own site.

- **Host**: loopback (`127.0.0.1`, `localhost`, `[::1]`, any port) or a host of `SCRAP_PUBLIC_URL`,
  else `421`. Case and a trailing dot are ignored; two `Host` headers are refused. On these routes
  an IP address is always accepted, so the web app of an install reached at
  `http://192.168.1.10:8000` needs no setting; `/mcp` does not accept a bare IP address, so list it
  in `SCRAP_PUBLIC_URL` for Claude Code. A *name* (a MagicDNS or LAN hostname, a Cloudflare or Serve
  address) must be in `SCRAP_PUBLIC_URL`, as it must for `/mcp`. `X-Forwarded-Host` is ignored: the
  proxy must pass the public `Host` through (Tailscale Serve and Cloudflare do).
- **Origin**, on `POST`, `PUT`, `PATCH` and `DELETE` only: none sent (curl, the CLI, the hooks) is
  accepted; one sent must be a loopback `http://` origin or a `SCRAP_PUBLIC_URL` origin, or, for an
  install reached by IP address, that address and port. Anything else, `null` included, is `403`, as is any write the browser marks `Sec-Fetch-Site: cross-site`. The one exception is `Origin: null` on `/share` with `Sec-Fetch-Site: none` or `same-origin` (the Android share sheet, if the service worker did not take the POST).
  Browsers always send `Origin` on a cross-site write, so a page cannot leave it off.

### Errors

An error is a JSON body `{"detail": ...}` with the status that says what went wrong (the two
Host and Origin refusals are plain text instead, `Invalid Origin header` and `Invalid Host header (add its
address to SCRAP_PUBLIC_URL)`):

- `403`: a write (any method but `GET`, `HEAD` or `OPTIONS`) whose `Origin` header is not allowed.
- `421`: a `Host` header that is not allowed. Both checks are described under [Host and Origin](#host-and-origin).
- `400`: a multipart text field (`text`, `url`, `title`...) over 1 MiB, refused by the form
  parser before scrap sees it; the body is `{"detail": "Part exceeded maximum size of 1024KB."}`.
  A JSON body of that size is `422` instead (see `POST /api/captures`).
- `404`: no such page, capture, photo, drawing version or stored file. A page in the trash is not
  found either, except by the undelete route. So is a path that is neither under `/api` nor one of
  the PWA's routes below, and a PWA route with a trailing slash (`/todos/`): `{"detail": "Not
  Found"}`, with no fallback to the PWA.
- `409`: a conflict. A stale page edit (`rev` in the body is the current revision) or a stale
  drawing edit (`version` is the current version); also *Ask Claude* while enrichment is off.
- `413`: over the upload limit (`SCRAP_MAX_UPLOAD_MB`) or, for a drawing, 8 MB.
- `422`: invalid. `detail` is a string for a rule scrap checks, but a list of validation errors
  for a malformed query parameter or body (for example `limit=501` on `GET /api/pages`).
- `503`: as listed for the route (the worker has stopped, the PWA is not built, the preview decoder
  is busy). The one error that is not JSON is the PWA-not-built `503`, which is plain text: the
  build command, `npm --prefix web ci && npm --prefix web run build`, and a note that the API
  works without it. The service logs the first line of it as a warning at start. Without the PWA,
  `/api`, `/healthz` and `/mcp` work as usual.

Stored files and previews are content-addressed, so they are served with `Cache-Control: private,
max-age=31536000, immutable`, `X-Content-Type-Options: nosniff` and `Content-Security-Policy:
sandbox` (an uploaded HTML file cannot run scripts). Every stored file but a PDF, and an SVG's
preview too, has the stricter `Content-Security-Policy: sandbox; default-src 'none'; style-src 'unsafe-inline';
img-src data:; font-src data:`, so it fetches nothing either; a PDF keeps `sandbox` alone, which
the browser's PDF viewer needs. A SHA-256 is 64 lower-case hex digits (the
`uses` and drawing routes also take upper case); any other value, or a file not stored, is `404`. A page download is `Cache-Control: no-store`. The
PWA's hashed files under `/assets/` are `immutable`; its other files are `no-cache`, so a new
build is noticed.

(api-page-object)=
### The page object

`GET /api/pages/{id or slug}` (and every route that returns "the page") gives `id`, `slug`,
`title`, `body`, `kind`, `kind_fixed`, `fields`, `created_at`, `updated_at`, `rev`, `tags`,
`removed`, `tasks` (`{done, total}`), `due` and `priority` (`null` unless a todo), and:

- `made_by`: who made the page, `human`, `claude` (written over MCP), `topic` (Claude's topic page)
  or `gc` (a `scrap gc` review page), stored when the page is made and never changed, whatever
  its kind or edits later; see {ref}`page-roles`. Not `fields.origin`, the
  provenance Claude gives a page it writes over MCP.
- `public`: whether the page is public ([](public-site.md)); `published_at`: when it was last
  published to the public site (UTC), `null` if it never was or was unpublished since (a page
  with `public` true and `published_at` null is "not published yet"); `public_url`: its address
  there while it is public (`SCRAP_PUBLISH_SITE` + `/p/<slug>`), else `null`; `site`: the
  public site's base URL (`SCRAP_PUBLISH_SITE`), or `null`.
- `captures`: each with `id`, `kind` (`note`, `url`, `photo` or `file`), `text`, `url`, `source`,
  `author`, `created_at`, `received_at` and `blobs`: one `{sha256, mime, size, filename, ingest}`
  per file, where `ingest` is as described for the route below.
- `model_tags`: the tags Claude added that are still on the page.
- `related`: pages Claude picked, pages that link here, pages a board's write-up links to or
  whose write-up links here, and pages a topic's synthesis cites or topics whose synthesis cites
  this page, each `{id, slug, title, kind, via}` (`via`: `claude`, `link`, `both`, `writeup` or
  `topic`).
- `enrich`: `{on, state, error, reason, summary, producer, source, at}`, or `null` if the page was
  never asked about and enrichment is off. `state` is `queued`, `processing`, `done`, `failed` or
  `paused` (queued while the shared queue is paused: `reason` says why, as in `GET /api/ingest`).
- `writeup`: Claude's write-up of a board page (a whiteboard or diagram image, not a list of
  actions): `{board, on, guidance, state, error, outdated, current}`, or `null` for a page with
  none and no board. `board`: whether one can be written; `on`: enrichment is on; `guidance`: the
  owner's for the next one; `state`: its latest write-up or Ask job's, as `enrich.state`;
  `outdated`: the current write-up is outdated, as `is:outdated` has it (a
  board image read, or the page's text, title, kind or tags changed, since it was made, or an
  older prompt, or its text links a page in the trash or gone; always `false` for a topic's synthesis, which the
  weekly pass remakes); `current`: `null` or `{producer, created_at, prompt_version,
  summary, text, guidance, links}` (`text` is Markdown, its links `[[slug]]`;
  `links`: the slugs of the pages, not in the trash, it links to: the PWA links no others; `titles`: their titles by slug).
  A topic page's synthesis is its write-up: the same object, with `topic` true (`board` false).
- `topics`: the topics (not in the trash) the page is a source of, each `{id, slug, title}`.
- `topic`: `null`, or for a topic page (one Claude made for a recurring thing, its `body` the
  owner's notes: see {ref}`ingest-topics`) `{key, sources}`, `sources` each `{id, slug, title,
  kind, added_by, added_at, excluded}` (not in the trash; `excluded` 1: the owner took it out).

The rows of `GET /api/pages` and of a search are the page without `body`, `captures`, `tags`,
`removed`, `rev`, `model_tags`, `related`, `enrich`, `writeup`, `topics` and `topic`, plus `thumb`; search rows also have no
`kind_fixed` or `created_at`, and have a `snippet`, and `topic` as a boolean: whether it is a topic page.

| Method and path | Does |
| --- | --- |
| `GET /healthz` | Liveness: `{"status": "ok", "version": ..., "max_upload_bytes": ..., "paused": ..., "publish": ...}` (`paused`: why the Claude queue is paused, `credit`, `auth`, `budget` or `outage`, else `null`); the PWA checks a capture's size against the limit before saving it. `503` (`"status": "ingest worker stopped"`) if ingest is on and its thread has died; `publish` is `{enabled, site, pages, last_reconcile, ok, error}` (public pages: whether publishing is on, the site's address, how many pages it holds, and the last reconcile's time, outcome and error), which never makes the check fail |
| `POST /api/captures` | Save a capture as a new page. JSON `{id, text?, url?, kind?, source?, created_at?, due?, priority?, tags?}`, or `multipart/form-data` with the same fields (a `tags` field per tag) plus one or more `file` parts (`file`, singular; `POST /share` takes `files`). `id` is a ULID minted by the client (any case; stored upper case): `201` the first time, `200` with the same reply on a retry. The `id` is all a retry is matched on: the repeat is checked like any request (`422` if it is invalid), then answered with the first save's reply and its text, files and tags thrown away, so changed content needs a new `id` (files sent with a retry are stored, but no page uses them; `scrap gc` clears them). `kind` (`note`, `todo`, `consume`, `whiteboard` or `document`) is the user's choice: it fixes the page kind (`kind_fixed`, so ingest keeps it) and fills its default `fields`; without it the page is an unfixed `note`. `hint` (`todo` or `consume`) is the older name for `kind`, still accepted. `source` is `pwa` (the default) or `share`; `mcp` is refused here: only the MCP server writes it, so it marks a page Claude wrote. `due` and `priority` are a todo's only (as for `PATCH`; `422` with another kind); a `snoozed_until` sent is ignored (Snooze was retired, 28 Oct 2026). `tags` (a list of strings) are the new page's tags, cleaned as `PATCH` cleans them (`422` for a bad one, and nothing is saved), indexed at once; Claude's tags for a photo join these on its first reading of it (a later Reprocess adds them only to a page with none, so a tag the user took off stays off). A bare URL sent as `text` is treated as `url`. The capture's own kind is derived: `photo` if any of its files is an image (its images are sent to Claude, and stored before its other files, keeping the order sent, so the page's title, tags and kind come from an image; a PDF or text file among them is still read for search), `file` for other files only, else `url` if it has a `url`, else `note`. A file part sent with no content type, an empty one or `application/octet-stream` is typed from its filename's extension, else kept as `application/octet-stream`; one sent with a type that is neither an image nor a PDF, text or Markdown type (a phone's `text/x-markdown`) takes its filename's type when that is one of those; and a file's stored type is the one it was first saved with, whatever a later upload of the same bytes says. For each image file, `![](/api/blobs/<sha256>/preview?size=2048)` is added to the end of the page's `body`, after the `url` and `text` (once per distinct image, and not for one the text already links by its address), so the `body` of the page is longer than what was sent (see {ref}`images-in-the-text`). With the multipart field `place` set to `1` (the PWA composer's, #403), the `text` places its own files with tokens: `⟦image N⟧` or `⟦file N⟧` (also `⟦file N: any name⟧`), `N` the file's place among the request's `file` parts, from 1. Each is replaced by that file's embed: an image's as above, any other file's link `[filename](/api/blobs/<sha256>)` (brackets and backslashes left out of the name); the word `image` or `file` is a hint only, the stored type decides. A token for no file stays as typed; a file with no token is added as above (an image's embed at the end, another file only listed). Without `place` (or with no files) nothing is a token: text that looks like one is kept as sent, as it always was. The page's `title` is the first line of `text` with words (with `place`, its tokens left out), without a leading Markdown heading marker (`#` to `######` and a space), at most 200 characters; else the `url`; else `created_at` as `YYYY-MM-DD HH:MM`, in the offset it was sent with (UTC if it was not sent). The slug is made from that title once and stays when the title is edited (see [](../explanations/architecture/data-model.md)); Claude may replace the title of a photo with no text (with `place`, tokens alone are no text: the capture's `text` is then `null`), or of a saved link, with its own. `422` also for an `id` that is not a ULID, a `created_at` that is not ISO 8601, a capture with no text, url or file, a request that is neither JSON nor `multipart/form-data`, and a JSON body that is not an object. Returns `{capture_id, page_id, slug}`. `413` over the upload limit; `422` if the page's text (the `url` and `text`) is over 1 MB of UTF-8, before any file is kept. Files are stored as sent (HEIC too), but an SVG (`image/svg+xml`), which is stored sanitised: at most 1 MiB, a DOCTYPE that only names an external DTD (as matplotlib writes) dropped unread and one with an internal subset or an entity refused, its scripts, event handlers, `foreignObject`, animation and links out of the file removed (`422` if it is not an SVG, `413` over 1 MiB; see {ref}`mcp-uploads`) |
| `GET /api/pages?limit=50&kind=` | Pages, most recently changed first (by their text: a kind or `fields` change doesn't count); the trash is left out. `limit` is 1 to 500 (default 50; `422` outside). `kind=todo` lists only that kind (`note`, `todo`, `consume`, `whiteboard` or `document`; another is `422`; empty means all). Each has `id`, `slug`, `title`, `kind`, `kind_fixed`, `fields`, `created_at`, `updated_at`, `tasks`, and a todo's `due` and `priority` (`null` on other kinds); `thumb` is the SHA-256 of the page's first image, or `null` |
| `GET /api/pages/{id or slug}` | A page with `rev` (its current revision), `tags` (most used first), `removed` (the SHA-256s of files an earlier revision's text referred to and the current text does not: the PWA no longer lists them), `titles` (slug to title for each page, not in the trash, its text links to or names by its `/p/<slug>` address: a link with no label of its own shows that title, read with the page so a rename shows at once; a dangling link has none), and its captures, each with its files as `blobs` (see {ref}`the page object <api-page-object>`, which also lists `model_tags`, `related` and `enrich`). Each file (`{sha256, mime, size, filename, ingest}`) of a photo has `ingest`: `null` if it was never queued, else `{state, failures, error, text, description, image_kind, drawing}` (`drawing`: `null`, or the current version's `{id, version, producer, width, height}`), `state` being `queued`, `processing`, `done`, `failed` or `paused` (see [](../explanations/ingest.md)). An image's file also has `guidance`: the owner's guidance for its next reading, or `null`. A PDF or text file (`text/plain`, `text/markdown`), whose text scrap reads itself for search (no model), has instead `ingest`: `null` if it was never queued (a file attached before scrap read files: `POST /api/captures/{id}/reprocess` queues it), else `{state, failures, error, kind, pages, pages_read, truncated, text_chars, searchable}`: `state` `queued`, `processing`, `done` or `failed` (never `paused`: a pause does not hold a file), `error` the reason it failed (for example `content: the PDF could not be read`, `too large: 41 MB, over the 30 MB limit`, `timeout: ...`), `kind` `pdf` or `text`, `pages` and `pages_read` a PDF's page count and how many were read (`null` until it is read, and for a text file), `truncated` whether the text was cut (at the first 200 pages or 500,000 characters), `text_chars` the length of the text kept and `searchable` whether there is any (`false` for a scan with no text layer). The text itself is not sent with the page (MCP `read` gives it) |
| `POST /api/pages/{id or slug}/captures` | Add photos or files to a page that exists: an older build's page editor **Add photo** (one still queued on a phone), or any other client; the page editor now sends its files with `PATCH` (below). `multipart/form-data` with `id` (a ULID minted by the client: the retry key, as for `POST /api/captures`), one or more `file` parts, all images, PDFs, or text or Markdown files (`text/plain`, `text/markdown`; typed as for a capture, so an untyped `.md` or `.txt` is typed by its name), `source?` and `created_at?` as for a capture, and `alt?` (the images' embeds' alt text, one line, at most 300 characters) and `after?` (text of the body, occurring exactly once: the embeds go right after the line it ends on, inside its list item or quote: a paragraph is split there, a tight list's line ends with a hard break (`\`) and the embeds follow on the next lines, so the list stays tight, and elsewhere they are paragraphs of their own; after a block that cannot be split (a code block, a table, display maths, a heading); and at the end if no place there keeps the page as it was, instead of at the end; see {ref}`mcp-uploads`; CR LF line ends match as LF; `422` if it is not there or occurs more than once); `415` for any other file, and nothing is saved. An SVG is stored sanitised, as for a capture. The files become a capture of the page, received now, so they come after the page's own (captures are in the order the server received them): a `photo` capture if any of them is an image (its images are sent to Claude as Add photo's always were, and a file among them is read as a file), else a `file` capture. A PDF or text file is not put in the text: the page lists it below, and its text is read for search as a capture's is (see [](../explanations/ingest.md)). For each image the text does not show already, `![](/api/blobs/<sha256>/preview?size=2048)` is added to the end of `body` as it is now, all in one new revision by the caller (none if there is nothing to add, as when only files are added). No revision is checked (a `base_rev` sent is ignored): appending loses nothing. The photos are queued for Claude as a capture's are, and are read and searched, but never change the page's kind, title or tags, nor whether enrich or ingest maintains it: a page's first capture decides that (see [](../explanations/ingest.md)). `404` for an unknown page or one in the trash; `422` for a topic page (its synthesis is its write-up, which a board photo would replace), an `id` that is not a ULID or is another page's capture, no file, a bad `source` or `created_at`, a request that is not `multipart/form-data`, or a `body` that would be over 1 MB; `413` over the upload limit. Returns the page (as `GET` does) with `appended_on`, the revision the embeds went onto (the page's revision before, so a client holding it can take them into its own text; `null` when no image was added): `201`, or `200` with the page as it is now and `appended_on` `null` on a retry with the same `id`; `added`, each file sent's `{sha256, mime, filename, download_url}`, and `placed`: `after` (the embeds went where `after` asked), `end` (no place there kept the page as it was, so they went at the end) or `null` (no `after`, or no new embed). Also served as `POST /mcp/pages/{id or slug}/captures` for MCP clients, behind the MCP bearer token, unless `SCRAP_MCP_UPLOAD=0`: there the change is Claude's (author `claude`, source `mcp`, whatever `source` or identity header is sent), `download_url` is under `/mcp/blobs/`; see {ref}`mcp-uploads` |
| `PATCH /api/pages/{id or slug}` | `{base_rev?, title?, body?, kind?, fields?, tags?, tags_shown?, due?, priority?, public?}`. A `title` or `body` writes a new revision and needs `base_rev`: `409` with the current `rev` if it is stale (and nothing in the request is saved). A `title` has runs of whitespace collapsed to single spaces; one that is empty or only whitespace is `422` (`title must not be empty`) and nothing in the request is saved. A `title` equal to the title at `base_rev` counts as unchanged, so it never undoes a title Claude set meanwhile. The slug does not change with the title. `kind` (as for a capture) sets `kind_fixed` (ingest then never changes it); a different kind keeps every field the page has and adds that kind's defaults for keys it lacks: `todo` `{"status": "open"}`, `consume` `{"subtype": "read", "status": "open", "source"}` (`source`: the capture's URL, if any). A default is added only if the page has room (at most 32 fields): a page with 31 fields gets one of them, a full one none, and a field the page has is never dropped for it (so the change is not a `422`). So nothing is lost by changing a kind and back; a `status` already there keeps its value. `fields` changes them, e.g. `{"status": "done"}` ticks a todo; a `null` value removes one; names are 1 to 40 characters, values flat (a string of at most 200 characters, a number or a boolean; a whole number is within ±2^53 and any other finite, so `NaN`, `Infinity` and `1e400` are `422`), and a page has at most 32 fields (`422` otherwise). `tags` replaces all the page's tags (`[]` removes them), each cleaned as an import's are (one space between words, no leading `#`, lower case, 1 to 64 characters, each once; at most 64 tags, `422` otherwise). A tag Claude added that is not in `tags` is taken off for good (Claude never adds it back); with `tags_shown`, the tags the editor started from (the PWA sends them), one Claude added that is in neither list is kept instead, since the user never saw it. `due` (a date, `YYYY-MM-DD`: no time, no time zone) and `priority` (`high`, `medium` or `low`) are a todo's own values, not fields (a `snoozed_until` sent is ignored: Snooze was retired): sent, they set it, `null` clears it, not sent keeps it; a value is `422` on a page of another kind (after any `kind` in the same request: `{"kind": "todo", "due": ...}` works), as is a malformed one. A page that stops being a todo keeps them (a stray tap loses nothing), but they are ignored, and returned as `null`, until it is one again. Neither `kind`, `fields`, `tags`, `due` nor `priority` is a text revision, nor changes `updated_at`; a change of `kind` or `tags` is a metadata revision (`page_meta_rev`: who, kind and tags before and after; see [](mcp.md)), as Claude's are, so it can be seen and reverted over MCP. A `body` (here, in a capture or over MCP) is counted for its task boxes, `tasks: {done, total}` on the page and its rows (exactly as the PWA draws them: a list item whose first line starts `[ ]` or `[x]` as typed; none in code, escaped `\[ ]`, or a link named `[x]`): a `note` whose kind nobody chose (`kind_fixed` 0: no `kind` was ever sent for it; the PWA sends one only when the user picks a kind, never for boxes) that has boxes becomes a `todo`, filed as a new capture of that text would be (all ticked: `done`); a todo's `status` becomes `done` when the last box is ticked, and, on a done one whose boxes were all ticked, `open` again when one is unticked or a new unticked one is added (a todo marked done by hand with some boxes unticked stays done); never on a `cancelled` one; a status set by hand ticks no box, and a `fields.status` in the same request as the `body` is kept, whatever its boxes. A `body` over 1 MB (1,000,000 bytes) of UTF-8 is `422` (a page's text is at most that, however it is written; nothing in the request is saved), unless it is exactly the page's current body, so a page imported over the limit can still be retitled, retagged or given another kind. With files (the page editor's Camera, Gallery and File), the request is `multipart/form-data`: the edit as JSON in an `edit` part (with `body` and `base_rev`), and `file` parts, images, PDFs and text or Markdown files only (`415` otherwise; an untyped part is typed by its name), within the upload limit (`413`). Each `⟦image N⟧` or `⟦file N⟧` (also `⟦file N: name⟧`) in `body` becomes the Nth file's embed (`![](/api/blobs/<sha256>/preview?size=2048)`) or link (`[name](/api/blobs/<sha256>)`), as a capture's tokens do with `place`, and the files are recorded on the page as a new capture (a `photo` capture if any is an image, so its images are read by Claude as Add photo's are, else a `file` capture; PDFs and text files are read for search), in the same revision as the text. A stale `base_rev` is `409` and records no file. Returns the page. `public` (`true` or `false`) sets the public flag as a metadata revision ([](public-site.md)): `true` only sets it (the PWA then publishes the page); `false` also takes the page off the public site at once, and if the site cannot be reached the answer has `"unpublish": "pending"` and the next reconcile finishes it |
| `POST /api/captures/{id}/reprocess` | `{photo?, replace_human?}`: send a photo capture's images (or the one with SHA-256 `photo`) to Claude again; also the *Retry* of a failed one. A capture's PDFs and text files (or the one with SHA-256 `photo`) are read again, locally, for search: the *Retry* of a failed file. A file read again this way never sends its text to Claude: only a capture's own first read of a file page does, with `SCRAP_FILE_ENRICH=auto`. To send it later, use *Ask Claude*. A new drawing replaces one the user drew only with `replace_human: true` (otherwise it is kept in the drawing's history only); the confirmation also reaches a photo already queued or running. Returns `{queued, already}`: how many were added, and how many were queued or running already. `404` for an unknown capture or one whose page is in the trash. The jobs are queued whether or not `SCRAP_INGEST` is on; with it off they wait for a worker |
| `GET /api/captures/{id}/files/{sha256}/text` | The text scrap read from one of a capture's PDFs, for search (no model): `{text, truncated}`, its first 20,000 characters (`truncated` if there is more, or the PDF was longer than scrap reads); `text` is `null` while none was read (not read yet, failed, or a scan with no text layer). `404` for a file that is not a PDF of that capture (a text file's own address serves its text), or a capture whose page is in the trash. The page view's file card |
| `PUT /api/captures/{id}/photos/{sha256}/guidance` | `{guidance}`: the owner's guidance for the image's next reading (#538; at most 2,000 characters, else `422`; trimmed; empty or `null` removes it), kept apart from what Claude read. Queues nothing: the next `reprocess` of it (Regenerate) sends it with the image, delimited as the owner's, and it steers the description, the text read and the drawing (a drawing the user drew is still replaced only with `replace_human`). It never steers a write-up, and a page's write-up guidance never steers a reading. Never published. Returns `{guidance}`, what is kept. `404` for an unknown capture, one whose page is in the trash, or a file it does not hold; `422` for a file that is not an image, or an image of a capture Claude made over MCP (Claude doesn't read its own images) |
| `GET /api/captures/{id}/photos/{sha256}/drawing` | The photo's current drawing: `{id, version, producer, author, run_id, created_at, current, superseded_at, format, restored_from, data}` (`superseded_at` is `null` while it is the current version). `data` is `{format, width, height, elements}`: `format` `skeleton` (Claude's, Excalidraw element skeletons the PWA expands) or `scene` (a saved edit: full Excalidraw elements, and `appState.viewBackgroundColor`). `width` and `height` are the frame the coordinates are in: the pixels of the JPEG sent to Claude, so the PWA lines the drawing up with the photo. `404` if there is none |
| `GET /api/captures/{id}/photos/{sha256}/drawing/versions` | Every version, newest first, as above without `data`. `producer` is the model, or `human` with its `author`; a restored version has `restored_from` |
| `PUT /api/captures/{id}/photos/{sha256}/drawing` | `{base_version, scene: {elements, appState?, width?, height?, files?}}`: save an edit as a new current `human` version, its text elements indexed for search. `base_version` is the version it was edited from (`null` if there was none): `409` with the current `version` if that is no longer current. The scene is checked (see [](../explanations/ingest.md)): only rectangle, ellipse, diamond, text, arrow, line and freedraw elements; finite numbers; Excalidraw's colours; `http(s)` links only; nesting at most 32 levels deep; valid Unicode (no lone surrogates); `files` dropped; at most 5 MB as compact JSON, else `422` and nothing is saved. `413` if the request is over 8 MB. Returns the new current drawing, as `GET` does (with `data`) |
| `POST /api/captures/{id}/photos/{sha256}/drawing/restore` | `{version, base_version}`: make an old version current again as a new copy (versions never change). `base_version` is the current version as the client last saw it: `409` with the current `version` if it has changed, so a stale history view cannot restore over a newer edit. `404` if there is no such version; `422` unless both are integers from 1. Returns the new current drawing, as `GET` does (with `data`) |
| `POST /api/pages/{id or slug}/enrich` | *Ask Claude*: queue the page for Claude now (tags, summary, a board's write-up, related pages; a saved link is fetched). Returns the page, whose `enrich` says `{on, state, error, summary, ...}`, `model_tags` the tags Claude added, and `related` Claude's related pages merged with backlinks (pages whose text links here with `[[slug]]` or a Markdown link to its address, as for `is:orphan`; `via`: `claude`, `link` or `both`). `409`, queueing nothing, while enrichment is off (`SCRAP_INGEST` off or `SCRAP_ENRICH=off`) |
| `POST /api/pages/{id or slug}/writeup` | *Regenerate*: queue a board page's write-up, or a topic's synthesis, again, with its guidance as it is when it runs. Returns the page (`writeup.state` `queued`). `409` while enrichment is off, `422` if the page is not a topic and has no whiteboard or diagram image. Guidance below works on a topic's synthesis too |
| `PUT /api/pages/{id or slug}/writeup/guidance` | `{guidance}`: the owner's guidance for the page's next write-up (at most 2,000 characters, else `422`; trimmed; empty or `null` removes it), kept apart from its text. Queues nothing. Returns the page |
| `GET /api/topics` | The topics (not in the trash), made from data with no model call: each `{id, slug, title, summary, sources, updated_at, area}`. `summary` is its synthesis's lead (`""` before one), `sources` how many sources are in, `updated_at` its synthesis's time (else the page's), `area` the top-level tag most common among its sources (a source counts a top-level tag once; ties by name; `null` if they have no tags). Grouped by `area`, the biggest group first and `null` last, by title within one |
| `GET /api/pages/{id or slug}/topic/sources` | A topic's sources, not in the trash, first added first, those the owner took out too: each `{id, slug, title, kind, added_by, added_at, excluded}`. `422` if the page is not a topic |
| `DELETE /api/pages/{id or slug}/topic/sources/{id or slug}` | Take a source out of a topic (the ✕): it is kept with `excluded` 1, so discovery never adds it again, and the topic's synthesis is queued again (with enrichment on). Returns the topic page. `422` if the page is not a topic, `404` if the other page is not one of its sources still in |
| `POST /api/pages/{id or slug}/topic/sources/{id or slug}` | Put a source the owner took out back into a topic (undoing the ✕): `excluded` 0 again, and the synthesis queued again (with enrichment on). Returns the topic page. `422` if the page is not a topic, `404` if the other page is not an excluded source of it |
| `GET /api/ingest` | The ingest queue: `{enabled, paused, reason, paused_until, jobs, enrich: {enabled, jobs}, spent_today_usd}`. `jobs` is `{state: count}` for ingest jobs as stored (for example `queued`, `running`, `done`, `dead`): photos' and files' (PDFs and text files, read locally) together, and `enrich.jobs` the same for text pages. A pause does not hold a file's job, and a file spends nothing. While `paused`, `reason` is `credit` (out of credit, or at a spend limit) or `auth` (the key was refused), both paused until *Resume* (`paused_until` is `null`), `budget` (the day's budget is spent: until UTC midnight) or `outage` (the API was unreachable, failing or rate limiting: a backoff that starts at 60 s and doubles up to an hour, or the API's own retry-after, at most an hour). *Resume* ends any of them. The day's spend counts photos, text pages and topic discovery. With `SCRAP_NOTIFY_URL` set, the first pause since the last success or *Resume*, and one for a new reason, is announced once |
| `POST /api/ingest/resume` | End a pause; returns the queue as `GET /api/ingest` |
| `GET /api/pinned` | The Pinned page as `GET /api/pages/pinned` has it, or `null` (status 200) while there is none; the search screen asks on every visit, so it is not a 404 |
| `POST /api/pinned` | The Pinned page (slug `pinned`): a page whose body is a list of Markdown links, which the PWA shows under the empty search box (`- [People](/search?q=tag:person%20group:letter)` for a search, `- [Contacts](/p/contacts)` for a page). Made at most once (title Pinned, body `# Pinned`), brought back from the trash if it is there, and a page that already has the slug `pinned` is it, so two first pins at once make one page. Returns the page as `GET /api/pages/pinned`; edit it with `PATCH` and `base_rev` (`409` on a stale one: read and retry). A slug never changes with the title, so a link to a page by slug survives a rename. Claude does not read this page (no enrich, write-up or related call; Ask Claude still works) |
| `DELETE /api/pages/{id or slug}` | Move a page to the trash (`204`). Its captures and files are kept until the nightly `scrap gc` purges it, `SCRAP_TRASH_DAYS` later (`GET /api/trash` says when). A published page is taken off the public site at once (its `public` flag stays) |
| `POST /api/pages/{id or slug}/undelete` | Bring a page back from the trash (until it is purged). A public page stays public, not published yet (`published_at` null) until the PWA renders it again |
| `PUT /api/pages/{id or slug}/published` | Publish a public page: JSON `{html, base_rev, defer?}` (at most 25 MiB) from the PWA's publish mode, checked, its placeholders replaced and pushed with its files to the public site; then, unless `defer`, the home, tag and search units. `200` `{url, published_at, files}`. Errors are `{error, detail}`: `409` `not_public` (not public, or in the trash), `409` `stale` (with `rev`), `413` `too_large`, `422` (`bad_request`, `bad_html`, `bad_placeholder`, `too_many_files`, `file_too_large`, `unit_too_large`, `stale_tags`), `502` `publish_failed` (`detail`: the receiver's code), `503` `not_configured`. Only under `/api`: no `/mcp` route publishes. See [](public-site.md) |
| `GET /api/public?links_to=` | The public pages not in the trash: `[{id, slug, title, published_at}]` (`published_at` null: not published yet). With `links_to=<slug>`, only those that refer to that page in scrap (their text or write-up links to it, or they are related to it either way), whatever the published copy shows |
| `POST /api/public/rebuild` | Build the public site's home, tag, search and assets units once and push those that changed (after Refresh all's deferred publishes): `200` `{units: [...]}`, or `502` / `503` as for a publish |
| `GET /api/store/download` | Every page not in the trash as Markdown with its files, as `scrap-export-YYYY-MM-DD.zip` (the UTC date): the files `scrap export` writes, laid out as [](export-format.md) says (Settings, **Export everything**). The zip is sent as it is made: the pages are read first, in one transaction, then each file follows a chunk at a time, so it is never held whole in memory or on disk. Photos and other files are stored in the zip as they are, the Markdown compressed. It changes nothing in the store. A file deleted (by `scrap gc`) after the pages were read is left out |
| `GET /api/import` | What Settings shows before an import: `{max_upload_bytes, enrich, photos, page_usd, photo_usd}`: the largest zip `POST /api/import` takes, whether Claude can tag and summarise text pages (`enrich`) and read photos (`photos`) on this server, and roughly what each costs in dollars, a text page (its enrich call and its first related call, on the text model) and a photo (one read, on the vision model), from typical token counts and the models' prices |
| `POST /api/import` | Import a zip of Markdown pages (a scrap export, laid out as [](export-format.md), or an Obsidian vault): multipart `file`, and `enrich` and `photos` (`1` queues each new note, todo or read for Claude to tag and summarise, and each new page's photos for Claude to read; both off unless sent). Every new page gets the tag `import/YYYY-MM-DD` (the UTC date); a page here already (by its front matter `id`, in the trash too, else by its path and bytes) is skipped and never overwritten. Folders starting `.` or `_` (`_claude/`) are not read. Returns `{new, skipped, enrich, photos, files, tag, kinds, issues}`: the pages imported and skipped, what was queued (`files`: PDFs and text files read locally, always), the tag, the new pages by kind, and `issues`, each kind of problem with its `count` and its first 20 `items`. `413` over the upload limit (`scrap import` takes any size); `422` for a file that is not a zip, or a zip refused whole: a name reaching outside it (`../`, `/`, a drive, a backslash), more than 100,000 files, or more than 20 times the upload limit unpacked. See [](../how-to/import-your-pages.md) |
| `GET /api/trash?limit=100` | The pages in the trash, most recently deleted first (`limit` 1 to 500): `{days, total, results}`. `total` is how many pages are in the trash (`results` stops at `limit`); `days` is how long a page stays there (`SCRAP_TRASH_DAYS`; `null` with it `off`). Each result is `{id, slug, title, kind, deleted_at, updated_at, topic, purge_at}`: `purge_at` is `deleted_at` plus `days`, from when the nightly `scrap gc` purges the page for good; `null` for a topic page (`topic` true: never purged, its key stops Claude making the topic again) or with purging off. The PWA's Trash screen (Settings) |
| `GET /api/search?q=...&limit=20&offset=0` | `limit` at most 100; `offset` (at most 10 000) skips that many hits, for the next page. Full-text search of titles, tags, the `summary` field, bodies, what Claude read from photos (their text and descriptions) and the text scrap read from PDFs and text files: every word must match somewhere in the page, as a prefix (`vsc` finds `vscode`), wherever it stands in the query; one hit per page. Pages with every word in the title come first, then those with every word in the title, tags or summary, a matching topic page first within each of those, then by bm25. Link and image addresses in Markdown are not indexed, only their text. The query may also have `-word` or `!word` (the exact word in no part of the page), `"a phrase"` (exact), `kind:K` (a kind, or the PWA's `later` (also `read`), `board`, `doc`; two mean either), `tag:T` (a tag with `T` as whole `/` parts: `tag:a` finds `a/b`, `tag:b` finds `a/b`; `tag:"two words"`), `is:personal`, `is:work` (`personal` not true), `is:claude` (written over MCP: `made_by` `claude`), `is:claude-edited` (text changed by Claude after it was made: a text revision from source `mcp` after the first, over MCP or the task boxes Claude adds), `is:topic` (Claude's topic pages), `is:public` (public pages, published or not), `is:S` (status `S`; spaces in a status are written `_`), `has:image` (an image in one of its captures), `has:drawing` (a current drawing; another `has:` value matches nothing), the store's health checks (each also counted by `GET /api/health`; [](../how-to/use-the-pwa.md) says what each finds): `is:orphan` (no page link in its text or its current write-up either way to another page not in the trash (a `[[slug]]` or `[[slug#heading]]`, or a Markdown link or `<autolink>` to `/p/<slug>`, as a path or on any address in `SCRAP_PUBLIC_URL`; outside code, not an embed or image), no related pick either way, and not a topic with a source in or a source in a topic, nor a `scrap gc` Orphaned assets page), `is:untagged` (no tags; not a topic page or a `scrap gc` Orphaned assets page), `is:stale` (an open todo, not done or cancelled, not due today or later, whose text, title, kind and tags have not changed for 60 days; not an Orphaned assets page), `is:duplicate` (a file of one of its captures is also another page's, neither in the trash nor an Orphaned assets page), `has:dangling` (a `[[...]]` in its text the server parses as a page link, or a Markdown link to a page's address as for `is:orphan`, outside code and not an embed, to no page or to one in the trash; this includes forms the PWA draws as plain text, such as `[[Some Title]]` or `[[#heading]]`; a `[[slug#heading]]` is a link to `slug`, as the PWA, downloads and exports link it, and a `/p/<slug>#heading` address is one too), `has:open-questions` (a page with a Markdown heading (any level, setext too; not in code, a quote or a list item) that reads "Open questions" in any case (emphasis and links in it are ignored, a trailing colon too) and whose section is open. The section runs to the next heading of the same or a higher level, so deeper sub-sections and their boxes are part of it. A section with task boxes is decided by them alone: any unticked `- [ ]` makes it open, all ticked is done even with prose left. With no boxes it is open when it has any content (a paragraph, list, table or code); an empty one, or one with only sub-headings, is not. Topic pages are skipped (a "Contradictions and open questions" heading is not one), and the page-wide `tasks` counts are not the test), and `is:outdated` (a board page's current write-up older than a reading of its board images or a change to its text, title, kind or tags, or made with another prompt version, never one the owner restored; not a topic's synthesis or related pages, which their passes refresh), `image:K` (a photo Claude took for kind `K`: `photo`, `whiteboard`, `handwriting`, `diagram`, `screenshot`, `document`, `other`), `after:D` and `before:D` (last edited on or after, on or before `D`: a date `YYYY-MM-DD`, a UTC day with both ends included, or a time ago such as `1d`, `2w`, `1m` (30 days), `1y` (365 days): the number is 1 to 100 (so `100y` works but `101y` and `1000d` match nothing), at most three digits, and may be left out for 1; a value that is neither matches nothing, and after `-` everything) or `date:created` (`after:` and `before:` use the creation date instead), `due:D` (a todo due on `D` (also written `=D`), or with `<`, `<=`, `>`, `>=` before or after it: `D` is a date `YYYY-MM-DD`, a time ahead such as `7d`, `2w`, `1m` or `1y` (as for `after:`, from today; negative or past times are not accepted), or `today`, `tomorrow`, `yesterday` on the server's own calendar, its local time zone (`TZ`); `due:<today` is overdue; `due:none` a todo with no due date; another value matches nothing), `priority:P` (a todo's: `high`, `medium`, `low`, or `none`; two mean either; Thoth's `urgent` means `high`), `media:M` (a Later page's medium: `book`, `article`, `film`, `tv`, `video`, `podcast`, `music`, `game`; two mean either) and `later:S` (what it is for: `read`, `watch`, `listen`, `play`; a page with no medium counts as `read`), and `group:G` (`letter`, `tag`, `type`, `now`, `due`, `area`; the last known one counts) which does not filter but orders the results by the app's sections first (a section is never split across pages of results, `offset` and all): `letter` by title's first letter and then A to Z unless `sort:` says otherwise, `tag` by the first `tag:` filter's sub-tags, `type` by `later:` subtype, `now` and `due` for todos (Overdue, Today ... as the Todos screen); done and cancelled pages come last in every group; a filter after `-` excludes. `sort:updated` (last edited first), `sort:created` (newest first), `sort:title` (A to Z, ignoring the case of ASCII letters only: `é` and `É` sort apart), `sort:due` (soonest first) or `sort:priority` (high first; for both, pages without one, and pages that are not todos, come last either way) orders the results instead of best match (or newest first), with `-sort:` the other way (`-sort:created`: oldest first); the last one counts, and another value is ignored. `sort:` alone lists every page in that order. Filters alone list their pages, newest first, with the summary or the start of the text as the snippet. Words without a letter or digit (`&`, `?`) are ignored. At most 64 distinct words and 64 filters are allowed: a query with more is refused with a 422 (nothing is searched), so a NOT word or `-tag:` is never silently left out. A filter with no value (`tag:`) is ignored. Tags have no empty path segments: `/lead`, `trail/` and `a//b` are stored as `lead`, `trail` and `a/b`, and `tag:/` matches nothing. Each hit has `id`, `slug`, `title`, `kind`, `fields` (so a todo's row can show its status), `due`, `priority`, `tasks`, `updated_at`, `thumb` (as in `GET /api/pages`), `topic` (whether it is a topic page: the PWA labels it), `tags` and an HTML-escaped `snippet` with matches in `<mark>`. `group:area` orders topic pages by their area, as `GET /api/topics` does (the biggest area first, none last), other pages after them; a hit that is a topic page has `area` (the top-level tag most common among its sources, `null` if none) with its `topic: true` |
| `GET /api/search/facets?q=...&top=12` | What the pages a search finds have in common: `{total, kinds: [[kind, n]], tags: [[tag, n]], personal: {work, personal}, active: [{token, key, value, neg}]}`. `tags` are the `top` most used (12 unless asked, at most 1000) (a tag counts for its parents too, `a` for `a/b`), leaving out those the query asks for (its `tag:` filters, not `-tag:`) and those on every page found (unless only one page is found, when all its tags are given); `active` are the query's filters as typed (`sort:` too) |
| `GET /api/health` | The store's health checks, each one count of the pages (not in the trash) its search finds: `{checks: [{term, count}]}` with `term` `is:orphan`, `is:untagged`, `is:stale`, `is:duplicate`, `has:dangling`, `has:open-questions` and `is:outdated` (a board's outdated write-up only, also one linking a page in the trash or gone, not related pages or a topic's synthesis, which their passes refresh), in that order (the PWA's Store health screen). One query a check; nothing is stored and no model is called. It also has `publish`, as `/healthz` has it |
| `GET /api/tags` | Every tag on a page not in the trash, with the parents of `a/b` tags, how many pages have it (or one under it), and how many have that very tag (`0` for a parent no page was given): `[[tag, n, own]]`, in order |
| `GET /api/pages/{id or slug}/download` | The page to keep: `<slug>.md` (front matter, with its `tags`, `status` and `topics` if it has them and a todo's `due` and `priority` if it has them and its `tasks` count if it has task boxes, then the body, then Claude's notes and *Linked from*, see below), or if it has files `<slug>.zip` holding the `.md` and `files/` with the originals, linked relatively so it reads offline. Each file keeps the name it was uploaded under, made safe: any folder part is dropped, a run of characters other than letters, digits, `_`, `.`, space and `-` becomes one `_`, and the name is cut to 120 characters; a file with no usable name is named by the first 12 characters of its SHA-256 and an extension from its type. When two names are the same ignoring case, the later gets `-2`, `-3`... before the extension (`scrap export` names files differently: see {ref}`api-cli-folders`). Each photo is followed by Claude's description (a quote) and the text it read (a fenced block), and its drawing is beside it as `<photo name>.excalidraw` (Claude's skeleton converted approximately on the server: Excalidraw re-measures the text when it opens it) The query `notes=0` leaves out Claude's notes (below); `photos=preview` puts each photo in the zip as its 2048 pixel JPEG preview, named `.jpg`, which carries no EXIF or GPS (Share's **File** sends this; a photo that cannot be previewed is left out, and `503` with `Retry-After` means the photo decoder was busy). After the body and the file links come a section headed `## Claude's notes (written by a model, may be wrong)` (Claude's summary, the tags it added, a board's write-up or a topic's synthesis, as block quotes, then related pages; left out when there are none or with `notes=0`) and `## Linked from` (pages whose text links here). The text scrap read from a PDF or text file follows its link in a fenced block of at most 20,000 characters. With `SCRAP_PUBLIC_URL` set, `[[slug]]`, `[[slug|label]]` and `[[slug#heading]]` links outside code are written as `[label](address/p/slug)` using its first address (the label is the page's title if none is typed, or the slug, or `slug#heading`, for a link to no page); without it, a `[[slug]]` with no label is written `[[slug|title]]`; a `[[...]]` whose target is not a slug, such as `[[Some Title]]`, is kept as typed. `scrap export` and `GET /api/store/download` write a page differently: only your text in the page's file, Claude's notes in a file of their own (see [](export-format.md)) |
| `GET /api/blobs/{sha256}/uses?page={page id}` | `{pages}`: how many pages other than `page` (the trash left out) show the file, by an embed in their text or as a file of their own capture. The editor's warning when an embed is deleted. `404` if the file is not stored |
| `GET /api/blobs/{sha256}` | An original file, byte for byte, with its stored content type. `?download=1` adds a `Content-Disposition` with the name it was uploaded under, made safe as for a page download |
| `GET /api/blobs/{sha256}/preview?size=2048` | A JPEG of a photo (HEIC too), turned upright and without its EXIF, at most `size` (`480` or `2048`) pixels a side; for an SVG, the file itself (`image/svg+xml`, at either `size`), which scales, with nothing rasterised. Made on first request and kept in `cache/previews/` under the data folder, which is never backed up and can be deleted at any time. Made one at a time, shrunk as it is decoded, within a memory budget that fits a 1 GB container: `422` for another `size`, if the file is not an image Pillow can read or would need more memory than that to decode (roughly over 50 megapixels for HEIC, 80 for PNG; JPEG is decoded shrunk and fits at any size), `503` with `Retry-After` if the decoder stays busy for 20 s. The colour profile is kept |
| `GET /docs`, `/redoc`, `/openapi.json` | FastAPI's interactive documentation (Swagger UI and ReDoc) and the OpenAPI schema, generated from the routes. The PWA, `/share`, `/mcp` and the `/mcp/...` downloads are not in them, nor are the request bodies of `POST /api/captures` and the drawing `PUT`, and most responses are only `object`: this page is the fuller description. Open to anyone who can reach the service, like the rest |
| `POST /mcp` | The MCP server (Streamable HTTP, stateless, JSON replies) for Claude Code sessions: see [](mcp.md). `401` if `SCRAP_MCP_TOKEN` is set and no matching bearer token is sent, `421` if the `Host` header is not loopback or a `SCRAP_PUBLIC_URL` host, `403` for another `Origin`, `413` over 2 MB |
| `GET /mcp/blobs/...`, `/mcp/pages/{key}/download`, `/mcp/captures/{id}/photos/{sha256}/drawing...` | The read-only downloads above for MCP clients, by the same handlers, behind the `/mcp` bearer token (`401` without it when `SCRAP_MCP_TOKEN` is set), plus `.../drawing.excalidraw`, the drawing as an `.excalidraw` file. Not in `/openapi.json`. See {ref}`mcp-downloads` |
| `POST /share` | The PWA's share target, for when its service worker did not take the share (which it normally queues itself): saves the multipart `title`, `text`, `url` and `files` as a capture with a server-minted id (source `share`), then `303` to `/`; other fields are ignored. The fields are trimmed. With no `url`, a link that ends `text` after whitespace (where Android puts it) moves to `url`; a `title` that is not already in `text` goes first, with a blank line after it; a file part with no content is skipped. The page is then made as for `POST /api/captures` (titles, image embeds, `413` over the upload limit, `400` for an over-large text field). A share with no text, url or file saves nothing and still answers `303`, so the status cannot tell a saved share from an empty one |
| `GET /`, `/recent`, `/search`, `/tags`, `/todos`, `/topics`, `/health`, `/trash`, `/p/{slug}` | The PWA (`index.html`; `/recent` and the retired `/tags` open Search); its other files are served from the root too. `503` with build instructions if it was not built; see [](../how-to/use-the-pwa.md) |

## CLI

`scrap -v` (or `--version`) prints the version.

| Command | Does |
| --- | --- |
| `scrap serve [--host 127.0.0.1] [--port 8000]` or `scrap serve --uds PATH` | Run the service, migrating the database first. `--uds` listens only on a Unix socket (mode 600, in a folder of its own closed to others) and cannot be combined with `--host`/`--port` |
| `scrap backup DIR` | Consistent snapshot of the database (SQLite online backup) plus `files/`; safe while serving. `DIR` is created if missing, and a second run into it replaces the database and adds only new files: see {ref}`api-cli-folders` |
| `scrap restore DIR [--force]` | Restore a backup into an empty data folder, after checking its integrity and that its schema is not newer than this scrap; an older schema is upgraded after the copy. `DIR` may be read-only: only a copy of its database, made in the data folder, is opened. `--force` replaces the whole database (pages added since the backup are lost; files are kept); stop scrap first, or a running server goes on writing to the replaced database and those writes are lost |
| `scrap export DIR` | Every page not in the trash as `<slug>.md` (front matter with your data only, then your text) with its originals in `DIR/files/` and what Claude wrote about it in `DIR/_claude/<slug>.md`: the layout in [](export-format.md), the same files as `GET /api/store/download`'s zip. `DIR` opens as an Obsidian vault. Files are named by content, not as uploaded, and a second run into the same `DIR` deletes nothing: see {ref}`api-cli-folders` |
| `scrap import PATH [--dry-run] [--enrich] [--photos]` | Import Markdown pages from a folder or a zip of any size, as `POST /api/import` does (see [](../how-to/import-your-pages.md)): each new page tagged `import/YYYY-MM-DD`, a page here already skipped, nothing overwritten. `--dry-run` writes nothing and lists every page's problems and roughly what Claude would cost. `--enrich` queues the new text pages for Claude to tag and summarise (with `SCRAP_INGEST` on and `SCRAP_ENRICH` not off in its environment); `--photos` queues their photos for Claude to read. The service's worker runs what is queued. Symlinks in a folder are not followed; a zip with a name reaching outside it is refused |
| `scrap gc [--delete] [--dry-run] [--grace-hours H] [--run-days D] [--trash-days T] [--job-days J] [--report]` | Find the stored files nothing uses any more: no page's text (the trash's too), capture, Claude result or ingest job, or model run refers to them. It only counts them (and their size) unless `--delete`; `--dry-run` overrides `--delete`. Files younger than `H` hours (default 24) are kept, used or not, since an upload may be on its way. With `--delete` it also gathers each file of a capture that no page shows any more (its last embed or link was deleted more than `H` hours ago) onto the month's review page, a todo titled `Orphaned assets YYYY-MM`, due 7 days after it is made; it does not delete one then, only once the review page that shows it is purged from the trash (see {ref}`scrap-gc`). It also prunes `model_run` and `enrich_run` records older than `D` days (default 30, 1 to 36500) unless something refers to them (`enrichment`, `writeup`, `derived`, a later turn) or they are the latest for their page or photo, or still running; it counts them the same way. `topic_run` records are not pruned. It purges each page in the trash for more than `T` days (default 30, 1 to 36500; `off`: never), but a topic page (never purged: its key stops Claude making the topic again), with its revisions, captures, Claude's results, runs and jobs, tags, search index and write-ups; its files then go as unused ones do, in the same run. A purged *Orphaned assets* page lets go of the files it showed that no page shows: they are taken off their captures (with the photo's results) and deleted. It deletes `job` and `enrich_job` records done or dead and last changed more than `J` days ago (default 30, 1 to 36500), but each photo's latest and each page's latest of a reason. The purge and the pruning are committed in a transaction of their own before any file is looked at, so no file is deleted while they could still roll back. A dry run counts all of this, and the files a purge would free, and changes nothing. `--report` only prints the store's sizes (the database by table with `dbstat`, else estimated; the stored files in use, shown only on *Orphaned assets* pages in the trash (a purge lets them go), used only by pages in the trash, and unused; the trash by age and how many pages `T` would purge; the Claude call records by month; the job records by state and how many `J` would prune): it reads, and deletes nothing. Nothing is deleted if the references cannot be read. Only one `--delete` run at a time (a second stops, see below). Safe while scrap serves; see {ref}`scrap-gc` |
| `scrap enrich (--backfill \| --related \| --writeups \| --topics) [--all] [--outdated] [--limit N] [--dry-run]` | Say exactly one of `--backfill`, `--related`, `--writeups` and `--topics`. `--topics` is the weekly topics pass ({ref}`ingest-topics`): one discovery call now, in this process, over every page (new topics are made at once, at most 5, and sources added to existing ones; it prints what it made), then a synthesis queued for each topic due, those with none first, for the service's worker (`--all`: every topic; `--limit N`: at most `N` syntheses queued). With `--dry-run` the discovery call is still made (and paid for) and its proposals printed, but nothing is written or queued. It has `--related`'s conditions, stops and exit rules, and exits non-zero if the discovery call fails. `--writeups` writes Claude's write-up now, in this process, for every board page (a whiteboard or diagram image, not a list of actions) with none, oldest first (`--all`: every board page; `--outdated`: instead, each board page whose write-up is outdated, `is:outdated`'s test: a board image read, or its text, title, kind or tags changed, since it was made, or made with another prompt version: how a new write-up prompt is rolled out), under the same conditions and stops as `--related`; run `--related` after it, so related pages see the write-ups. `--outdated` goes only with `--writeups`. `--backfill` queues every page with no tags that Claude has not read yet, oldest first; the service's worker runs them within the daily budget. `--related` picks related pages now, in this process, for every page changed, or with a photo read, since its last picks (the nightly pass); `--all` takes every page instead. `--related` (and `--writeups`) needs `SCRAP_INGEST` on, `SCRAP_ENRICH` not off and `ANTHROPIC_API_KEY` (or, with `SCRAP_LLM_BACKEND=cli`, `CLAUDE_CODE_OAUTH_TOKEN`) set in its environment; without them it says so, does nothing and exits `0` (`--backfill` only queues, and needs none of them). It stops, exiting non-zero, at a queue pause or when the day's budget is spent. `--limit N` takes at most `N` pages; `--dry-run` only counts them. See [](../explanations/ingest.md) |
| `scrap db-init` | With Litestream, before the service starts: keep the database on the emptyDir, or take up a staged restore, restore the replica, bring an old `scrap.db` across once or make a new database, or refuse. The rules and the settings are in [ADR 46](../explanations/decisions/0046-database-on-an-emptydir-replicated-with-litestream.md) |
| `scrap db-hold [--clear]` | With Litestream: show the state file, which says why `scrap gc` only counts (an override used, a restore taken up); `--clear` once you have looked lets it delete again |
| `scrap schedule [--run JOB]` | With Litestream, in the pod's `jobs` container: run the jobs in `SCRAP_SCHEDULE` on their schedules, and stamp the database for the backup jobs' staleness check. `--run JOB` runs one now and exits with its status (75 if it is already running). An empty `timeZone` is UTC here. On a Litestream install `scrap gc --delete` also only counts past a safety limit (see ADR 46), and `--allow-large` goes ahead after a person has looked |

Every command but `db-init` and `schedule` (which read the environment) takes `--data-dir` (default
`$SCRAP_DATA_DIR`, else `./data`), which holds `scrap.db` and `files/` (originals by SHA-256), and
`--db-path` (default `$SCRAP_DB_PATH`), for a database elsewhere. `scrap backup DIR
--from-replica REPLICA [--max-replica-age-hours H]` backs up a restore of a Litestream replica
instead of the live database, refusing one whose stamp is older than `H` hours (default 3; 0: no
check), and `scrap restore DIR --stage` leaves the backup in the data folder for the next pod start
to take up in place of the replica ({ref}`restoring with Litestream <k8s-restore-litestream>`).

(api-cli-status)=
### Exit status and common errors

A command that works exits `0`. One that refuses prints a single line on stderr and exits `1`.
Arguments it cannot use (`scrap enrich` with none or more than one of `--backfill`, `--related`,
`--writeups` and `--topics`, or `--outdated` without `--writeups`, an unknown option) print the
usage and exit `2`. `scrap` alone prints the help and exits `0`.

`scrap serve`, `restore`, `import` and `enrich` (every mode) create the data folder and bring `scrap.db` up to
date. `backup`, `export`
and `gc` never create or upgrade anything: with no `scrap.db` they stop at once. That makes
a wrong `--data-dir` easy to miss. With no `--data-dir` and no `SCRAP_DATA_DIR` the data folder is
`./data` under the current directory, so run from the wrong place, `serve` and `enrich`
quietly start a new empty one, while `backup`, `export` and `gc` say `no database at
data/scrap.db`.

| Message | Cause |
| --- | --- |
| `no database at <data folder>/scrap.db` (`scrap gc: no database at ...` for `gc`) | `backup`, `export` or `gc` was pointed at a folder with no database: the wrong `--data-dir`, or the working directory is not the one you expected |
| `scrap gc: no files folder at <data folder>/files` | The database is there but the data folder has no `files/` |
| `scrap gc: the grace period must not be negative` | A negative `--grace-hours` |
| `scrap gc: the run retention must be at least 1 day` | A `--run-days` below 1 |
| `scrap gc: the trash retention must be at least 1 day`, `scrap gc: the job retention must be at least 1 day` | A `--trash-days` or `--job-days` below 1 |
| `scrap gc: the run retention must be at most 36500 days` (or `trash`, `job`) | A `--run-days`, `--trash-days` or `--job-days` over 36500 (a hundred years) |
| `scrap gc: cannot purge the trash or prune old records, so nothing was deleted: ...` | The database refused the purge (read-only, disk full); nothing changed |
| `scrap gc: cannot read what is in use, so nothing was deleted: ...` | The database could not be read, or it is newer than this scrap |
| `scrap gc: cannot put back a file left aside by an earlier run, so nothing was deleted: ...` | A file an earlier run set aside could not be restored (permissions or disk) |
| `scrap gc: another scrap gc is deleting (it holds <data folder>/gc.lock); ...` | Another `--delete` run is in progress, such as the nightly one; try again when it ends. Counting runs take no lock |
| `no backup at <DIR>` | `scrap restore`'s `DIR` has no `scrap.db` |
| `bad backup: ...` | The backup's database is not a database, fails its integrity check, or has a schema newer than this scrap. The data folder is left as it was |
| `<data folder> is not empty; stop scrap and use --force` | `scrap restore` into a data folder that has a database or any file; see `--force` above |
| `stopped: the queue is paused (<reason>)` | `scrap enrich --related`, `--writeups` or `--topics` met a pause or spent the day's budget (exit `1`) |

A setting scrap cannot use stops it with its own message: see {ref}`api-settings`.

(api-cli-folders)=
### Backup and export folders

`scrap backup DIR` makes `DIR` if it is missing, then replaces `DIR/scrap.db` with a fresh snapshot
(written as `scrap.db.part` and renamed, so a run that stops half way leaves the earlier database
whole) and copies `files/` (laid out as in the data folder, `files/<first two hex digits>/<sha256>`).
Files are only ever added: one already in `DIR/files/` is neither touched nor removed, and names that
start with `.` (uploads in flight) are skipped. The preview cache (`cache/previews/` in the data
folder) is not copied. So backing up again into the same `DIR` is incremental, and `DIR` goes on
holding files no page uses any more (after a `scrap gc`, say). Use a new `DIR` for each backup if
that matters; the single-server timer does, and keeps the newest `SCRAP_BACKUP_KEEP` of them (see
{ref}`scrap-backup-restore`).

`scrap export DIR` writes one `<slug>.md` per page at the top of `DIR`, Claude's notes on a page in
`DIR/_claude/<slug>.md`, and one `DIR/files/` for all the pages, in which a file is named `<sha256><extension>`, the extension being the usual one for its
content type (none if it has none). It is not the name it was uploaded under, and a file on several
pages is stored once. A photo's drawing is `<sha256>-<capture id>.excalidraw`, since a drawing
belongs to a capture. The page download names files differently (by their uploaded names: see
`GET /api/pages/{id or slug}/download`).

Run it again into the same `DIR` and every `.md` and drawing is written afresh, while a stored file
already there is left alone. Nothing is deleted: a page put in the trash since, and its `.md`, stay in
`DIR`. Export into a new folder for a clean copy.

(api-settings)=
## Settings

The settings are environment variables of the service, read at start. A setting that is on or off
(`SCRAP_INGEST`, `SCRAP_ENRICH`) is on for `on`, `1`, `true` or `yes`, in any case; anything else
is off. The variables below are scrap's own; those that install scrap (the image, where it
listens, the data and backup folders) are in [](../how-to/deploy-single-server.md), and the
chart's values in [](../how-to/deploy-kubernetes.md).

| Variable | Default | Meaning |
| --- | --- | --- |
| `SCRAP_DATA_DIR` | `./data` | The data folder. With its `.scrap-db-state.json` from Litestream and no `SCRAP_DB_PATH`, `scrap serve` and `scrap gc` refuse: {ref}`k8s-litestream` |
| `SCRAP_DB_PATH` | unset | The database, where it is not `scrap.db` in the data folder: the chart's `litestream.enabled` sets it ([ADR 46](../explanations/decisions/0046-database-on-an-emptydir-replicated-with-litestream.md)). Its lock, WAL and `scrap restore`'s copy go beside it; `files/` and everything else stay in the data folder. `--db-path` overrides it, and so does `--data-dir` on its own, which means the database in that folder |
| `SCRAP_LITESTREAM_REPLICA` | unset | Set by the chart with `litestream.enabled`: the Litestream replica's folder, which `scrap db-init` restores from. While set, a plain `scrap restore` is refused ({ref}`restoring with Litestream <k8s-restore-litestream>`) |
| `SCRAP_LITESTREAM_BIN` | `litestream` | The `litestream` program, for `scrap db-init` and `scrap backup --from-replica` |
| `SCRAP_DB_INIT_OVERRIDE` | unset | `on`: `scrap db-init` goes past its refusals: a missing or badly lagging replica, an install with history, a changed `scrap.db` in the data folder, a staged restore older than the replica. Using it makes `scrap gc` only count until `scrap db-hold --clear`. The chart's `litestream.init.override`: for after a person has looked |
| `SCRAP_DB_SIZE_LIMIT` | unset | Set by the chart from `litestream.sizeLimit`: `scrap db-init` refuses a database over half of it, and `scrap schedule` warns |
| `SCRAP_SCHEDULE` | `[]` | The jobs `scrap schedule` runs, as JSON; the chart writes it with `litestream.enabled` |
| `SCRAP_WEB_DIR` | the packaged PWA | Serve the PWA from this folder instead: a new build there shows without a restart |
| `SCRAP_PUBLIC_URL` | unset | The address the service is reached at, e.g. `https://host.tailnet.ts.net:8446` (several, comma separated). Every route but `/healthz` answers only requests whose `Host` is loopback or one of these, or (except on `/mcp`) an IP address, and a write only an `Origin` that is (DNS rebinding and cross-site protection; see [Host and Origin](#host-and-origin)), and the first makes the page URLs MCP tools return; without it they are paths (`/p/<slug>`). A Markdown link to `/p/<slug>` on any of them is a page link (links, backlinks, `is:orphan`, `has:dangling`), and the PWA opens it in place: the server lists their origins in the `index.html` it serves (`<meta name="scrap-origins">`). An installed app keeps that page offline, so it picks up a changed list on its next app update; the address it was opened at always works. An address without a scheme and host (`foo`) stops scrap at start with `SCRAP_PUBLIC_URL: 'foo' is not a URL`; see [](mcp.md) |
| `SCRAP_MCP_TOKEN` | unset | Bearer token(s) `/mcp` requires, comma separated. Unset (with `SCRAP_MCP_TOKEN_FILE` also unset): `/mcp` is open as before. Only `/mcp` is guarded; `/healthz`, the API and the PWA are not. A set but empty value stops scrap at start. See [](mcp.md) |
| `SCRAP_MCP_TOKEN_FILE` | unset | A file with one bearer token per line (blank lines and `#` comments skipped), for a mounted secret; its tokens add to `SCRAP_MCP_TOKEN`'s. A missing or token-less file stops scrap at start |
| `SCRAP_IDENTITY_HEADER` | `Tailscale-User-Login` | The request header recorded as the author of a change, else `tagged`. Only this header is read; set it to what your authenticating proxy sends (e.g. `X-Auth-Request-User`). Blank counts as unset, and so does a value of `claude` (any case), MCP's author |
| `SCRAP_OIDC_ISSUER` | unset | The login proxy's token check ([ADR 45](../explanations/decisions/0045-scrap-verifies-the-login-proxys-keycloak-token.md)): an OpenID Connect issuer URL (https, or http on loopback), e.g. `https://identity.diamond.ac.uk/realms/dls`. Set, every request but `/healthz` and `/mcp` (and the paths under it) needs an access token from it in `X-Auth-Request-Access-Token` (oauth2-proxy's `pass_access_token`, passed on by the ingress): signed RS256 or ES256 by a key from the issuer's discovery document, `typ` `Bearer` (not an ID token), `iss` this exactly, issued to `SCRAP_OIDC_CLIENT_ID` (`azp`, or in `aud`), in date (30 s leeway), and `preferred_username` (the Keycloak username; at DLS the fedid) in `SCRAP_ALLOWED_USERS`. Else `401` (JSON under `/api`, text elsewhere), or `503` if the issuer's keys cannot be fetched. The author of a change is then the token's username, and `SCRAP_IDENTITY_HEADER` is not read. Set without the two below, or without an `/mcp` bearer token (`SCRAP_MCP_TOKEN` or `SCRAP_MCP_TOKEN_FILE`), stops scrap at start |
| `SCRAP_OIDC_CLIENT_ID` | unset | With `SCRAP_OIDC_ISSUER`: the client a token must be issued to |
| `SCRAP_ALLOWED_USERS` | unset | With `SCRAP_OIDC_ISSUER`: the Keycloak usernames let in (a token's `preferred_username`; at DLS the fedids), comma separated and lower-cased when read. The token's username must be lower-case ASCII and match one exactly |
| `SCRAP_REVISION_WINDOW` | `300` | Seconds within which one author's changes to a page's kind and tags fold into one metadata revision, and one author's text edits from one source (Claude's over MCP) into one text revision; `0`: each change is one. A value that is not a number of at least 0 stops scrap at start |
| `SCRAP_MCP_CONTENT_EDIT` | `1` | `0` (or `false`, `no`, `off`): MCP does not offer `edit_page`, `edit_drawing`, `trash_pages` or `undelete`, and refuses `revert` with `to_rev`, so no MCP session can edit the text of a page it did not write; it can still add an image embed to one (`add_svg` and the `/mcp` upload) unless `SCRAP_MCP_UPLOAD=0` (see [](mcp.md)). Unset is on; any other value, empty too, stops scrap starting |
| `SCRAP_MCP_UPLOAD` | `1` | `0` (or `false`, `no`, `off`): MCP clients cannot add images or files: `add_svg` is not offered and `POST /mcp/pages/{key}/captures` is not served (see [](mcp.md), {ref}`mcp-uploads`). Unset is on; any other value, empty too, stops scrap starting |
| `SCRAP_MAX_UPLOAD_MB` | `50` | Largest request body (a capture's files together), enforced as it arrives: `413`. May be fractional; a value that is not a number stops scrap at start. `/mcp` has its own fixed cap of 2 MB |
| `SCRAP_TRASH_DAYS` | `30` | How long a page stays in the trash (1 to 36500 days, or `off`) before the nightly `scrap gc` purges it, as the server shows it (`GET /api/trash`, the PWA's Trash, the MCP `trash` tool and instructions). Set it to the `--trash-days` the gc runs with (the chart does, from `gc.trashDays`); `off` when no gc purges the trash, so no date is shown. A number below 1, or anything else, stops scrap at start. It does not purge anything itself |
| `SCRAP_INGEST` | `off` | `on` starts the worker that sends photos to Claude and reads PDFs' and text files' text for search, locally (both are queued either way: with it off, files wait unread, as photos do) |
| `ANTHROPIC_API_KEY` | | The key for the Claude API; keep it in the service user's mode-600 env file |
| `SCRAP_LLM_BACKEND` | `api` | `api`: the Messages API with `ANTHROPIC_API_KEY`. `cli`: Claude Code (`claude -p`) on a Claude seat, with `CLAUDE_CODE_OAUTH_TOKEN`; needs the `claude` binary (`Dockerfile.claude`). See [ADR 0021](../explanations/decisions/0021-ingest-on-a-claude-seat-through-claude-p.md) |
| `CLAUDE_CODE_OAUTH_TOKEN` | | With `SCRAP_LLM_BACKEND=cli`: the seat token from `claude setup-token`; mode-600 env file only, it can use the whole seat |
| `SCRAP_VISION_MODEL` | `claude-opus-5-5` | The model photos are sent to. Empty counts as unset. A name the API does not know is a `404`, which pauses the queue as `auth` until *Resume* (see {ref}`ingest-spend`). The effort of the photo call is fixed at `medium`; no setting changes it |
| `SCRAP_ENRICH` | `on` | With ingest on, Claude also tags, summarises and links text pages and fetches saved links; `off` keeps it to photos |
| `SCRAP_FILE_ENRICH` | `auto` | `auto`: with ingest and enrich on, a page made from a PDF or text file (its first capture a file capture, not an import) is titled, tagged, summarised and related once its files are read, sent the start of their text (at most 4,000 characters to enrich, 1,500 to related; never the file), on the text model, the shared budget and pause. `ask`: a file's text goes to Claude only on *Ask Claude*. Ask works either way; `SCRAP_ENRICH=off` stops both. Case-insensitive, empty counts as unset; any other value stops scrap at start when `SCRAP_INGEST` is on. See [ADR 38](../explanations/decisions/0038-file-text-is-sent-to-claude-on-capture-bounded.md) |
| `SCRAP_TEXT_MODEL`, `SCRAP_TEXT_EFFORT` | `claude-haiku-5-5`, `low` | The model and effort for text pages. Empty counts as unset. Both go to the API as given, with no check: a model it does not know pauses the queue as `auth`, and an effort it rejects is a `400`, class `content`, which fails every text job (each charged; the third failure of a job ends it, and *Ask Claude* retries) |
| `SCRAP_ENRICH_DEBOUNCE` | `60` | Seconds after a note's capture or save before Claude reads it (later saves move it). A value that is not a number means `60`; a negative one means `0` (no wait) |
| `SCRAP_BUDGET_DAY_USD` | `3` | Daily spend, from recorded usage, after which the queue pauses until UTC midnight. A number of dollars, fractions allowed; `0` pauses the queue at the first job it claims. A value that is not a number stops scrap at start when `SCRAP_INGEST` is on, and `scrap enrich --related` when it runs, with a Python `ValueError` traceback rather than a message; with ingest off the value is never read. With `cli`, the CLI's notional cost at API prices |
| `SCRAP_NOTIFY_URL` | unset | If set, the first pause of an episode POSTs one fixed line naming its reason here (an ntfy topic works), and so does `scrap gc --delete` when it starts a month's *Orphaned assets* page (where it has network: not the single server's gc container) |
| `SCRAP_PUBLISH_TO` | unset | Public pages: the URL of the `scrap-public` upload receiver that public pages are pushed to, e.g. `http://scrap-receiver.scrap-public.svc:8081` (the chart sets it when `public.enabled`). Unset, or without a token file: no page is published, and making a page private still works. Not an `http(s)` URL: scrap does not start. See [](public-site.md) and [](../how-to/publish-pages.md). |
| `SCRAP_PUBLISH_SITE` | unset | Public pages: the public site's base URL, e.g. `https://pages.example.org`, for the addresses scrap shows (`public_url`, a publish's `url`). Unset: publishing still works, with no addresses. See [](public-site.md). |
| `SCRAP_PUBLISH_TOKEN_FILE` | unset | Public pages: a file holding the upload receiver's bearer token (its content, stripped; at least 32 characters, no whitespace; the receiver's `RECEIVER_TOKEN_FILE`), read on each use, so a rotated Secret is picked up. scrap reads the token only from a file. The chart mounts `public.tokenSecret` and sets this. Set but unreadable or too short: scrap does not start. See [](public-site.md). |
| `TZ` | the host's; `UTC` in the image | The server's time zone, e.g. `Europe/London`. It decides what "today" is for the search filters `due:today`, `due:tomorrow` and `due:yesterday`. The image has the tz database, so the setting works there |