# Deploy on Kubernetes
Run scrap on a Kubernetes cluster (k3s, a home lab, a cloud cluster) with the Helm chart in
[`charts/scrap`](https://github.com/gilesknap/scrap/tree/main/charts/scrap). It runs the same
container image as the [single-server deployment](deploy-single-server.md): one pod, a persistent
volume for the data and, by default, a nightly backup job.
How the pieces fit (one pod; every CronJob runs beside it on the same node):
```mermaid
flowchart LR
phone["Phone and laptops (tailnet)"]:::user
cc["Claude Code"]:::claude
subgraph cluster ["Kubernetes cluster"]
ts["Tailscale operator proxy"]
ing["Ingress / Service"]
subgraph node ["One node"]
pod["scrap pod (1 replica)"]
data[("data PVC /data")]:::store
jobs["CronJobs: backup, gc, related, topics"]
bak[("backups PVC /backups")]:::store
end
end
api["api.anthropic.com"]:::claude
phone --> ts --> ing --> pod
cc -->|"/mcp"| ts
pod --- data
jobs --- data
jobs --> bak
pod -.->|"ingest on"| api
classDef store fill:#e8f1ff,stroke:#4a7bd0
classDef claude fill:#fff3e0,stroke:#e08a00
classDef user fill:#e9f7ef,stroke:#2e9d5b
```
```{warning}
scrap has **no login of its own**. Anyone who can reach its Service can read and change everything.
Expose it only on your tailnet (see {ref}`k8s-tailscale`) or behind an authenticating proxy
(see {ref}`k8s-ingress`), never directly to the internet. Inside the cluster, any pod that can
reach the Service has full access too, and can send its own `Tailscale-User-Login` header: turn
on the NetworkPolicy (below) if other workloads share the cluster.
```
## What you get
| Object | Notes |
| --- | --- |
| Deployment | Exactly **one replica**, strategy `Recreate`. scrap keeps everything in one SQLite database with a single writer, so the chart has no `replicas` value: the old pod always stops before the new one starts. |
| PersistentVolumeClaim | `/data`: the database, files and the preview cache. Kept on `helm uninstall`. With `litestream.enabled` the database is on an emptyDir instead: {ref}`k8s-litestream`. |
| Service | `ClusterIP`, port 80. |
| ServiceAccount | No API token mounted; scrap never talks to the Kubernetes API. |
| CronJobs + PVC (on by default) | Nightly `scrap backup` to a second claim, keeping the newest 7 runs, and a monthly copy in its `monthly/` folder, keeping 12. |
| CronJobs (gc on by default, the Claude ones with ingest) | gc, related pages and topics: see {ref}`k8s-gc` and [](../reference/scheduled-jobs.md). |
| Ingress, NetworkPolicy (off by default) | See below. |
The pod meets the `restricted` [Pod Security Standard](https://kubernetes.io/docs/concepts/security/pod-security-standards/):
it runs as uid/gid 10001, with a read-only root filesystem (only `/data` and an `emptyDir` at
`/tmp` are writable), no capabilities, no privilege escalation and the `RuntimeDefault` seccomp
profile.
## Prerequisites
- Kubernetes 1.27 or newer, and Helm 3 or 4.
- A StorageClass that provides `ReadWriteOnce` volumes on a **local or block** filesystem (see
{ref}`k8s-storage`).
- A way in that is not the open internet: the
[Tailscale Kubernetes operator](https://tailscale.com/kb/1236/kubernetes-operator) is the
recommended one.
- For ingest: a Claude API key.
## Get the image
Releases publish a public, multi-arch (amd64 and arm64) image to `ghcr.io/gilesknap/scrap`, and
the chart's `appVersion` names the release it was made with (set from the git tag when the chart is
published; a checkout has the placeholder `0.0.0`, so set `image.tag` with one). By default the chart
runs that release, `ghcr.io/gilesknap/scrap:`, so there is nothing to build: set
`image.tag` only to run a different release.
To run your own build instead, push it to a registry your cluster can pull from, and point the
chart at it:
```
podman build --platform linux/amd64,linux/arm64 --manifest registry.example.com/scrap:dev .
podman login registry.example.com
podman manifest push registry.example.com/scrap:dev
```
```yaml
image:
repository: registry.example.com/scrap
tag: dev
pullPolicy: Always
imagePullSecrets:
- name: my-registry-credentials # if the registry needs a login
```
Building the arm64 half on an x86 machine needs `qemu-user-static`; on a mixed cluster you can
instead pin scrap to nodes of one architecture with `nodeSelector: {kubernetes.io/arch: amd64}`
and build only that.
## Install
A minimal `my-values.yaml` for the recommended setup (the Tailscale operator for access, ingest
on). It runs the release image the chart names, and assumes the Secret from {ref}`k8s-ingest`;
add an `image` block as above to run your own build, and drop `ingest` to run without Claude:
```yaml
ingest:
enabled: true
existingSecret: scrap-secrets
ingress:
enabled: true
className: tailscale
tls:
- hosts: [scrap]
```
From the released chart (version = the release tag, with a pre-release `0.4.0rc1` as `0.4.0-rc.1`):
```
kubectl create namespace scrap
helm install scrap oci://ghcr.io/gilesknap/charts/scrap --version --namespace scrap -f my-values.yaml
kubectl -n scrap rollout status deploy/scrap
```
The commands in this guide assume the release is called `scrap` in the namespace `scrap`. For
another name, use the chart's fullname instead (`kubectl -n scrap get deploy,cronjob,pvc` lists
the names, and `helm install` prints them in its notes).
Check it without exposing it anywhere:
```
kubectl -n scrap port-forward svc/scrap 8000:80
curl http://127.0.0.1:8000/healthz
```
and open for the PWA. The PWA needs HTTPS to install on a phone (see
[](use-the-pwa.md)), so use one of the exposures below for that.
## Values
Every value is described in
[`values.yaml`](https://github.com/gilesknap/scrap/blob/main/charts/scrap/values.yaml); unknown
top-level keys, and unknown keys under `image`, `ingest`, `persistence`, `ingress`,
`mcp`, `gc`, `enrichRelated`, `enrichTopics` and `public` (and under `backup`, including `backup.monthly`)
are refused, so a typo there fails the install instead of being ignored. Other groups (`service`,
`networkPolicy`, `serviceAccount`, `resources` and the security contexts) are not checked, so a typo
inside them is silently ignored. The main ones:
| Value | Default | Meaning |
| --- | --- | --- |
| `image.repository`, `image.tag` | `ghcr.io/gilesknap/scrap`, the chart's `appVersion` | Image to run. |
| `maxUploadMB` | `50` | `SCRAP_MAX_UPLOAD_MB`: largest capture. Raise a proxy's body limit to match. |
| `ingest.enabled` | `false` | `SCRAP_INGEST`: send photos to Claude (see {ref}`k8s-ingest`). |
| `ingest.visionModel`, `ingest.budgetDayUSD` | app default, `3` | `SCRAP_VISION_MODEL`, `SCRAP_BUDGET_DAY_USD` (US dollars per UTC day). Like `ingest.notifyUrlKey`, they take effect only while `ingest.enabled` is true. |
| `persistence.size`, `.storageClass`, `.existingClaim` | `5Gi`, default class, none | The data volume. |
| `litestream.enabled` | `false` | The database on an emptyDir, replicated to the data volume, for a data volume on NFS: {ref}`k8s-litestream`. Under the restricted or baseline Pod Security Standard a `backup.persistence.hostPath` then also stops the scrap pod starting, as it mounts the backups. |
| `backup.enabled`, `.schedule`, `.timeZone`, `.keep` | `true`, `30 3 * * *`, the cluster's default (usually UTC), `7` | The backup CronJob. |
| `gc.enabled`, `.schedule`, `.timeZone` | unset (follows `backup.enabled`), `30 4 * * *`, the cluster's default | The `scrap gc` CronJob (see {ref}`k8s-gc`). |
| `gc.delete`, `gc.graceHours`, `gc.runDays`, `gc.requireBackup` | `true`, `24`, `30`, `true` | `delete: false`, or `graceHours: 0`, makes the job only count (a dry run in its log). Keep files younger than this many hours; keep model and enrich run records younger than this many days (at least 1); delete only after a backup less than 26 hours old. |
| `enrichRelated.enabled`, `.schedule`, `.timeZone` | `true`, `41 3 * * *`, the cluster's default | The related-pages CronJob. It exists only while `ingest.enabled` is true. Every scheduled job: [](../reference/scheduled-jobs.md). |
| `enrichTopics.enabled`, `.schedule`, `.timeZone` | `true`, `11 4 * * 0`, the cluster's default | The topics CronJob (Sundays). It exists only while `ingest.enabled` is true. |
| `backup.monthly.enabled`, `.schedule`, `.timeZone`, `.keep` | `true`, `50 3 1 * *`, the cluster's default, `12` | The monthly copy CronJob (needs `backup.enabled`; see Backup and restore). |
| `backup.persistence.size`, `.storageClass`, `.existingClaim` | `20Gi`, default class, none | The backups volume. Each run is a full copy: allow about `keep + monthly.keep + 1` times the data (or lower `keep` and `monthly.keep`). |
| `backup.persistence.hostPath` | none | An absolute directory on every node (a shared filesystem) to keep the backups in, instead of a claim. It must exist and be writable by the `podSecurityContext` user. A namespace that enforces the `baseline` or `restricted` Pod Security Standard rejects hostPath volumes, so the backup jobs never start there: use the claim. |
| `resources.limits.memory` | `1Gi` | scrap's photo decoding is budgeted for a 1 GB container; a lower limit risks the pod being OOM-killed on large photos. |
| `ingress.*`, `service.annotations` | off | Exposure, below. |
| `service.type`, `service.port` | `ClusterIP`, `80` | The Service. |
| `networkPolicy.enabled`, `.from` | off | Who may connect, below. |
| `mcp.tokenSecret.name`, `.key` | none, `SCRAP_MCP_TOKEN` | An existing Secret and the key in it holding the bearer token(s) `/mcp` requires (`SCRAP_MCP_TOKEN`, comma separated for several). Never a token as a value; the chart creates no Secret. Empty name: no token needed. See {ref}`k8s-mcp-token`. |
| `identityHeader` | `Tailscale-User-Login` | `SCRAP_IDENTITY_HEADER`: the header recorded as the author, e.g. `X-Auth-Request-User` behind oauth2-proxy. Not read with `authProxy.enabled`, where the author is the token's username. |
| `authProxy.enabled`, `.service`, `.port`, `.allowedUsers` | off, none, `80`, none | A login for the web app through an oauth2-proxy you run, with ingress-nginx only ({ref}`k8s-auth-proxy`). |
| `authProxy.oidc.issuer`, `.clientId` | none | Required with `authProxy.enabled`: the identity provider's issuer URL (https) and the proxy's client ID, whose access tokens scrap checks on every request (`SCRAP_OIDC_ISSUER`, `SCRAP_OIDC_CLIENT_ID`; {ref}`k8s-auth-proxy`). |
| `extraEnv`, `extraEnvFrom` | none | Anything else for the container, such as `SCRAP_PUBLIC_URL` (see {ref}`k8s-mcp`) or `SCRAP_ENRICH`. |
| `public.enabled`, `.site`, `.tokenSecret.*`, `.ingress.*`, `.networkPolicy.webFrom` | off | Public pages: the public site and its upload receiver in their own namespace ({ref}`k8s-public-pages`). |
| `public.namespace` | `scrap-public` | The public side's namespace. It may be the release namespace; then one Secret serves both sides. |
| `public.networkPolicy.enabled` | `true` | `false`: no NetworkPolicy for the public side, where you may not create one. The receiver still needs its token. |
| `nodeSelector`, `tolerations`, `affinity`, `podAnnotations` | none | Scheduling and annotations. |
| `nameOverride`, `fullnameOverride`, `serviceAccount.*`, `terminationGracePeriodSeconds`, `backup.resources` | see `values.yaml` | Naming, the ServiceAccount, the stop timeout (`30`) and the backup job's resources. |
Leave `service.type` at `ClusterIP`: the chart assumes access goes through the Ingress or a
port-forward. `LoadBalancer` and `NodePort` open the Service to whatever network the node or load
balancer is on, and scrap has no login, so use them only on a private network. With `ingress.hosts`
empty, the Ingress has one rule that matches any host (the Tailscale example below relies on
this); with hosts set, each gets its own host-specific rule. The
[`charts/scrap/ci`](https://github.com/gilesknap/scrap/tree/main/charts/scrap/ci) directory holds
the value sets the chart is tested with (defaults, no backup, existing claims, a generic ingress,
ingest, long names, and the Tailscale Ingress and Service), which are worked examples to copy
from.
The settings themselves are in the [API reference](../reference/api.md). The chart sets
`SCRAP_DATA_DIR=/data`; `SCRAP_WEB_DIR` (for trying out a PWA build) has no value of its own,
use `extraEnv`.
Set `TZ` the same way, to your own time zone: the server decides what "today" means for
`due:today` searches from its own clock (UTC by default), while the PWA uses the phone's date.
```yaml
extraEnv:
- name: TZ
value: Europe/London
```
The preview cache (`cache/previews` under `/data`) stays on the data volume: it is regenerable and
never backed up, and keeping it saves redoing every preview after a restart, which is slow on
small ARM nodes.
(k8s-ingest)=
## Secrets and ingest
The chart never takes the API key as a value and never creates a Secret: it only reads a Secret
you create. Create it from a file or stdin so the key stays out of your shell history and out of
any values file:
```
read -rs KEY && printf %s "$KEY" | kubectl -n scrap create secret generic scrap-secrets \
--from-file=ANTHROPIC_API_KEY=/dev/stdin; unset KEY
```
Optionally add the notification URL (an ntfy topic URL is itself a secret) to the same Secret
under another key, e.g. `SCRAP_NOTIFY_URL`. Then:
```yaml
ingest:
enabled: true
existingSecret: scrap-secrets
apiKeyKey: ANTHROPIC_API_KEY # the default
notifyUrlKey: SCRAP_NOTIFY_URL # optional
```
With ingest on, Claude also tags, summarises and links text pages and fetches the links you save
(`SCRAP_ENRICH`, on by default). That shares the daily budget with photos (`ingest.budgetDayUSD`,
$3 per UTC day by default), and the pod then makes outbound HTTP and HTTPS requests (ports 80 and
443) to public sites. To keep Claude to photos, switch it off:
```yaml
extraEnv:
- name: SCRAP_ENRICH
value: "off"
```
With enrich on, a page made from a PDF or text file also has the start of its text (at most
4,000 characters, never the file) sent to Claude once it is read, for a title, tags and a summary
([ADR 38](../explanations/decisions/0038-file-text-is-sent-to-claude-on-capture-bounded.md)). To
keep file text on the host unless you press **Ask Claude**, set `SCRAP_FILE_ENRICH` to `ask`:
```yaml
extraEnv:
- name: SCRAP_FILE_ENRICH
value: "ask"
```
With GitOps, create the Secret with whatever your cluster uses for secrets (Sealed Secrets,
External Secrets, SOPS) under the same name; never commit the key. A changed Secret is not picked
up by a running pod: `kubectl -n scrap rollout restart deploy/scrap`.
## Exposure
(k8s-tailscale)=
### Tailscale Kubernetes operator (recommended)
With the [Tailscale Kubernetes operator](https://tailscale.com/kb/1236/kubernetes-operator)
installed, an Ingress of class `tailscale` publishes scrap over HTTPS on your tailnet only, with a
MagicDNS name and a Let's Encrypt certificate
([cluster ingress](https://tailscale.com/kb/1439/kubernetes-operator-cluster-ingress),
[L7 ingress how-to](https://tailscale.com/docs/kubernetes-operator/ingress/expose-workload-to-tailnet-l7)).
Your tailnet needs MagicDNS and [HTTPS certificates](https://tailscale.com/kb/1153/enabling-https)
enabled.
```yaml
ingress:
enabled: true
className: tailscale
tls:
- hosts:
- scrap # becomes https://scrap..ts.net
```
The operator's proxy accepts traffic through Tailscale Serve
([architecture](https://tailscale.com/docs/kubernetes-operator/concepts/architecture)). Never add
the Funnel annotation: that publishes scrap to the internet.
The alternative, the Service annotation `tailscale.com/expose: "true"` (with
`tailscale.com/hostname`), forwards plain TCP: there is no HTTPS, so the PWA cannot install or
work offline, and there are no identity headers. Use the Ingress.
**Identity headers.** scrap records the `Tailscale-User-Login` header as the author of each change.
Tailscale Serve sets it, and strips any value the client sent, for requests from user-owned
devices, and sets none for tagged devices. The operator's documentation does not say whether its
Ingress proxy does the same. Its proxy configures Serve through the same local API, and in
Tailscale's source the Serve reverse proxy adds the headers to every proxied request
([`ipn/ipnlocal/serve.go`](https://github.com/tailscale/tailscale/blob/main/ipn/ipnlocal/serve.go),
`addTailscaleIdentityHeaders`), so they are expected to arrive, but this is **not yet verified**
end to end. To check on your cluster: make a change from a phone or laptop through the Ingress,
then look at the page's history (or `GET /api/pages/`): your tailnet login should be its
author, not the fixed fallback author `tagged`. If it is not, every change is recorded as
`tagged`; nothing else breaks.
The chart leaves `networkPolicy` off and `/mcp` without a token by default; see
[](../explanations/architecture/mcp-and-access.md) for the values to set on a shared cluster.
Because a pod inside the cluster can send its own `Tailscale-User-Login` header straight to the
Service, let only the operator's proxies in (they run in the operator's namespace, `tailscale` by
default):
```yaml
networkPolicy:
enabled: true
from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: tailscale
```
A NetworkPolicy needs a CNI that enforces it (k3s's built-in controller does). Check that the
policy exists (`kubectl -n scrap get networkpolicy`): a misspelt key inside `networkPolicy` is
not rejected. Whether kubelet probes still get through depends on the CNI; if the pod never
becomes ready after turning the policy on, check that first.
#### Tailnet ACL: deny tagged devices by default
The operator's proxies are tagged devices (`tag:k8s` by default). As for the single server, use a
tailnet policy that lets your own devices reach them and gives other tagged devices (servers, CI,
anything unattended) no access unless you grant it:
```json
"grants": [
{"src": ["autogroup:member"], "dst": ["tag:k8s"], "ip": ["tcp:443"]}
]
```
(k8s-ingress)=
### Any other ingress controller
scrap has no login, so put an authenticating proxy in front: for example
[oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/) through your controller's external
auth annotations, or your SSO's forward-auth middleware. Raise the controller's request body
limit a little above `maxUploadMB`, so an upload just over it gets scrap's error rather
than the controller's.
```yaml
ingress:
enabled: true
className:
annotations:
# Your controller's annotations for external authentication and for a
# request body limit a little above maxUploadMB; see its documentation.
: : 51m
hosts:
- scrap.example.com
tls:
- hosts:
- scrap.example.com
secretName: scrap-tls
```
The annotation names differ for every controller (Traefik uses Middleware objects instead), so
the chart takes them as plain `ingress.annotations`. Make sure the proxy removes any
`Tailscale-User-*` headers sent by clients; changes are then recorded under the fixed fallback
author `tagged`. Never expose the Service without authentication.
The Ingress sends every path to scrap, so the proxy covers `/mcp` and `/healthz` too. `/mcp` is
called by Claude Code, not a browser, so a login page in front of it will block it unless the
proxy exempts that path or the client can pass its check; the tailnet route avoids the problem.
**Behind oauth2-proxy.** Set `identityHeader: X-Auth-Request-User` (the header oauth2-proxy
returns and the controller copies to the request), so the author is the real login. scrap trusts
only that header, but most controllers do not strip a client-sent copy, so the proxy plus the
NetworkPolicy must be the only way to the Service; a forged value only changes the recorded author.
**Exposing `/mcp` (Cloudflare Access, a second ingress).** For a complete walk-through with a
Cloudflare Tunnel straight to the Service, see [](../tutorials/install-kubernetes.md). With an
ingress controller, give `/mcp` its own Ingress rule or
host that skips the login proxy, and guard it another way: a Cloudflare Access service token in
front, and scrap's own bearer token behind it (below), so a leaked service token alone does not
open scrap. Keep the PWA and API on the login-proxied host.
(k8s-auth-proxy)=
### A central oauth2-proxy with ingress-nginx
If you run oauth2-proxy in the cluster (with `reverse_proxy` and `set_xauthrequest` on, and no
fixed `redirect_url`), the chart wires it in:
```yaml
authProxy:
enabled: true
service: oauth2-proxy.scrap.svc.cluster.local # the proxy's in-cluster DNS name
allowedUsers: [abc12345] # who may use the web app (usernames)
oidc:
issuer: https://identity.example.org/realms/main # the realm, as in its tokens' iss
clientId: scrap # the proxy's client
```
The Ingress then gets ingress-nginx's `auth-url` (the proxy's `/oauth2/auth`, with no
`allowed_emails` query, which some ingress-nginx annotation validation refuses), `auth-signin` and `auth-response-headers: X-Auth-Request-Email,X-Auth-Request-Access-Token`
annotations, on top of `ingress.annotations`. A second Ingress, `-open`, with the same class, hosts and
`tls` and no auth, sends `/mcp` to scrap (set `mcp.tokenSecret`) and `/oauth2` to the proxy
through an `ExternalName` Service, so the sign-in, its callback
(`https:///oauth2/callback`, which the identity provider must allow) and the cookie are
on scrap's own host. Both Ingresses get `proxy-buffer-size: 16k` unless `ingress.annotations`
sets it, since the login's cookies overflow nginx's 4k default. The author of a change is
the username (`preferred_username`) in the proxy's token, so `identityHeader` is not read. The install fails without `ingress.enabled`, `ingress.tls` (the cookie is
https only), `mcp.tokenSecret.name`, `authProxy.service`, `authProxy.allowedUsers`,
`authProxy.oidc.issuer` or `authProxy.oidc.clientId`.
The login alone guards the Ingress only: without a NetworkPolicy, any pod can reach the Service
and send its own `X-Auth-Request-Email`. So scrap checks the proxy's access token itself
([ADR 45](../explanations/decisions/0045-scrap-verifies-the-login-proxys-keycloak-token.md)): the
Deployment gets `SCRAP_OIDC_ISSUER`, `SCRAP_OIDC_CLIENT_ID` and `SCRAP_ALLOWED_USERS`, and every
request but `/healthz` and `/mcp` needs a valid token for an allowed username (`401` without).
This is also the only owner check: anyone the proxy lets log in passes the Ingress. Run
the proxy with `pass_access_token = true`, and a `cookie_refresh` shorter than the access tokens'
life. Other controllers
need their own annotations, above. DLS runs one proxy for every user this way:
[](run-the-dls-login-proxy.md).
(k8s-mcp-token)=
### Bearer token on /mcp
Create the Secret yourself (the chart never takes a token as a value) and name it:
```yaml
mcp:
tokenSecret:
name: scrap-secrets # may be the Secret that holds ANTHROPIC_API_KEY
key: SCRAP_MCP_TOKEN # one token, or several comma separated to rotate
```
Generate one with `python -c 'import secrets; print(secrets.token_urlsafe(32))'`. To rotate, set
the key to `old,new`, roll the pod, move the clients to `new`, then drop `old`. `/healthz` stays
open for the probes. Clients send `Authorization: Bearer `: see
[](connect-claude-code.md).
(k8s-mcp)=
### Claude Code (MCP)
Claude Code sessions talk to scrap's `/mcp` endpoint (see [](connect-claude-code.md) and
[](../reference/mcp.md)). It answers `421` to any `Host` that is not loopback or listed in
`SCRAP_PUBLIC_URL`, and the chart has no value for it, so set it through `extraEnv` to the address
you reach scrap at:
```yaml
extraEnv:
- name: SCRAP_PUBLIC_URL
value: https://scrap..ts.net
```
The same address makes the page links the MCP tools return absolute.
```{warning}
The web app and API check `Host` as well: a hostname missing from `SCRAP_PUBLIC_URL` gets `421`
on every page, not just `/mcp`. Set it to the address you reach scrap at (`/healthz`, which the
probes use, answers to any name), and roll the pod after changing it.
```
### Port-forward
For a quick try, `kubectl -n scrap port-forward svc/scrap 8000:80`. Nothing is exposed beyond your own machine.
(k8s-public-pages)=
## Public pages
scrap can publish the pages you mark Public (issue 497) to a static site on the internet, for
example `https://pages.example.org`. The site is served by nginx from a separate image,
`ghcr.io/gilesknap/scrap-public`, which also holds a small upload receiver. scrap is to push each
public page to the receiver; nothing on the public side can reach scrap. scrap's publisher
arrives with the scrap side of issue 497: until then this section runs the public side only,
and the site stays empty. The contract between the two sides (the receiver's API, what each
unit may hold, nginx's paths and headers) is [](../reference/public-site.md), and why it is built
this way is [ADR 43](../explanations/decisions/0043-public-pages-a-static-site-scrap-pushes-to.md). The chart's `public`
section, off by default, runs it:
- it runs in a namespace of its own (`public.namespace`, default `scrap-public`), labelled Pod
Security `restricted`, which you create first (below);
- one Deployment with two containers sharing the site volume: `web` (nginx, the site mounted read
only) and `receiver` (the only one given the token, mounted as a file), both non-root with
read-only root filesystems;
- two ClusterIP Services, and an Ingress for nginx alone (`public.ingress`); the receiver is never
on an Ingress;
- a NetworkPolicy: the receiver's port only from scrap's pods (the release namespace and scrap's
labels), nginx's port from `public.networkPolicy.webFrom` (empty: anyone, as nginx serves only
public files; set it to your ingress controller's namespace to narrow it), and no egress at all.
The site lives in an `emptyDir` (`public.sizeLimit`, default `3Gi`: the receiver's 2 GiB quota
plus room for an upload). It starts empty when the pod moves. With the scrap side of issue 497,
scrap fills it again: it checks the receiver's manifest at start and every 10 minutes and pushes
what is missing.
First create the public namespace, labelled Pod Security `restricted`, and the upload token. The
token is a Secret you create, never a value. The receiver runs in the public namespace and scrap
in its own, and a pod reads only its own namespace's Secrets, so create it, with the same name
and key, in **both**:
```sh
kubectl create namespace scrap-public
kubectl label namespace scrap-public pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/audit=restricted pod-security.kubernetes.io/warn=restricted
token=$(openssl rand -hex 32)
for ns in scrap scrap-public; do
kubectl -n "$ns" create secret generic scrap-publish --from-literal=token="$token"
done
```
Both pods read the token as a mounted file, never an environment variable: the receiver through
`RECEIVER_TOKEN_FILE`, scrap through `SCRAP_PUBLISH_TOKEN_FILE`.
`public.createNamespace: true` makes the chart create the namespace instead, with the same
labels. Argo CD's `CreateNamespace=true` does not do it: that creates only the Application's
destination namespace (scrap's), so `scrap-public` must exist first or come from
`public.createNamespace`. With the chart creating it, the Secret can only be made after the first
install, and the receiver waits for it. **`helm uninstall` (or an Argo CD prune)
then deletes the namespace, and the Secret in it**, which is why it is off by default.
Then:
```yaml
public:
enabled: true
site: https://pages.example.org # SCRAP_PUBLISH_SITE
tokenSecret:
name: scrap-publish
key: token
ingress:
className: nginx # or your controller's
hosts: [pages.example.org]
tls:
- hosts: [pages.example.org]
secretName: pages-tls
```
The install fails if `public.enabled` is set without `public.tokenSecret.name` or `public.site`.
scrap's pod then gets `SCRAP_PUBLISH_TO` (the receiver's Service,
`http://-receiver..svc:8081`, `scrap-receiver.scrap-public` for a release named `scrap`), `SCRAP_PUBLISH_SITE` and
`SCRAP_PUBLISH_TOKEN_FILE` (see the [settings](../reference/api.md)). The site is `noindex`: every
answer has `X-Robots-Tag: noindex, nofollow` and `robots.txt` disallows all, unless
`public.indexing` is `true`.
**Behind Cloudflare Access**: if your domain has a wildcard Access application
(`*.example.org`), it covers every subdomain, the public site's too, and visitors meet a login.
Give the site's host (`pages.example.org`) its own Access application with a policy whose Action
is **Bypass**, so anyone may reach it. A more specific application takes precedence over the
wildcard.
**Cloudflare's browser cache**: a zone's Browser Cache TTL (4 hours by default) rewrites the
site's `Cache-Control: no-cache` on `/assets/*.css` and `*.js` to `max-age=14400`, so readers
see last release's stylesheet (#534). Add a Cache Rule for the site's host with **Browser TTL:
Respect origin TTL**, then check: `curl -sI https://pages.example.org/assets/page.css` shows
`cache-control: no-cache`.
## Upgrades
Back up first, then upgrade to the newer chart version (its image follows, unless your values set
`image.tag`: change that too).
```
kubectl -n scrap create job --from=cronjob/scrap-backup scrap-backup-pre-upgrade
kubectl -n scrap wait --for=condition=complete job/scrap-backup-pre-upgrade --timeout=30m
helm upgrade scrap oci://ghcr.io/gilesknap/charts/scrap --version -n scrap -f my-values.yaml
```
From a checkout, `helm upgrade scrap ./charts/scrap -n scrap -f my-values.yaml --set
image.tag=` instead.
The old pod stops before the new one starts (`Recreate`), so there is a short outage; the PWA
queues captures meanwhile. A clean stop gives any ingest job in flight back to the queue. scrap
runs its database migrations at startup, before it answers `/healthz`: the startup probe allows
them five minutes.
Migrations are **forward only**, and an older scrap refuses a database migrated by a newer one.
So `helm rollback` alone does not go back: roll back to the old image *and* restore the
pre-upgrade backup with `--force` (below). Anything captured since the upgrade is then lost from
the database; its files stay.
## Backup and restore
The daily CronJob runs `scrap backup` into a new timestamped directory on the backup claim
(`/backups/20261019T033000Z`), then keeps only the newest `backup.keep` runs, using the same
rules as the single-server deployment: `keep` must be a whole number of at least 1 (the chart
refuses anything else), and only real directories named like a run are ever deleted; anything
else on the volume, including symlinks, is left alone. A failed run removes only its own
directory. `scrap backup` is safe while scrap is serving. The job runs on the same node as the
scrap pod (pod affinity), because the data volume is `ReadWriteOnce`; while scrap is scaled to
zero, a run waits and gives up after an hour. Runs never overlap, and one that cannot start
within an hour of its schedule is skipped. A failed run is retried once and then counted as
failed; the last 3 successful and the last 3 failed Jobs are kept, so `kubectl logs` still works
on them.
**Monthly copy.** A second CronJob, `-backup-monthly` (03:50 on the 1st), makes the same
full backup into the `monthly/` folder of the backup claim and keeps the newest `backup.monthly.keep`
(12) there. It takes the same lock as the daily backup and gc, so they never overlap. The daily
prune, gc's recent-backup check and the restore commands below look only at runs directly in the
claim, so none of them touches `monthly/`; to restore one, point `restore` at
`/backups/monthly/`. Switch it off with `backup.monthly.enabled: false`.
```mermaid
flowchart LR
d["daily CronJob 03:30"] -->|"keep 7"| r[("/backups/ YYYYMMDDTHHMMSSZ")]:::store
m["monthly CronJob 03:50 on the 1st"] -->|"keep 12"| mm[("/backups/monthly/ YYYYMMDDTHHMMSSZ")]:::store
g["gc CronJob 04:30"] -.->|"needs a COMPLETE run under 26 h old"| r
classDef store fill:#e8f1ff,stroke:#4a7bd0
```
The daily CronJob is called `-backup`, which is `scrap-backup` for the release used in this
guide. When a CronJob's name would be longer than 52 characters, the chart shortens the fullname and
adds a 6-character hash, so the name differs: read it from `kubectl -n scrap get cronjob` and use it in place of
`scrap-backup` in the commands below and under Upgrades.
Backups are on by default because the data is personal and irreplaceable; set
`backup.enabled: false` if you back up the data volume another way (volume snapshots, Longhorn
or Velero backups). Either way the backup claim lives in the same cluster: copy it somewhere
else too.
Run one now and follow it:
```
kubectl -n scrap create job --from=cronjob/scrap-backup scrap-backup-now
kubectl -n scrap logs -f job/scrap-backup-now
```
To restore, stop scrap (and anything that would restart it), then run `scrap restore` in a
one-off pod with both claims mounted:
```
kubectl -n scrap patch cronjob scrap-backup -p '{"spec":{"suspend":true}}'
kubectl -n scrap scale deploy/scrap --replicas=0
kubectl -n scrap wait --for=delete pod -l app.kubernetes.io/name=scrap --timeout=2m
kubectl -n scrap apply -f restore.yaml
kubectl -n scrap wait --for=jsonpath='{.status.phase}'=Succeeded pod/scrap-restore --timeout=10m
kubectl -n scrap delete pod scrap-restore
kubectl -n scrap scale deploy/scrap --replicas=1
kubectl -n scrap patch cronjob scrap-backup -p '{"spec":{"suspend":false}}'
```
With Argo CD or another GitOps controller, pause its automatic sync first, or it scales scrap
straight back up. `restore.yaml`, with the run to restore, the image of the scrap version you
want (the chart's `image.repository` and tag of the release that wrote the backup, or a newer
one: restore refuses a backup from a newer scrap, see {ref}`k8s-restore-schema`) and, for a non-empty data volume, `--force` (which replaces the whole database; see
`scrap restore --help`):
```yaml
apiVersion: v1
kind: Pod
metadata:
name: scrap-restore
spec:
restartPolicy: Never
automountServiceAccountToken: false
# imagePullSecrets: [{name: my-registry-credentials}] # if the registry needs a login
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: restore
image: ghcr.io/gilesknap/scrap:
args: ["restore", "/backups/", "--force"]
env:
- name: SCRAP_DATA_DIR
value: /data
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
volumeMounts:
- {name: data, mountPath: /data}
- {name: backups, mountPath: /backups, readOnly: true}
- {name: tmp, mountPath: /tmp}
volumes:
- name: data
persistentVolumeClaim:
claimName: scrap-data
- name: backups
persistentVolumeClaim:
claimName: scrap-backups
- name: tmp
emptyDir: {}
```
The claim names are `-data` and `-backups`, where the fullname is the
release name if it contains `scrap`, else `-scrap`: `kubectl -n scrap get pvc` shows
them. `scrap restore` prints nothing when it works and exits non-zero with a message when it does
not (`kubectl -n scrap logs pod/scrap-restore`; the `wait` above then times out instead of
returning). It works on a copy and never writes to the
backup, so the backups claim can be mounted `readOnly: true`, as above. The same pod with other
arguments runs any `scrap` command against the data, for example `["export", "/backups/export"]`
(which does need the claim writable; pruning only ever deletes directories named like a backup
run, so an export there is kept). To restore into a fresh volume instead, create a new claim,
restore into it without `--force`, and point `persistence.existingClaim` at it.
(k8s-restore-litestream)=
**With `litestream.enabled`.** Do not restore over the database: a plain restore would be undone,
because the next start restores the replica. Follow the same steps, with `--stage` in the pod's
arguments:
```yaml
args: ["restore", "/backups/", "--stage", "--force"]
```
`--force` is needed unless `files/` is empty: files are only ever added. This checks and migrates
the backup and leaves it at `/data/scrap.db.restored`. The next start takes it up in place of the
replica, and renames the old replica `/data/litestream.retired-