Search

One query syntax, parsed once by query.py, serves the PWA’s search box, its facets and Store health screen, and the MCP search tool. The index is SQLite FTS5 over page chunks; it is a cache that migrations rebuild, and the store-health checks are ordinary search filters (8. Search is SQLite FTS5 with a shared query syntax; embeddings are deferred, no embeddings).

From query to results

A query string becomes SQL conditions plus one FTS5 MATCH, ranked by bm25, one hit per page.

        flowchart LR
    Q["Query string"] --> P["parse<br/>query.py"]
    P -->|"words, phrases, -word"| T["FTS5 terms"]
    P -->|"key:value"| F["Filters"]
    T --> W["_where<br/>SQL conditions"]
    F --> W
    W --> M["chunk_fts MATCH<br/>+ page filters"]
    M --> R["Rank tiers + bm25"]
    R --> G["Best chunk per page"]
    G --> S["Snippets<br/>with mark"]
    S --> API["PWA<br/>GET /api/search"]
    S --> MCP["MCP<br/>search tool"]

    classDef store fill:#e8f1ff,stroke:#4a7bd0;
    classDef claude fill:#fff3e0,stroke:#e08a00;
    classDef user fill:#e9f7ef,stroke:#2e9d5b;
    class M,R store;
    class MCP claude;
    class API user;
    

Step	What happens
Parse	One linear pass. Space means AND. "phrase", -word / !word (NOT) and key:value (also key:"two words", -key:value). Words with no letter or digit are dropped. At most 64 terms and 64 filters: a larger query is refused (HTTP 422, MCP tool error), never silently cut
Match	Each word is a prefix (vsc finds vscode) wherever it stands, in the title, tags, summary, text, a photo’s text or description, or a PDF’s or text file’s text. Phrases and NOT words are exact. Tokenizer porter unicode61
Rank	Tiers: every word in the title, then every word in title, tags or summary, with a topic page first within each of those tiers; then bm25 with column weights 10, 5, 3, 1 (title, tags, summary, text)
Group	A page’s best chunk wins. bm25() and snippet() refuse GROUP BY, so it is done in Python
Snippet	snippet() on the text column, HTML-escaped, hits in <mark>. With filters only: the summary, else the start of the text, newest first
Facets	GET /api/search/facets runs the same _where and counts kinds, tags (a/b counts for a) and personal/work

An empty query finds nothing over HTTP; over MCP it lists the most recently changed pages.

Filters

Generated from KEYS in query.py and Store._where in store/search.py. A - before a filter excludes. The full per-filter wording is under GET /api/search in HTTP API and CLI; the tool arguments are in MCP server; the user’s view is in Capture from your phone (the PWA).

Filter	Matches
kind:K	A kind. Aliases later / read (consume), board (whiteboard), doc (document). Two kind: mean either
tag:T	A tag with T as whole / segments: tag:a and tag:b both find a/b. tag:/ matches nothing (19. Kinds and tags editable over MCP, made safe by metadata revisions)
is:S	Any status (is:done, is:in_progress; _ for spaces)
is:personal, is:work	personal field true / not true
is:claude, is:claude-edited	Pages whose made_by is claude (written over MCP); text changed over MCP after the first revision
is:topic	Claude’s topic pages
has:image, has:drawing	An image capture; a current drawing. Other has: values match nothing
image:K	A photo Claude described as kind K (photo, whiteboard, handwriting, diagram, screenshot, document, other)
after:D, before:D	Last edited on or after / before D: a UTC date YYYY-MM-DD or a time ago (1d, 2w, 1m = 30 days, 1y = 365, up to 100). With date:created, creation date instead
due:D	A todo’s due day. D is a date, today, tomorrow, yesterday, optionally after <, <=, >, >=, =; due:none has no due date. due:<today is overdue
priority:P	high, medium, low, or none. urgent means high. Two mean either
sort:K	Order, not a condition: updated, created (newest first), title (A to Z), due (soonest), priority (high first). -sort:K reverses; the last valid one wins. Pages with no due or priority come last
media:M, later:S	A Later page’s medium (book, article, film, tv, video, podcast, music, game) or what it is for (read, watch, listen, play)
group:G	Not a condition: puts results in sections (letter, tag, type, now, due, area) on the server, so paging never splits one (31. Search orders results by section on the server). Pins are saved searches and pages kept on the pinned page (32. Pins are an ordinary page of links)
Health checks	is:orphan, is:untagged, is:stale, is:duplicate, has:dangling, has:open-questions, is:outdated (below)

“Today” is the server’s local day, so set TZ (default UTC in the image) for due:today. The web app only mirrors this syntax to add and remove tokens (web/src/terms.ts, filters.ts); parsing and matching stay on the server.

The index lifecycle

The index is derived data in the same database file, never a source of truth.

        flowchart TD
    subgraph write ["Write path, one transaction"]
        SV["Page save, tag or kind edit,<br/>summary, write-up"] --> IP["index_page"]
        IP --> C0["chunk ord 0<br/>title, tags, summary, text"]
        IP --> C1["chunk ord 1<br/>current write-up"]
        C0 --> TR["Triggers<br/>chunk_ai, chunk_ad"]
        C1 --> TR
        TR --> FTS[("chunk_fts")]
    end
    DER["Photo ingest<br/>derived row"] -->|"trigger derived_index"| CD["chunk with derived_id"]
    CD --> TR
    MIG["Baseline / a migration"] -->|"rebuild"| FTS

    classDef store fill:#e8f1ff,stroke:#4a7bd0;
    classDef claude fill:#fff3e0,stroke:#e08a00;
    class FTS,C0,C1,CD store;
    class DER claude;
    

• One transaction. index_page deletes the page’s own chunks (derived_id IS NULL) and inserts a fresh set, so a saved page is searchable at once and a failed save leaves the old index.

• Chunk shape. Chunk 0 holds title, tags, summary and the body as words (markdown.py: link text not addresses, code as typed, table cells). The summary is the page’s summary field, else Claude’s (current write-up, else enrichment). Chunk 1 is the current write-up: a board’s write-up or a topic’s synthesis. Control characters become spaces, since snippets mark hits with \x02 and \x03.

• Photo text. What ingest read (description, OCR) is its own chunk, made by a trigger when the derived row is inserted and removed when that row goes (a reading is deleted when a newer one replaces it; a drawing version when it is superseded). It fills only the text column. A PDF’s or text file’s text, which scrap reads itself (Files: read locally, for search), is such a chunk too.

• Disposable. There is no reindex command. index_page rebuilds one page’s own chunk, and a migration that changes what is indexed rebuilds every page’s (the trash’s too: undo brings a page back).

Store health as search

A health check is a search term, defined as one SQL fragment in store/health.py. Nothing is stored and no model is called, so a finding is always current.

        flowchart LR
    H["store/health.py<br/>SQL fragments"] --> W["_where<br/>search filter"]
    H --> CNT["counts()"]
    W --> SR["Search results<br/>is:orphan ..."]
    CNT --> HA["GET /api/health<br/>term + count"]
    HA --> SC["Store health screen"]
    SC -->|"tap a row"| SR

    classDef store fill:#e8f1ff,stroke:#4a7bd0;
    classDef user fill:#e9f7ef,stroke:#2e9d5b;
    class H store;
    class SC,SR user;
    

Check	Finds (pages not in the trash)
is:orphan	No page link either way (text, current write-up, related pick, topic and source); not a scrap gc review page
is:untagged	No tags; not a topic or gc review page
is:stale	An open todo, not due today or later, unchanged for 60 days (STALE_DAYS)
is:duplicate	A file of one of its captures is also another page’s
has:dangling	A page link in its text to no page or one in the trash
has:open-questions	A page with a Markdown heading (any level, setext too; not in code, a quote or a list item) that reads “Open questions” in any case (emphasis and links in it are ignored, a trailing colon too) and whose section is open. The section runs to the next heading of the same or a higher level, so deeper sub-sections and their boxes are part of it. A section with task boxes is decided by them alone: any unticked - [ ] makes it open, all ticked is done even with prose left. With no boxes it is open when it has any content (a paragraph, list, table or code); an empty one, or one with only sub-headings, is not. Topic pages are skipped (a “Contradictions and open questions” heading is not one), and the page-wide tasks counts are not the test
is:outdated	A board’s write-up older than its inputs or of an older prompt version, or linking a page that is in the trash or gone

Related pages and a topic’s synthesis have outdated definitions in the same module, but they are not findings: the nightly related pass and weekly topics pass refresh them (Job queue and Claude features). is:outdated is for the owner to regenerate, or scrap enrich --writeups --outdated.

Invariants

• One parser: the PWA, facets, health screen and MCP never re-implement filtering.

• The index is a cache: any migration may drop and rebuild it, and writes keep it in step inside the same transaction.

• Ranking prefers title, tag and summary matches over body repetition; no embeddings today.

• Query limits are refused loudly: a dropped NOT word would show more pages than asked for.

Related

• 8. Search is SQLite FTS5 with a shared query syntax; embeddings are deferred, 19. Kinds and tags editable over MCP, made safe by metadata revisions, 29. Store health checks as search filters; outdated write-ups are the owner’s to regenerate

• Data model and storage for the chunk, derived and tag tables; MCP server and access for the tool; The PWA for the Search screen; Architecture for the landing page.