Skip to content

Upload to Roam via the backend API instead of a headless browser - #146

Merged
dbieber merged 2 commits into
mainfrom
claude/roam-backend-api
Sep 5, 2026
Merged

Upload to Roam via the backend API instead of a headless browser#146
dbieber merged 2 commits into
mainfrom
claude/roam-backend-api

Conversation

@dbieber

@dbieber dbieber commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Why

Roam uploads from the Pi 400s have been failing. Sign-in works, but the graph never loads in the headless Firefox: the loading astrolabe stays for 90+ seconds and the uploader crashes with RuntimeError: Astrolabe still there after timeout every few minutes. An earlier screenshot from the same device shows why: Roam's hosting now serves a Vercel Security Checkpoint ("Failed to verify your browser, Code 99") to the old headless browser. The subscription is fine; the browser is being blocked.

Worse, devices still on pre-#114 code ignored the failed graph load, "inserted" notes at block uid None, and then committed them out of the queue. On one device 42 notes since Sep 1 were dropped that way (recovered from the process log).

What

  • roam_backend_api.py: minimal client for Roam's backend API (q and write), following the official SDK: Bearer token headers, one-time 308 redirect to the graph's peer host, retries on 503 / connection errors, client-generated 9-char block uids, order: "last".
  • roam_api_uploader.py: same output structure as helper.js (Daily Notes page → [[Go Note Go Notes]]:HH:MM AM session → notes, with indent/unindent stack and audio embeds), but via HTTP. Returns False on API errors so the runner keeps notes queued.
  • Runner picks the API uploader whenever ROAM_API_TOKEN is configured, otherwise falls back to the browser uploader with a hint.
  • settings.get(key, default) for optional settings; ROAM_API_TOKEN added to the template, masked in the settings server, and shown in the settings UI.
  • requests declared as a dependency; root conftest.py stubs the untracked secure_settings module so settings-importing tests run in CI.

Small intentional differences from the browser path: the Daily Notes page is chosen from the note's effective timestamp (same time already used for the session title since #145) rather than upload time, and an audio note with no blob storage configured no longer gets an empty {{audio: }} child.

Setup

In Roam: Settings → Graph → API tokens → new token with edit access. Then on the device: :set ROAM_API_TOKEN <token> (or via the settings UI).

Testing

  • 31 new unit tests (fake HTTP session / fake client): redirect handling, auth and 503 paths, uid generation, daily-note title/uid formatting incl. ordinals, full upload tree incl. indent/unindent/enter-empty/clear-empty/end-session, audio tagging, failure returns False. pytest: 63 passed.
  • Live probe against api.roamresearch.com with a bogus token: redirect to peer-24…:3001 handled, then a clean "Token cannot be verified" error.

🤖 Generated with Claude Code

dbieber and others added 2 commits September 5, 2026 08:20
Roam's web app now sits behind a Vercel bot-verification challenge that the
headless Firefox on the Pi fails: sign-in succeeds, but the graph never
finishes loading (or renders "Failed to verify your browser"), so the browser
uploader crashes on every batch. On devices still running the pre-#114 code
it also silently committed notes it never inserted (block uid None).

This adds roam_backend_api.py, a small client for Roam's backend API (q and
write endpoints, peer redirect handling, retries), and roam_api_uploader.py,
which builds the same Daily Notes > [[Go Note Go Notes]]: > session > notes
structure as helper.js without a browser. The runner uses it whenever
ROAM_API_TOKEN is configured and otherwise falls back to the browser uploader.

Also:
- settings.get() accepts a default for optional settings.
- ROAM_API_TOKEN added to the settings template, masked in the settings
  server, and exposed in the settings UI.
- requests is now a declared dependency.
- conftest.py stubs the per-device secure_settings module so tests that
  import settings can run in CI.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Lets ':set ROAM_API_TOKEN <token>' on a running device take effect without a
restart, and covers is_configured with a test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dbieber
dbieber force-pushed the claude/roam-backend-api branch from 3f03025 to b2f5aa4 Compare September 5, 2026 12:22
@dbieber
dbieber merged commit 14c0711 into main Sep 5, 2026
1 of 2 checks passed
@dbieber
dbieber deleted the claude/roam-backend-api branch September 5, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant