Skip to content
Merged
41 changes: 41 additions & 0 deletions agentic/docs/project-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -1065,6 +1065,47 @@ place, and the residual is narrow: it needs the previous candidate to pass every
well, so the worst case is promoting a revision that was believed smoked, not shipping a
page known to be broken.

### Rollback

Every deploy names its revision deterministically (`--revision-suffix=b$BUILD_ID`) and
promotes it explicitly, so earlier revisions are still there, still healthy, and one
command away. Rolling back needs no rebuild and takes under a minute:

1. List the recent revisions with their creation times:

```bash
gcloud run revisions list --service anyplot-app --region europe-west4 \
--format='table(name, creationTimestamp, status.conditions[0].status)' --limit 10
```

2. Send all traffic to the revision you want back:

```bash
gcloud run services update-traffic anyplot-app --region europe-west4 \
--to-revisions=<chosen-revision>=100
```

Same shape for `anyplot-api`. This is the lever for anything that ships inside the image
and only reveals itself in production.

**Pick the target by what it contains, not by its position.** "The previous revision" is
the right answer only while the change you are undoing is the most recent deploy; one more
deploy later, the previous revision carries it too. The creation timestamps in step 1 are
there for that — find the last revision created before the change landed. The current
example is the CSP nonce path in `app/security-headers.conf`: if Cloudflare ever stops
stamping its edge-injected script with the nonce it reads from our response header, any
revision from before that PR still serves the `'unsafe-inline'` policy byte for byte.
Once no such revision is left, the lever is a revert PR through the normal pipeline
instead — slower, but by then the question has long been answered live.

One thing is deliberately not built for that case: a Cloud Run environment variable that flips the
policy inside a running revision. nginx cannot read the process environment from its
configuration, so it would take a startup templating step (`envsubst` into a writable
path — and `/etc/nginx` is read-only in the unprivileged image), which neither CI nor a
local checkout here can exercise. A mechanism whose failure mode is "the container does
not start", and which nothing can test, is a worse rollback than a traffic split the
deploy pipeline already proves on every build.

## Debugging Tips

### Database Connection Issues
Expand Down
37 changes: 37 additions & 0 deletions app/cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,43 @@ steps:
}
# Humans get the SPA shell.
expect "$$HUMAN" "/" '<div id="root">' "a browser did not get the SPA shell"
# …and every <script> tag in that shell carries the CSP nonce from its
# own response header. The two halves are one `$$request_id` read twice
# — `add_header` in security-headers.conf, `sub_filter` in nginx.conf —
# and losing the stamp is the silent failure this probe exists for: a
# precompressed shell served by `gzip_static`, a location the filter
# never reaches, a renamed variable. The page keeps its `<div id="root">`
# and looks perfectly healthy to every probe above while the browser
# blocks every inline script on it, because a nonce in the policy makes
# 'unsafe-inline' inert. Cheap to check here, four weeks of nobody
# noticing if it is not.
#
# `--compressed` is the load-bearing flag, not a courtesy. Plain curl
# sends no Accept-Encoding, so `gzip_static` never reaches for the
# `.gz` — and the single most likely way to lose the stamp is exactly a
# precompressed shell coming back, which this probe would then pass
# while every browser got the untouched file (Copilot review).
# Reproduced against a local nginx with an index.html.gz planted in the
# docroot: plain curl saw 7 stamped tags, `curl --compressed` saw 0.
curl -fsS $$RETRY --compressed -A "$$HUMAN" -D head.out -o body.out "$$URL/" \
|| { echo "candidate did not serve the shell for the nonce probe"; exit 1; }
# Exactly 32 hex digits, which is what nginx's $$request_id always is.
# `[0-9a-f]*` would have accepted a bare `nonce-` — and `test -n` calls
# that non-empty — so a policy with an empty nonce and `nonce=""` on
# every tag would have matched itself all the way through this probe
# while the browser blocked the lot (Copilot review).
NONCE=$$(grep -i '^content-security-policy:' head.out | grep -oE 'nonce-[0-9a-f]{32}' | head -1)
test -n "$$NONCE" || { echo "the shell's Content-Security-Policy carries no 32-hex-digit nonce"; exit 1; }
# Every tag against THAT nonce, not "has some nonce": a stale or
# mismatched value is refused by the browser exactly like a missing one,
# and counting attributes rather than comparing them would wave it
# through (Copilot review).
TAGS=$$(grep -o '<script[^>]*>' body.out | wc -l)
MATCHING=$$(grep -o "<script nonce=\"$${NONCE#nonce-}\"" body.out | wc -l)
test "$$TAGS" -gt 0 || { echo "the shell served no <script> tag at all"; exit 1; }
test "$$MATCHING" = "$$TAGS" \
|| { echo "$$MATCHING of $$TAGS <script> tag(s) carry the header's $$NONCE — the rest would be blocked"; exit 1; }
echo "OK: all $$TAGS script tags of the shell carry the header's nonce"
# Crawlers get the prerendered page through @seo_proxy. The canonical
# link is the marker: the SPA shell carries none at all, and its value
# names the route, so one grep proves both "the bot hop ran" and "the
Expand Down
11 changes: 8 additions & 3 deletions app/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,14 @@
</script>

<!-- On-device debug console (Eruda), gated behind ?debug=1 so it never
ships to normal users. Plain <script> (not type="module") so it runs
before module parse-time errors and can capture them — critical for
debugging mobile-only crashes where remote DevTools aren't available. -->
ships to normal users. A classic script element (not type="module") so
it runs before module parse-time errors and can capture them — critical
for debugging mobile-only crashes where remote DevTools aren't
available. The opening tag is described rather than written out: nginx
stamps the CSP nonce with a blunt `sub_filter` over the literal string,
so a comment containing it comes back to the browser with a stray
`nonce=` inside — harmless, but confusing in the one artefact anyone
debugging CSP will read. -->
<script>
(function () {
try {
Expand Down
43 changes: 43 additions & 0 deletions app/nginx.conf
Original file line number Diff line number Diff line change
Expand Up @@ -165,11 +165,35 @@ server {
# levels — see security-headers.conf).
include /etc/nginx/security-headers.conf;

# The delivery half of the CSP nonce; the header half and the whole
# reasoning are in security-headers.conf. Stamp the SAME `$request_id` the
# policy names onto every `<script` tag of the HTML we serve. A nonce makes
# 'unsafe-inline' inert, so a tag that misses the stamp is a tag that stops
# running — and the first one in the shell is the theme resolver.
#
# Server level rather than per-location, deliberately: four locations can
# end up serving index.html — the exact match, the SPA fallback, and in the
# python server block below two regex routes whose `try_files /index.html
# =404` serves the file IN PLACE, without the internal redirect that would
# re-run location matching. A stamp missing from one of those is a blank
# page on exactly those routes and nowhere else.
#
# `<script` cannot match `</script>`, and sub_filter's default type list is
# text/html alone, so this never reaches into a .js chunk or a JSON body.
# It does also pass over the proxied crawler pages from @seo_proxy, which
# carry no executable script of ours — one JSON-LD data block, which no
# browser executes — and crawlers run no JS either way.
sub_filter '<script' '<script nonce="$request_id"';
sub_filter_once off;

# Compression - serve pre-compressed gzip files from Vite build when available
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/javascript application/xml+rss application/json;
# Serves `<file>.gz` untouched when it exists, which is exactly what
# sub_filter cannot rewrite. index.html is therefore excluded from
# precompression in app/vite.config.ts — the nonce depends on it.
gzip_static on;

# Vite-fingerprinted assets — filename changes on every build, safe to cache forever
Expand Down Expand Up @@ -380,6 +404,13 @@ server {
# main server block above).
include /etc/nginx/security-headers.conf;

# CSP nonce, same as the main server block — and this block is the reason
# it sits at server level: the two spec-route regex locations below serve
# `/index.html` through `try_files` as a FILE, so they never reach the
# `location = /index.html` block and would carry no stamp at all.
sub_filter '<script' '<script nonce="$request_id"';
sub_filter_once off;

gzip on;
gzip_vary on;
gzip_min_length 1024;
Expand Down Expand Up @@ -440,10 +471,20 @@ server {
}

# /:specId -> hub on main domain (no language segment; canonical is /{spec_id})
#
# `try_files /index.html =404` serves the shell as a FILE, in THIS location —
# no internal redirect, so `location = /index.html` and its headers are never
# reached. That is why the two routes below repeat the shell's Cache-Control
# instead of inheriting it: the shell now carries a per-request CSP nonce, and
# a stored copy would pair an old `nonce="…"` in the body with a fresh one in
# the header. Measured before the fix: these two routes answered with no
# Cache-Control at all while anyplot.ai/<spec> answered `no-store`.
location ~ "^/(?<spec_id>[A-Za-z0-9][A-Za-z0-9-]*)/?$" {
set $python_seo_uri /seo-proxy/$spec_id;
error_page 418 = @seo_proxy_python;
if ($is_bot) { return 418; }
add_header Cache-Control "no-cache, no-store, must-revalidate";
include /etc/nginx/security-headers.conf;
try_files /index.html =404;
}

Expand All @@ -452,6 +493,8 @@ server {
set $python_seo_uri /seo-proxy/$spec_id/python/$library;
error_page 418 = @seo_proxy_python;
if ($is_bot) { return 418; }
add_header Cache-Control "no-cache, no-store, must-revalidate";
include /etc/nginx/security-headers.conf;
try_files /index.html =404;
}

Expand Down
Loading
Loading