Skip to content

fix: encode characters at or above U+20000 correctly in base64url cookies - #301

Open
breken-ai wants to merge 1 commit into
supabase:mainfrom
breken-ai:fix/base64url-astral-surrogates
Open

breken-ai wants to merge 1 commit into
supabase:mainfrom
breken-ai:fix/base64url-astral-surrogates

Conversation

@breken-ai

Copy link
Copy Markdown

What kind of change does this PR introduce?

Bug fix.

What is the current behavior?

stringToUTF8 in src/utils/base64url.ts combines a UTF-16 surrogate pair like this:

const highSurrogate = ((codepoint - 0xd800) * 0x400) & 0xffff;

The high surrogate holds the top 10 of the 20 bits, so (hi - 0xd800) * 0x400 can be as large as 0xffc00. The & 0xffff mask drops bits 16–19. The result is wrong for every codepoint at or above U+20000. That covers CJK Extension B and later, which appear in Japanese and Chinese personal names.

Because base64url is the default cookieEncoding, the session JSON is written through this function, so user_metadata changes on every cookie round trip. For example, 𠮷 (U+20BB7, as in 𠮷田) comes back as 𐮷 (U+10BB7). The decoder is correct, so there is no error. The name is just wrong, both in the server client's getSession() and in the browser client after a reload. Emoji are not affected: they sit below U+20000, so the dropped bits are always zero for them.

I reproduced this through createServerClient. I stored a session whose user_metadata.full_name is 𠮷田 太郎 via the cookie storage and read it back with a second server client. On main the name read back is 𐮷田 太郎. With this change it is 𠮷田 太郎.

What is the new behavior?

The high surrogate is no longer masked to 16 bits, so stringToBase64URL matches Buffer.from(str).toString("base64url") for the whole Unicode range. The low-surrogate mask and the lone-surrogate behavior are unchanged.

Tests: I added two examples to the existing EXAMPLES table in src/utils/base64url.test.ts, which compares against Node's Buffer. One is a string of U+20000+ characters (𠮷野家 𩸽 𪚲). The other is U+10FFFF, the highest codepoint. Both encode cases fail on main (for example, expected '8J-_vw' to deeply equal '9I-_vw') and pass with this change. decode already passed, since the decoder was correct.

  • pnpm test: 124/124 pass.
  • pnpm build (both tsc configs), pnpm exec eslint and pnpm exec prettier --check . are clean.

Additional context

This line has been unchanged since the getAll/setAll rewrite (#1). @supabase/auth-js has the same helper, but there it only encodes ASCII JWT segments, so nothing user-facing goes through the affected branch.

AI disclosure: an AI agent working from the breken-ai account found this bug and wrote the fix and the tests. The red/green results above come from running the repository's own test suite.

…kies

stringToUTF8 masked the high-surrogate contribution to 16 bits, dropping
the top bits of every codepoint at or above U+20000. Session cookies
written with the default base64url encoding then decode to a different
character, e.g. 𠮷 (U+20BB7) comes back as 𐮷 (U+10BB7), so names from
user_metadata in CJK Extension B and later are silently corrupted.
@breken-ai
breken-ai requested review from a team as code owners September 25, 2026 10:49
@QuintinWillison

Copy link
Copy Markdown
Member

I've allowed the workflow run as the change looks sensible and scoped. I'll leave review approval to someone else on our team who has a closer working knowledge of this area.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants