Skip to content

fix(traces): slide the trace list and drop what storage evicted - #77

Merged
htcom-code merged 1 commit into
mainfrom
fix/trace-rows-sliding-window
Oct 12, 2026
Merged

htcom-code merged 1 commit into
mainfrom
fix/trace-rows-sliding-window

Conversation

@htcom-code

Copy link
Copy Markdown
Owner

Why

The trace list only ever grew: every batch merged in and nothing came out except a clear. That causes two problems.

  1. Cost. A long session re-sorts the whole list on every batch. At 24 h × 10 req/s that is about 860k rows.
  2. Honesty. Eviction deleted rows from IndexedDB but not from the list. Measured on main with a retention limit of 8, after ten arrivals:
6/6  7/7  8/8  9/8  10/8 … 15/8   (shown / retained)
rows on screen that are not in storage: 7

At the real limit (50,000) this takes longer to reach, but the list grows without bound, so a long session ends up in the same state.

What

Sliding window. The list holds the newest keep rows: one page to start, plus one page for every "load older". New traffic pushes the oldest rows out of the bottom. Those rows are still in storage, so hasMore turns on and "load older" brings them back. A clear resets the window to one page.

Eviction sync. pruneOldest now reports how far it reached: the last deleted row's (epochMillis, _key), in index order. Rows at or before that position are dropped from the list and from any search result. The screen therefore loses exactly what storage lost, including rows that share the boundary millisecond.

Infinite scroll. Nothing changes while the user is parked at the bottom. A batch leaves the row count and last index the same, so the load does not fire again.

Verification

  • 5 new scenarios in trace-window.test.ts: no ghost rows after eviction, ghost rows dropped from search results too, window slides, window grows only by explicit loads, hasMore turns off when storage is exhausted. Against the old store, 4 fail. The fifth guards the end-of-storage case.
  • Not checked in a browser. It needs a platform with traffic past the retention limit, and the backend was not running.
  • npm run lint: 7 warnings, the same set as main
  • npm test: 18 files, 207 tests pass

The trace list only ever grew. Every batch merged in and nothing came
out except a clear, so a long session re-sorted hundreds of thousands of
rows per batch. Worse, eviction deleted rows from IndexedDB without
removing them from the list. Measured on main with a limit of 8: after
ten arrivals the header read "15 shown · 8 retained", and 7 rows on
screen existed nowhere but the screen.

- The list holds the newest `keep` rows: one page to start, plus a page
  for every "load older". New traffic pushes the oldest out of the
  bottom; they are still in storage, hasMore turns on, and loading older
  brings them back. A clear resets it to one page
- pruneOldest now reports how far it reached (the last deleted row's
  epoch and key, in index order). Rows at or before that position are
  dropped from the list and from a search result, so the screen loses
  exactly what storage lost, including rows that share the boundary
  millisecond
- Infinite scroll is unaffected while parked at the bottom: a batch
  leaves the row count and last index unchanged, so it does not re-fire
- 5 new scenarios; against the old store, 4 fail (the fifth guards the
  end-of-storage case)
- Not checked in a browser: it needs a platform past the retention
  limit, and the backend was not running
- Gates: lint clean (same 7 warnings as main), 207 tests pass

Tags: #traces #storage #honesty

Co-Authored-By: htjulia <htjulia1@gmail.com>
@htcom-code
htcom-code merged commit 2f3364d into main Oct 12, 2026
4 checks passed
@htcom-code
htcom-code deleted the fix/trace-rows-sliding-window branch October 12, 2026 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant