Skip to content

obsv: add loader-facing write-pressure counters (graph edge writes, dispatch capacity busy) #372

Description

@EnRaiha

Version / build tested against

origin/main @ 1ff3551

Deployment mode

Origin, single node (local)

Engine(s) involved

graph, cluster, observability

Summary

I checked what /metrics already exposes before asking for anything, and most of what a
loader needs is there: bridge_queue_depth (registry field at
nodedb/src/control/metrics/database.rs:32, writer at :109) covers queue pressure,
wal_commit_latency_p99 covers commit stalls, and qps covers generic request rate.
The registry lives at nodedb/src/control/metrics/database.rs for per-database metrics
and nodedb/src/control/metrics/system/fields.rs:16 (SystemMetrics) for system ones.
Two things I could not find:

  1. A write counter per engine, for example edges written per second for graph
    (nodedb_graph_edges_written_total). Loaders size their pacing from arrival vs apply
    rate, and today they estimate it from qps, which mixes reads in.
  2. A counter for dispatch capacity busy (nodedb_dispatch_capacity_busy_total). Moved to cluster: a saturated dispatch queue has no documented retryable class — clients cannot back off correctly; add one and a counter #371, which asks for the same counter.

If both already exist under names I did not find, close this with a pointer and I will
document them on my side. The capacity-busy counter is tracked in #371. This issue tracks the per-engine write counter.

What actually happened? (check all that are true)

  • Acknowledged/committed data was lost, corrupted, or silently wrong
  • The server crashed, hung, or failed to start
  • A security or isolation boundary was crossed
  • Core functionality is broken with no acceptable workaround
  • A workaround exists (estimate from qps, or pace conservatively)

Proposed severity

SEV-4, observability gap. No correctness impact.

Reproducibility

Not applicable.

Last known-good version / commit (if a regression)

Not a regression.

Environment & logs

Linux x86_64, single-node origin. Catalogue read from docs/architecture.md (Per-Database
and Per-Tenant metric lists) on 2026-09-24.

Before submitting

  • I searched existing issues and the docs before asking.
  • This is not a security vulnerability.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions