dashboard: query and display per-service serving metrics #55

Problem

The home dashboard shows deployment and scheduling health: service state, ready instances, GPU allocation, and deployment activity. These management signals do not tell operators how an inference service is serving requests. Operators need serving latency, traffic, errors, and actual GPU/VRAM utilization alongside deployment health.

Ownership and agreed scope

Console is the unified entry point for the whole ModelSphere platform, not only a frontend for Swiss. Swiss owns deployment operations and deployment state; serving-metrics queries and aggregation belong to Console for this delivery.

This issue covers both the Console backend and the frontend. The Console backend will query an existing Prometheus directly, aggregate the results, and expose a read-only interface to the authenticated frontend. The browser must not access Prometheus directly or contain PromQL and exporter-label conventions.

This replaces the previous plan to consume a new metrics endpoint from swissd. modelsphere/swiss#19 is superseded by this issue; a Swiss metrics implementation or release is not a prerequisite.

browser -> Console (authentication, authorization, metrics query/aggregation) -> Prometheus
browser -> Console -> Swiss (deployment operations and state)

Use case and user experience

An operator selects an inference service and a time window on the home dashboard to distinguish deployment activity from serving performance and resource utilization.

• Add a service-quality panel alongside the deployment-activity chart.
• Show per-service p95 TTFT, QPS, error rate with explicit 5xx/429 semantics, GPU utilization, and VRAM utilization where their sources are available.
• Support a selectable time window, defaulting to the last hour.
• Show missing, unconfigured, unreachable, or failed metric data as unknown, with an explanation and retry for query failures. Never substitute zero-filled curves for unavailable data.
• Keep available metrics visible when another metric source fails; deployment status and history must remain usable when metrics are unavailable.
• Final chart placement can be refined once real data is available; serving quality and deployment activity must remain clearly distinguished.

Backend scope and contract

• Configure and verify direct server-side access to the existing Prometheus, including reachability and any required credentials. Do not infer a working metrics query path from scaler configuration.
• Verify which gateway, inference-engine, and device-exporter metrics are deployed and scraped before defining queries. Do not assume inference-engine exporters provide GPU/VRAM utilization.
• Define stable service identifiers and the mapping to deployment, namespace, pod, and GPU labels. Do not fabricate per-service GPU attribution when only shared or node-level data is available.
• Define metric units, rate denominators, status-code treatment, time-window boundaries, bucket/step semantics, and source availability. Derive p95 TTFT from compatible histogram data, not by averaging per-instance percentiles.
• Return explicit availability/error information per metric so partial failures are distinguishable from genuine zero values.
• Keep the interface read-only and enforce Console authentication and authorization before querying. Keep upstream credentials and query construction on the server.
• Use explicit query timeouts and window/resolution limits. Do not silently truncate services or time-series data.
• Review the Console-owned endpoint, configuration, permissions, and response contract in a design proposal before coding, following the repository design-proposal process. This issue does not declare a new endpoint already available.

Delivery

1. Verify the existing metric sources and service/resource mappings; record gaps and agree on the API/configuration design.
2. Implement and test direct Prometheus queries and aggregation in the Console backend.
3. Integrate the service-quality panel against the reviewed backend contract and test its available, partial, unknown, and retry states.

Acceptance criteria

• The Console backend queries Prometheus directly; serving metrics do not require a new Swiss endpoint.
• Metric definitions, units, label mappings, and service/resource attribution are documented and tested.
• The authenticated frontend renders the agreed per-service metrics and selectable window using only the Console interface.
• Missing sources, unconfigured Prometheus, timeouts, failed queries, and partial failures are reported honestly without fabricated zeros.
• Tests cover aggregation, window boundaries, time zones, genuine zero values, missing data, failed queries, authorization, and frontend retry behavior.
• Existing deployment status/history remains usable independently of metrics availability.

Out of scope

• Installing or managing Prometheus, exporters, or scrape configuration; use existing sources and explicitly report gaps.
• Metrics query or aggregation implementation in Swiss.
• A full monitoring center, alerting, SLO editing, logs, or cross-cluster aggregation.

Relationship to existing dashboard work

The existing dashboard implementation in #54 and its current-capabilities correction in #57 are separate from this new metrics work. Neither an unmerged /api/runs/timeline prototype nor a new Swiss metrics API is a prerequisite. Earlier comments requiring a Swiss metrics release before this integration are superseded by the ownership decision above.