feat(core): wire a MeterProvider into the Python runtime (gated OTEL_METRICS_ENABLED) - #2672
MartinForReal wants to merge 2 commits into
Conversation
|
This pull request has been marked as stale because of no activity in the last 15 days. It will be closed in the next 5 days unless it is tagged "no stalebot" or other activity occurs. |
…METRICS_ENABLED) google-adk already defines and records its GenAI metric instruments (duration histograms, inference/tool-call counts, gen_ai.client.token.usage) under meter scope gcp.vertex.agent, but kagent.core.tracing.configure only installed a TracerProvider and LoggerProvider, so those data points were silently discarded on every invocation. Add a metric_exporter/PeriodicExportingMetricReader OTLP pipeline mirroring traces and logs (reusing _resolve_otlp_protocol/_resolve_otlp_timeout_seconds) behind a default-OFF OTEL_METRICS_ENABLED gate, flush metrics from force_flush alongside spans so the Agent Substrate pre-response checkpoint hazard is covered, and expose otel.metrics.* from the Helm chart. Signed-off-by: MartinForReal <fanshangxiang@gmail.com>
8b34914 to
2dea602
Compare
|
Hi @EItanya and @kagent-dev/developers, friendly ping for review when you have a moment. This wires a MeterProvider into the Python runtime behind OTEL_METRICS_ENABLED (follow-up to #2458) and exposes the otel.metrics.* Helm values. I have rebased it on latest main and resolved the conflicts, so it is ready for a look. Thanks! |
Signed-off-by: Shangxiang Fan (from Dev Box) <shafan@microsoft.com>
|
Addressed in 7a48296: OTEL_METRICS_EXPORTER_OTLP_ENDPOINT was never set by anything. The chart writes the standard OTel signal-specific OTEL_EXPORTER_OTLP_METRICS_ENDPOINT, and no kagent release ever emitted the legacy name, so that branch of the or-chain was dead code and the backward-compatibility comment was misleading. Removed it. Resolution is now the signal-specific variable falling back to the generic OTEL_EXPORTER_OTLP_ENDPOINT, which matches how the Go runtime resolves its OTLP endpoint. |
2 participants
What
Fixes the Python half of the metrics gap (#2458): the Python runtime never installed a
MeterProvider, so the seven GenAI metric instruments thatgoogle-adkalready defines and records (insrc/google/adk/telemetry/_metrics.pyunder meter scopegcp.vertex.agent) were silently discarded on every invocation. This wires a meter provider (gatedOTEL_METRICS_ENABLED, default-OFF) with no new instrument definitions and no new recording call sites — upstream already records them. Companion to #2148 / #2149 for Go.Changes
kagent.core.tracing.configure()(python/packages/kagent-core/src/kagent/core/tracing/_utils.py):metrics_enabled = os.getenv("OTEL_METRICS_ENABLED", "false") == "true"gate, mirroringOTEL_TRACING_ENABLED/OTEL_LOGGING_ENABLED._create_metric_exporter()helper reusing_resolve_otlp_protocol("METRICS"), the shared endpoint resolution, and_resolve_otlp_timeout_seconds("METRICS"), so protocol/endpoint/timeout semantics match traces and logs.MeterProvider(resource=resource, metric_readers=[PeriodicExportingMetricReader(OTLPMetricExporter(...))])and callsmetrics.set_meter_provider.force_flushto drain the meter provider, covering the Agent Substrate checkpoint hazard that motivatedKAGENT_PRE_RESPONSE_TRACE_FLUSH.helm/kagent/values.yaml: newotel.metricsblock alongsideotel.tracing/otel.logging(defaultenabled: false).helm/kagent/templates/controller-configmap.yaml: mapsotel.metrics.*toOTEL_METRICS_ENABLED/OTEL_EXPORTER_OTLP_METRICS_*with the same unified-vs-separate-endpoint branch as traces/logs.Default-OFF / no behaviour change
When
OTEL_METRICS_ENABLEDis unset (the default),configureinstalls no meter provider and behavior is byte-identical to before. Existing gates unchanged.Testing
uv run pytest packages/kagent-core/tests/test_tracing_configure.py -q→ 30 passed (added tests for default-off, endpoint/timeout reuse, meter-provider install, andforce_flushdraining the meter provider).uv run ruff checkanduv run ruff format --diffclean on changed files.Refs: #2458
DCO sign-off via
git commit -s.