diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index 6e0d39842b72..779424a97b56 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -1,6 +1,7 @@ ## 11.1.0 #### Project +* Update the e2e mock sender's OTLP proto to v1.11.1, so it replays recordings from a current OpenTelemetry Collector, which adds fields such as a metric's `metadata` that the old copy's strict JSON parser refused. * Replace the GenAI e2e cases' Spring AI application with an in-repo `e2e-spring-ai-service` module pinned to the released Spring AI 2.0.1, and move the mock LLM endpoints out of `e2e-service-provider` into a dedicated `e2e-mock-llm-server` module that all three GenAI cases share. The Spring AI application was previously built at test time by cloning `spring-projects/spring-ai-examples` and running Maven inside the image build, which resolved `spring-ai:2.0.0-SNAPSHOT`: a line that has since been abandoned, whose surviving builds all predate the 2.0.0 GA by a day and whose older builds have been pruned, so the fixture was pinned to nothing released and could no longer be rebuilt from the same bytes. Both modules are built by the existing e2e reactor and published to `ghcr.io/apache/skywalking` alongside the other e2e service images; `e2e-spring-ai-service` needs Java 17 and Spring Boot 4, so it is gated behind a new `jdk-17` profile in the e2e reactor and the jobs that build it now pin JDK 17 explicitly; `make -C test docker` refuses up front on an older JDK rather than packaging a missing or stale jar, since `clean` never reaches a module the profile excluded and compose would turn the absent jar into a directory. #### OAP Server @@ -29,6 +30,9 @@ * Refuse unsupported TraceQL with `400` instead of dropping the predicate and answering with unfiltered traces: syntax errors (`||` inside a spanset, regex, an attribute without a scope or leading dot), `!=` and other unsupported operators, negation, attribute-existence checks, multiple spansets, unknown intrinsics and values, and conditions a datasource cannot filter (`kind` on Zipkin and SkyWalking, `resource.instance` on Zipkin, `resource.remote.service` and `status = unset` on SkyWalking, `resource.instance` or `name` without a service on SkyWalking). The deprecated `tags` search parameter is parsed as logfmt on every datasource and goes through the same mapping and refusals. `kind = server` and `status = error` accept the bare keyword as Tempo does. Every search-result span carries a `status` attribute (`error`, `ok`, `unset`) next to `service.name` and `span.kind`, so a trace list can show failures. A trace the storage matched but none of whose spans satisfies every condition is left out instead of listed with every span, and the `/otlp` datasource compares names the way the receiver indexed them and keeps the `resource.` and `span.` scopes apart in its tag index, so `span.env` no longer matches a resource attribute. `duration >` and `<` are strict at microsecond precision. (#14093) * Support the OpenTelemetry Collector `hostmetrics` receiver as an alternative source for Linux and Windows host monitoring. `vm.yaml` and `windows.yaml` map it to the same `meter_vm_*` / `meter_win_*` metrics as node-exporter and windows_exporter, whose metrics keep their meaning, and add metrics only hostmetrics provides: CPU core count and normalized CPU usage, plus CPU load, file-system usage, system handle count and pagefile usage on Windows. node-exporter metrics without a matching hostmetrics source (`tcp_alloc`, `sockets_used`, `udp_inuse`, `filefd_allocated`) stay node-exporter only. The new `process-hostmetrics-linux` and `process-hostmetrics-windows` rules, enabled by default, report each process name as an instance of its host (`mp_process_linux_*` / `mp_process_windows_*`: process count, threads, CPU, resident memory, open handles and oldest process uptime). Reference Collector configurations for both systems are under `docs/en/setup/backend/`; they set the `job_name` and normalized `process_name` labels the rules route and group on, and sum same-named processes before export. * AI agent conversations adopt AI Sessionizer `130601c`. Calls to MCP servers: the `execution` kind the Sessionizer's Claude Code plugin writes, `streams//execution--.sd`, is stored like any other file; the document lists every `execution/1` record under `tool_executions`, joined to its step by tool-use id and kept once by its own id, a tool step names its records under `executions`, and a call to an MCP server carries `mcp_server` and `mcp_tool` in its `attrs`. The new `otel-rules/ai-agent/mcp_endpoint.yaml` gives one endpoint per MCP server and tool, `/`, under the agent's service, with `meter_ai_agent_mcp_calls`, `meter_ai_agent_mcp_calls_by_outcome` and `meter_ai_agent_mcp_duration` from the Sessionizer's `agent.mcp.calls` and `agent.mcp.duration`. The token rules read `agent.token.usage`, the Sessionizer's name for the metric. The document follows the Sessionizer's: an `llm.call` takes its provider bodies from the round's `provider_bodies` attribute and the session its count from `provider_bodies_landed`, and a round whose bodies do not read or point past its range is refused; talks are in the order they began across streams; only a `child` stream's talk is a child's, not an `auxiliary` one's; a stream's `opened_by`, the relations and a step's edges are in the order they happened, by record position inside one stream or workflow run and by time across them, never by id; change and execution records of one instant are in the order they were read; a record is read by the fields its format lists, and one of another shape is skipped; a round with a frame field of another type, or a negative sequence, row or round number, does not read; a record time is RFC 3339 as Session Data defines it, to the second, and anything else is no time; record times compare as instants. +* Fix Windows monitoring with windows_exporter v0.31.0 and later, which removed the `cs` collector and the `os` collector's memory metrics. `windows.yaml` read `windows_cs_physical_memory_bytes`, `windows_os_physical_memory_free_bytes`, `windows_os_virtual_memory_bytes` and `windows_os_virtual_memory_free_bytes`, so every `meter_win_memory_*` metric of the windows_exporter source was empty. It now reads the `memory` collector's `windows_memory_physical_total_bytes`, `windows_memory_physical_free_bytes`, `windows_memory_commit_limit` and `windows_memory_committed_bytes`, which carry the same values and are in windows_exporter's default collectors since v0.29.0, now the minimum. The Windows e2e replays a recording of windows_exporter v0.31.8 through OpenTelemetry Collector 0.158.0 in place of mock data in the old names. +* Fix `meter_win_cpu_total_percentage` counting interrupt and DPC time twice for windows_exporter. Windows' `% Privileged Time`, windows_exporter's `privileged` mode, already includes `% Interrupt Time` and `% DPC Time`, so busy time is `user` plus `privileged`. On a GitHub-hosted Windows runner the five modes summed to 1.005-1.009 seconds per second per core, and `idle`, `user` and `privileged` to exactly 1. +* Fix OpenTelemetry Collector v0.127.0 and later losing the host of Prometheus-scraped Linux and Windows metrics. The Collector's Prometheus receiver sends a target's host as `server.address` and no longer `net.host.name`, so `vm.yaml` and `windows.yaml`, which name a host by `node_identifier_host_name`, received no host and created no service. The OpenTelemetry receiver now takes `node_identifier_host_name` from `server.address` when the resource carries neither `net.host.name` nor `host.name`. * Fix the Elasticsearch `BulkProcessor` treating a bulk write as fully successful whenever the overall HTTP status was 200, even when Elasticsearch rejected individual items in the same response, for example a 429 `cluster_block_exception` when an index is switched to `read_only_allow_delete` by the flood-stage disk watermark. The response body's `errors` flag and per-item `status`/`error` are now decoded and checked on every 200 response; a rejected request's future now completes exceptionally instead of being treated as a success, while the rest of the batch still completes normally. One ERROR line per `_bulk` request summarizes the rejections grouped by status code and error type, never a document id or the raw ES error reason. A response whose `items` array is shorter than the request (a malformed or truncated response) also fails those requests instead of silently treating them as succeeded. This also fixes a related batching bug where, once a flush was split into multiple `_bulk` HTTP requests by `batchOfBytes`, the completion of one chunk's request completed or failed every request in the whole flush instead of just its own chunk, and a bug where requests whose bulk failed to even build (e.g. an encoding error) were left with a future that never completed, which could block a persistence round forever. #### UI diff --git a/docs/en/setup/backend/backend-win-monitoring.md b/docs/en/setup/backend/backend-win-monitoring.md index c4c901c5c512..dbcf5c50f850 100644 --- a/docs/en/setup/backend/backend-win-monitoring.md +++ b/docs/en/setup/backend/backend-win-monitoring.md @@ -9,10 +9,12 @@ Windows entity as a `Service` in OAP and on the `Layer: OS_WINDOWS`. 3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results. ## Setup **For OpenTelemetry receiver:** -1. Setup [Prometheus windows_exporter](https://github.com/prometheus-community/windows_exporter). +1. Setup [Prometheus windows_exporter](https://github.com/prometheus-community/windows_exporter) v0.29.0 or later, with its default collectors. The rules read its `cpu`, `memory`, `logical_disk` and `net` collectors. 2. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/). This is an example for OpenTelemetry Collector configuration [otel-collector-config.yaml](../../../../test/e2e-v2/cases/win/prometheus-windows_exporter/otel-collector-config.yaml). 3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md). +Each Windows host is a service named after its host: the resource attribute `host.name` or `net.host.name` when the Collector sends one, otherwise `server.address`, which is where the Collector's Prometheus receiver puts the scraped target's host from v0.127.0. + ### Native OpenTelemetry hostmetrics (expanded alternative) SkyWalking can also receive Windows host and process metrics directly from OpenTelemetry Collector Contrib, without Prometheus windows_exporter. diff --git a/oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/windows.data.yaml b/oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/windows.data.yaml index f90570bec591..df06ae9ff2fa 100644 --- a/oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/windows.data.yaml +++ b/oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/windows.data.yaml @@ -15,11 +15,29 @@ script: oap-server/server-starter/src/main/resources/otel-rules/windows.yaml input: + # windows_exporter's privileged time already includes its interrupt and dpc + # time, so only user and privileged count as busy. windows_cpu_time_total: - labels: node_identifier_host_name: test-host mode: user value: 100.0 + - labels: + node_identifier_host_name: test-host + mode: privileged + value: 60.0 + - labels: + node_identifier_host_name: test-host + mode: interrupt + value: 8.0 + - labels: + node_identifier_host_name: test-host + mode: dpc + value: 4.0 + - labels: + node_identifier_host_name: test-host + mode: idle + value: 200.0 # Native OpenTelemetry hostmetrics contract after Collector normalization. # Values are fixed for deterministic MAL assertions. system_cpu_logical_count: @@ -81,18 +99,23 @@ input: device: 'C:\\pagefile.sys' state: free value: 70.0 - windows_cs_physical_memory_bytes: + # windows_exporter's memory collector, v0.29.0 or later. + windows_memory_physical_total_bytes: - labels: + node_identifier_host_name: test-host value: 100.0 - windows_os_physical_memory_free_bytes: + windows_memory_physical_free_bytes: - labels: - value: 100.0 - windows_os_virtual_memory_free_bytes: + node_identifier_host_name: test-host + value: 30.0 + windows_memory_commit_limit: - labels: + node_identifier_host_name: test-host value: 100.0 - windows_os_virtual_memory_bytes: + windows_memory_committed_bytes: - labels: - value: 100.0 + node_identifier_host_name: test-host + value: 40.0 windows_logical_disk_read_bytes_total: - labels: node_identifier_host_name: test-host @@ -118,13 +141,21 @@ expected: samples: - labels: node_identifier_host_name: test-host - value: 3500.0 + value: 5000.0 meter_win_cpu_average_used: entities: - scope: SERVICE service: test-host layer: OS_WINDOWS samples: + - labels: + node_identifier_host_name: test-host + mode: idle + value: 6500.0 + - labels: + node_identifier_host_name: test-host + mode: interrupt + value: 400.0 - labels: node_identifier_host_name: test-host mode: user @@ -132,45 +163,57 @@ expected: meter_win_memory_total: entities: - scope: SERVICE + service: test-host layer: OS_WINDOWS samples: - labels: + node_identifier_host_name: test-host value: 100.0 meter_win_memory_available: entities: - scope: SERVICE + service: test-host layer: OS_WINDOWS samples: - labels: - value: 100.0 + node_identifier_host_name: test-host + value: 30.0 meter_win_memory_used: entities: - scope: SERVICE + service: test-host layer: OS_WINDOWS samples: - labels: - value: 0.0 + node_identifier_host_name: test-host + value: 70.0 meter_win_memory_virtual_memory_free: entities: - scope: SERVICE + service: test-host layer: OS_WINDOWS samples: - labels: - value: 100.0 + node_identifier_host_name: test-host + value: 60.0 meter_win_memory_virtual_memory_total: entities: - scope: SERVICE + service: test-host layer: OS_WINDOWS samples: - labels: + node_identifier_host_name: test-host value: 100.0 meter_win_memory_virtual_memory_percentage: entities: - scope: SERVICE + service: test-host layer: OS_WINDOWS samples: - labels: - value: -0.0 + node_identifier_host_name: test-host + value: 40.0 meter_win_disk_read: entities: - scope: SERVICE diff --git a/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessor.java b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessor.java index 3a9c2919bcf2..1d2955c0df54 100644 --- a/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessor.java +++ b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessor.java @@ -105,6 +105,14 @@ public class OpenTelemetryMetricRequestProcessor implements Service, MalConverte // in resource attributes (e.g., Envoy AI Gateway), it takes precedence via putIfAbsent. .put("service.name", "job_name") .build(); + + /** + * Where a Prometheus target's host arrives from an OTel Collector v0.127.0 or later: its Prometheus receiver + * sends {@code server.address} and no longer {@code net.host.name}. Read only when no legacy host attribute + * gave {@code node_identifier_host_name}, so an explicit {@code host.name} still wins. + */ + private static final String SERVER_ADDRESS = "server.address"; + private static final String NODE_IDENTIFIER_HOST_NAME = "node_identifier_host_name"; /** * Active MAL converters, keyed by {@code ":"} so boot-time entries and * runtime-rule entries share one namespace. A runtime {@code /addOrUpdate} for a rule that @@ -145,20 +153,7 @@ public void processMetricsRequest(final ExportMetricsServiceRequest requests) { log.debug("Resource attributes: {}", request.getResource().getAttributesList()); } - // First pass: collect all resource attributes with dots replaced by underscores - final Map nodeLabels = new HashMap<>(); - for (final var it : request.getResource().getAttributesList()) { - final String key = it.getKey().replace('.', '_'); - final String value = anyValueToString(it.getValue()); - nodeLabels.putIfAbsent(key, value); - } - // Second pass: apply fallback mappings — only if the target key is absent - for (final var it : request.getResource().getAttributesList()) { - final String targetKey = FALLBACK_LABEL_MAPPINGS.get(it.getKey()); - if (targetKey != null) { - nodeLabels.putIfAbsent(targetKey, anyValueToString(it.getValue())); - } - } + final Map nodeLabels = nodeLabels(request.getResource().getAttributesList()); // A request is analysed a minute at a time, oldest minute first. A MAL rule folds every sample of // an entity into one value stamped with the first sample's time, which is right for a scrape, whose @@ -255,6 +250,35 @@ public void start() throws ModuleStartException { } } + /** + * The labels every sample of a resource carries, from its resource attributes. + */ + static Map nodeLabels(final List attributes) { + // First pass: collect all resource attributes with dots replaced by underscores + final Map nodeLabels = new HashMap<>(); + for (final var it : attributes) { + final String key = it.getKey().replace('.', '_'); + final String value = anyValueToString(it.getValue()); + nodeLabels.putIfAbsent(key, value); + } + // Second pass: apply fallback mappings — only if the target key is absent + for (final var it : attributes) { + final String targetKey = FALLBACK_LABEL_MAPPINGS.get(it.getKey()); + if (targetKey != null) { + nodeLabels.putIfAbsent(targetKey, anyValueToString(it.getValue())); + } + } + if (!nodeLabels.containsKey(NODE_IDENTIFIER_HOST_NAME)) { + for (final var it : attributes) { + if (SERVER_ADDRESS.equals(it.getKey())) { + nodeLabels.put(NODE_IDENTIFIER_HOST_NAME, anyValueToString(it.getValue())); + break; + } + } + } + return nodeLabels; + } + private static Map buildLabels(List kvs) { return kvs .stream() diff --git a/oap-server/server-receiver-plugin/otel-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessorTest.java b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessorTest.java index 883ece04295b..38ac0ded4451 100644 --- a/oap-server/server-receiver-plugin/otel-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessorTest.java +++ b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryMetricRequestProcessorTest.java @@ -17,6 +17,8 @@ package org.apache.skywalking.oap.server.receiver.otel.otlp; +import io.opentelemetry.proto.common.v1.AnyValue; +import io.opentelemetry.proto.common.v1.KeyValue; import io.opentelemetry.proto.metrics.v1.ExponentialHistogram; import io.opentelemetry.proto.metrics.v1.ExponentialHistogramDataPoint; import io.opentelemetry.proto.metrics.v1.Metric; @@ -131,4 +133,33 @@ public void testAdaptExponentialHistogram() throws NoSuchMethodException, Invoca assertTrue(histogramMetric.getBuckets().containsKey(-Math.pow(base, 17))); assertEquals(2, histogramMetric.getBuckets().get(-Math.pow(base, 17))); } + + private static KeyValue attribute(final String key, final String value) { + return KeyValue.newBuilder().setKey(key).setValue(AnyValue.newBuilder().setStringValue(value)).build(); + } + + // OTel Collector v0.127.0 and later send a Prometheus target's host as server.address only. + @Test + public void testHostNameFromServerAddress() { + final Map labels = OpenTelemetryMetricRequestProcessor.nodeLabels(List.of( + attribute("service.name", "windows-monitoring"), + attribute("server.address", "172.25.0.1"), + attribute("service.instance.id", "172.25.0.1:9182") + )); + assertEquals("172.25.0.1", labels.get("node_identifier_host_name")); + assertEquals("windows-monitoring", labels.get("job_name")); + assertEquals("172.25.0.1", labels.get("server_address")); + } + + @Test + public void testLegacyHostAttributesWinOverServerAddress() { + assertEquals("win-host", OpenTelemetryMetricRequestProcessor.nodeLabels(List.of( + attribute("server.address", "172.25.0.1"), + attribute("host.name", "win-host") + )).get("node_identifier_host_name")); + assertEquals("10.211.55.3", OpenTelemetryMetricRequestProcessor.nodeLabels(List.of( + attribute("net.host.name", "10.211.55.3"), + attribute("server.address", "10.211.55.3") + )).get("node_identifier_host_name")); + } } diff --git a/oap-server/server-starter/src/main/resources/otel-rules/windows.yaml b/oap-server/server-starter/src/main/resources/otel-rules/windows.yaml index c7ad4a4d22a6..34e421528852 100644 --- a/oap-server/server-starter/src/main/resources/otel-rules/windows.yaml +++ b/oap-server/server-starter/src/main/resources/otel-rules/windows.yaml @@ -29,10 +29,13 @@ metricsRules: # --------------------------------------------------------------------------- # CPU + # windows_exporter's `privileged` time already includes its `interrupt` and + # `dpc` time (Windows' % Privileged Time counter), so busy time is user plus + # privileged; adding the other two would count them twice. - name: cpu_total_percentage exp: > (windows_cpu_time_total * 100) - .tagNotEqual('mode','idle') + .tagMatch('mode','^(user|privileged)$') .sum(['node_identifier_host_name']) .rate('PT1M') + @@ -52,15 +55,17 @@ metricsRules: .rate('PT1M') # Physical memory + # windows_exporter's memory collector, v0.29.0 or later. The cs collector and + # the os collector's memory metrics this used to read were removed in v0.31.0. - name: memory_total exp: > - windows_cs_physical_memory_bytes + windows_memory_physical_total_bytes + system_memory_limit.sum(['node_identifier_host_name']) - name: memory_available exp: > - windows_os_physical_memory_free_bytes + windows_memory_physical_free_bytes + system_memory_usage .tagEqual('state','free') @@ -68,7 +73,7 @@ metricsRules: - name: memory_used exp: > - (windows_cs_physical_memory_bytes - windows_os_physical_memory_free_bytes) + (windows_memory_physical_total_bytes - windows_memory_physical_free_bytes) + ( system_memory_limit.sum(['node_identifier_host_name']) @@ -82,7 +87,7 @@ metricsRules: # Memory\\Committed Bytes - name: memory_virtual_memory_free exp: > - windows_os_virtual_memory_free_bytes + (windows_memory_commit_limit - windows_memory_committed_bytes) + ( system_virtual_memory_commit_limit.sum(['node_identifier_host_name']) @@ -92,14 +97,14 @@ metricsRules: - name: memory_virtual_memory_total exp: > - windows_os_virtual_memory_bytes + windows_memory_commit_limit + system_virtual_memory_commit_limit.sum(['node_identifier_host_name']) - name: memory_virtual_memory_percentage exp: > ( - 100 - ((windows_os_virtual_memory_free_bytes * 100) / windows_os_virtual_memory_bytes) + (windows_memory_committed_bytes * 100) / windows_memory_commit_limit ) + ( diff --git a/test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/windows.yaml b/test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/windows.yaml index c7ad4a4d22a6..34e421528852 100644 --- a/test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/windows.yaml +++ b/test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/windows.yaml @@ -29,10 +29,13 @@ metricsRules: # --------------------------------------------------------------------------- # CPU + # windows_exporter's `privileged` time already includes its `interrupt` and + # `dpc` time (Windows' % Privileged Time counter), so busy time is user plus + # privileged; adding the other two would count them twice. - name: cpu_total_percentage exp: > (windows_cpu_time_total * 100) - .tagNotEqual('mode','idle') + .tagMatch('mode','^(user|privileged)$') .sum(['node_identifier_host_name']) .rate('PT1M') + @@ -52,15 +55,17 @@ metricsRules: .rate('PT1M') # Physical memory + # windows_exporter's memory collector, v0.29.0 or later. The cs collector and + # the os collector's memory metrics this used to read were removed in v0.31.0. - name: memory_total exp: > - windows_cs_physical_memory_bytes + windows_memory_physical_total_bytes + system_memory_limit.sum(['node_identifier_host_name']) - name: memory_available exp: > - windows_os_physical_memory_free_bytes + windows_memory_physical_free_bytes + system_memory_usage .tagEqual('state','free') @@ -68,7 +73,7 @@ metricsRules: - name: memory_used exp: > - (windows_cs_physical_memory_bytes - windows_os_physical_memory_free_bytes) + (windows_memory_physical_total_bytes - windows_memory_physical_free_bytes) + ( system_memory_limit.sum(['node_identifier_host_name']) @@ -82,7 +87,7 @@ metricsRules: # Memory\\Committed Bytes - name: memory_virtual_memory_free exp: > - windows_os_virtual_memory_free_bytes + (windows_memory_commit_limit - windows_memory_committed_bytes) + ( system_virtual_memory_commit_limit.sum(['node_identifier_host_name']) @@ -92,14 +97,14 @@ metricsRules: - name: memory_virtual_memory_total exp: > - windows_os_virtual_memory_bytes + windows_memory_commit_limit + system_virtual_memory_commit_limit.sum(['node_identifier_host_name']) - name: memory_virtual_memory_percentage exp: > ( - 100 - ((windows_os_virtual_memory_free_bytes * 100) / windows_os_virtual_memory_bytes) + (windows_memory_committed_bytes * 100) / windows_memory_commit_limit ) + ( diff --git a/test/e2e-v2/cases/win/expected/service.yml b/test/e2e-v2/cases/win/expected/service.yml index 940f9201ae95..a1c6f1e83b90 100644 --- a/test/e2e-v2/cases/win/expected/service.yml +++ b/test/e2e-v2/cases/win/expected/service.yml @@ -14,10 +14,10 @@ # limitations under the License. {{- containsOnce . }} -- id: {{ b64enc "10.211.55.3" }}.1 - name: 10.211.55.3 +- id: {{ b64enc "172.25.0.1" }}.1 + name: 172.25.0.1 group: "" - shortname: 10.211.55.3 + shortname: 172.25.0.1 normal: true layers: - OS_WINDOWS diff --git a/test/e2e-v2/cases/win/mock-data/otel-mock-metrics.json b/test/e2e-v2/cases/win/mock-data/otel-mock-metrics.json index 4cb9c82d67ae..77937f2d6dfc 100644 --- a/test/e2e-v2/cases/win/mock-data/otel-mock-metrics.json +++ b/test/e2e-v2/cases/win/mock-data/otel-mock-metrics.json @@ -10,25 +10,25 @@ } }, { - "key": "net.host.name", + "key": "server.address", "value": { - "stringValue": "10.211.55.3" + "stringValue": "172.25.0.1" } }, { "key": "service.instance.id", "value": { - "stringValue": "10.211.55.3:9182" + "stringValue": "172.25.0.1:9182" } }, { - "key": "net.host.port", + "key": "server.port", "value": { "stringValue": "9182" } }, { - "key": "http.scheme", + "key": "url.scheme", "value": { "stringValue": "http" } @@ -37,7 +37,10 @@ }, "scopeMetrics": [ { - "scope": {}, + "scope": { + "name": "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver", + "version": "0.158.0" + }, "metrics": [ { "name": "windows_cpu_time_total", @@ -45,27 +48,119 @@ "sum": { "dataPoints": [ { - "startTimeUnixNano": "1676140244999000000", - "timeUnixNano": "1676140395007000000", - "asDouble": 3.4375, "attributes": [ { "key": "core", "value": { - "stringValue": "0.0" + "stringValue": "0,0" + } + }, + { + "key": "mode", + "value": { + "stringValue": "dpc" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 1.9375 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,0" + } + }, + { + "key": "mode", + "value": { + "stringValue": "idle" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 84.6875 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,0" + } + }, + { + "key": "mode", + "value": { + "stringValue": "interrupt" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 3.0625 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,0" + } + }, + { + "key": "mode", + "value": { + "stringValue": "privileged" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 60.25 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,0" + } + }, + { + "key": "mode", + "value": { + "stringValue": "user" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 98.296875 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,1" + } + }, + { + "key": "mode", + "value": { + "stringValue": "dpc" } } - ] + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 0.203125 }, { - "startTimeUnixNano": "1676140244999000000", - "timeUnixNano": "1676140395007000000", - "asDouble": 707.984375, "attributes": [ { "key": "core", "value": { - "stringValue": "0.0" + "stringValue": "0,1" } }, { @@ -74,75 +169,149 @@ "stringValue": "idle" } } - ] + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 85.59375 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,1" + } + }, + { + "key": "mode", + "value": { + "stringValue": "interrupt" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 3.125 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,1" + } + }, + { + "key": "mode", + "value": { + "stringValue": "privileged" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 58.171875 + }, + { + "attributes": [ + { + "key": "core", + "value": { + "stringValue": "0,1" + } + }, + { + "key": "mode", + "value": { + "stringValue": "user" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 99.453125 } - ] - } - } - ] - } - ] - }, - { - "resource": { - "attributes": [ - { - "key": "service.name", - "value": { - "stringValue": "windows-monitoring" - } - }, - { - "key": "net.host.name", - "value": { - "stringValue": "10.211.55.3" - } - }, - { - "key": "service.instance.id", - "value": { - "stringValue": "10.211.55.3:9182" - } - }, - { - "key": "net.host.port", - "value": { - "stringValue": "9182" - } - }, - { - "key": "http.scheme", - "value": { - "stringValue": "http" - } - } - ] - }, - "scopeMetrics": [ - { - "scope": {}, - "metrics": [ + ], + "aggregationTemporality": 2, + "isMonotonic": true + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "counter" + } + } + ] + }, { - "name": "windows_os_virtual_memory_bytes", - "description": "OperatingSystem.TotalVirtualMemorySize", - "gauge": { + "name": "windows_logical_disk_read_bytes_total", + "description": "The number of bytes transferred from the disk during read operations (LogicalDisk.DiskReadBytesPerSec)", + "sum": { "dataPoints": [ { - "timeUnixNano": "1676140375004000000", - "asDouble": 8.8387584E9 + "attributes": [ + { + "key": "volume", + "value": { + "stringValue": "C:" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 1110196736 + }, + { + "attributes": [ + { + "key": "volume", + "value": { + "stringValue": "D:" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 17408 + }, + { + "attributes": [ + { + "key": "volume", + "value": { + "stringValue": "HarddiskVolume2" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 0 + }, + { + "attributes": [ + { + "key": "volume", + "value": { + "stringValue": "HarddiskVolume3" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 12578304 } - ] - } + ], + "aggregationTemporality": 2, + "isMonotonic": true + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "counter" + } + } + ] }, { - "name": "windows_logical_disk_write_seconds_total", - "description": "Seconds that the disk was busy servicing write requests (LogicalDisk.PercentDiskWriteTime)", + "name": "windows_logical_disk_write_bytes_total", + "description": "The number of bytes transferred to the disk during write operations (LogicalDisk.DiskWriteBytesPerSec)", "sum": { "dataPoints": [ { - "startTimeUnixNano": "1676140244999000000", - "timeUnixNano": "1676140375004000000", - "asDouble": 9.669203699999999, "attributes": [ { "key": "volume", @@ -150,28 +319,226 @@ "stringValue": "C:" } } - ] + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 1111252480 + }, + { + "attributes": [ + { + "key": "volume", + "value": { + "stringValue": "D:" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 814047232 }, { - "startTimeUnixNano": "1676140244999000000", - "timeUnixNano": "1676140375004000000", - "asDouble": 8.321E-4, "attributes": [ { "key": "volume", "value": { - "stringValue": "HarddiskVolume1" + "stringValue": "HarddiskVolume2" } } - ] + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 0 + }, + { + "attributes": [ + { + "key": "volume", + "value": { + "stringValue": "HarddiskVolume3" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 159744 + } + ], + "aggregationTemporality": 2, + "isMonotonic": true + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "counter" + } + } + ] + }, + { + "name": "windows_memory_commit_limit", + "description": "(CommitLimit)", + "gauge": { + "dataPoints": [ + { + "timeUnixNano": "1790731774528000000", + "asDouble": 10597691392 + } + ] + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "gauge" + } + } + ] + }, + { + "name": "windows_memory_committed_bytes", + "description": "(CommittedBytes)", + "gauge": { + "dataPoints": [ + { + "timeUnixNano": "1790731774528000000", + "asDouble": 2574090240 + } + ] + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "gauge" + } + } + ] + }, + { + "name": "windows_memory_physical_free_bytes", + "description": "The amount of physical memory currently available, in bytes. This is the amount of physical memory that can be immediately reused without having to write its contents to disk first. It is the sum of the size of the standby, free, and zero lists.", + "gauge": { + "dataPoints": [ + { + "timeUnixNano": "1790731774528000000", + "asDouble": 5790355456 } ] - } + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "gauge" + } + } + ] + }, + { + "name": "windows_memory_physical_total_bytes", + "description": "The amount of actual physical memory, in bytes.", + "gauge": { + "dataPoints": [ + { + "timeUnixNano": "1790731774528000000", + "asDouble": 8584425472 + } + ] + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "gauge" + } + } + ] + }, + { + "name": "windows_net_bytes_received_total", + "description": "(Network.BytesReceivedPerSec)", + "sum": { + "dataPoints": [ + { + "attributes": [ + { + "key": "nic", + "value": { + "stringValue": "Mellanox ConnectX-4 Lx Virtual Ethernet Adapter" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 236607072 + }, + { + "attributes": [ + { + "key": "nic", + "value": { + "stringValue": "Microsoft Hyper-V Network Adapter _3" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 253580969 + } + ], + "aggregationTemporality": 2, + "isMonotonic": true + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "counter" + } + } + ] + }, + { + "name": "windows_net_bytes_sent_total", + "description": "(Network.BytesSentPerSec)", + "sum": { + "dataPoints": [ + { + "attributes": [ + { + "key": "nic", + "value": { + "stringValue": "Mellanox ConnectX-4 Lx Virtual Ethernet Adapter" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 2690078 + }, + { + "attributes": [ + { + "key": "nic", + "value": { + "stringValue": "Microsoft Hyper-V Network Adapter _3" + } + } + ], + "timeUnixNano": "1790731774528000000", + "asDouble": 2613057 + } + ], + "aggregationTemporality": 2, + "isMonotonic": true + }, + "metadata": [ + { + "key": "prometheus.type", + "value": { + "stringValue": "counter" + } + } + ] } ] } ] } ] -} - +} \ No newline at end of file diff --git a/test/e2e-v2/cases/win/prometheus-windows_exporter/otel-collector-config.yaml b/test/e2e-v2/cases/win/prometheus-windows_exporter/otel-collector-config.yaml index 872c23d4291a..8b51751f353a 100644 --- a/test/e2e-v2/cases/win/prometheus-windows_exporter/otel-collector-config.yaml +++ b/test/e2e-v2/cases/win/prometheus-windows_exporter/otel-collector-config.yaml @@ -17,7 +17,7 @@ receivers: prometheus: config: scrape_configs: - - job_name: "windows-monitoring" # make sure to use this in the vm.yaml to filter only VM metrics + - job_name: "windows-monitoring" # windows.yaml keeps only this job scrape_interval: 10s static_configs: - targets: ["win-service:9182"] @@ -33,12 +33,12 @@ exporters: tls: insecure: true # Exports data to the console - logging: - logLevel: debug + debug: + verbosity: detailed service: pipelines: metrics: receivers: [prometheus] processors: [batch] - exporters: [otlp, logging] + exporters: [otlp, debug] diff --git a/test/e2e-v2/cases/win/win-cases.yaml b/test/e2e-v2/cases/win/win-cases.yaml index ebfd25223e0c..981912b9d184 100644 --- a/test/e2e-v2/cases/win/win-cases.yaml +++ b/test/e2e-v2/cases/win/win-cases.yaml @@ -18,5 +18,15 @@ cases: - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service ls expected: expected/service.yml - - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_win_memory_virtual_memory_total --service-name=10.211.55.3 + - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_win_memory_total --service-name=172.25.0.1 + expected: expected/metrics-has-value.yml + - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_win_memory_available --service-name=172.25.0.1 + expected: expected/metrics-has-value.yml + - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_win_memory_used --service-name=172.25.0.1 + expected: expected/metrics-has-value.yml + - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_win_memory_virtual_memory_free --service-name=172.25.0.1 + expected: expected/metrics-has-value.yml + - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_win_memory_virtual_memory_total --service-name=172.25.0.1 + expected: expected/metrics-has-value.yml + - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_win_memory_virtual_memory_percentage --service-name=172.25.0.1 expected: expected/metrics-has-value.yml diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/logs/v1/logs_service.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/logs/v1/logs_service.proto index 8260d8aaeb82..8be5cf75ef17 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/logs/v1/logs_service.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/logs/v1/logs_service.proto @@ -28,8 +28,6 @@ option go_package = "go.opentelemetry.io/proto/otlp/collector/logs/v1"; // OpenTelemetry and an collector, or between an collector and a central collector (in this // case logs are sent/received to/from multiple Applications). service LogsService { - // For performance reasons, it is recommended to keep this RPC - // alive for the entire life of the application. rpc Export(ExportLogsServiceRequest) returns (ExportLogsServiceResponse) {} } diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/metrics/v1/metrics_service.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/metrics/v1/metrics_service.proto index dd48f1ad3a16..bc0242844151 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/metrics/v1/metrics_service.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/metrics/v1/metrics_service.proto @@ -28,8 +28,6 @@ option go_package = "go.opentelemetry.io/proto/otlp/collector/metrics/v1"; // instrumented with OpenTelemetry and a collector, or between a collector and a // central collector. service MetricsService { - // For performance reasons, it is recommended to keep this RPC - // alive for the entire life of the application. rpc Export(ExportMetricsServiceRequest) returns (ExportMetricsServiceResponse) {} } diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/trace/v1/trace_service.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/trace/v1/trace_service.proto index d6fe67f9e553..efbbedbe4545 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/trace/v1/trace_service.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/trace/v1/trace_service.proto @@ -28,8 +28,6 @@ option go_package = "go.opentelemetry.io/proto/otlp/collector/trace/v1"; // OpenTelemetry and a collector, or between a collector and a central collector (in this // case spans are sent/received to/from multiple Applications). service TraceService { - // For performance reasons, it is recommended to keep this RPC - // alive for the entire life of the application. rpc Export(ExportTraceServiceRequest) returns (ExportTraceServiceResponse) {} } diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/common/v1/common.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/common/v1/common.proto index d233677c109a..85bd3f2c00ac 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/common/v1/common.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/common/v1/common.proto @@ -22,7 +22,7 @@ option java_package = "io.opentelemetry.proto.common.v1"; option java_outer_classname = "CommonProto"; option go_package = "go.opentelemetry.io/proto/otlp/common/v1"; -// AnyValue is used to represent any type of attribute value. AnyValue may contain a +// Represents any type of attribute value. AnyValue may contain a // primitive value such as a string or integer or it may contain an arbitrary nested // object containing arrays, key-value lists and primitives. message AnyValue { @@ -36,6 +36,19 @@ message AnyValue { ArrayValue array_value = 5; KeyValueList kvlist_value = 6; bytes bytes_value = 7; + // Reference to the string value in ProfilesDictionary.string_table. + // + // Note: This is currently used exclusively in the Profiling signal. + // Implementers of OTLP receivers for signals other than Profiling should + // treat the presence of this value as a non-fatal issue. + // Log an error or warning indicating an unexpected field intended for the + // Profiling signal and process the data as if this value were absent or + // empty, ignoring its semantic content for the non-Profiling signal. + // + // Status: [Alpha] + // + // [Since v1.10.0] + int32 string_value_strindex = 8; } } @@ -54,24 +67,94 @@ message ArrayValue { message KeyValueList { // A collection of key/value pairs of key-value pairs. The list may be empty (may // contain 0 elements). + // // The keys MUST be unique (it is not allowed to have more than one // value with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated KeyValue values = 1; } -// KeyValue is a key-value pair that is used to store Span attributes, Link +// Represents a key-value pair that is used to store Span attributes, Link // attributes, etc. message KeyValue { + // The key name of the pair. + // key_strindex MUST NOT be set if key is used. string key = 1; + + // The value of the pair. AnyValue value = 2; + + // Reference to the string key in ProfilesDictionary.string_table. + // key MUST NOT be set if key_strindex is used. + // + // Note: This is currently used exclusively in the Profiling signal. + // Implementers of OTLP receivers for signals other than Profiling should + // treat the presence of this key as a non-fatal issue. + // Log an error or warning indicating an unexpected field intended for the + // Profiling signal and process the data as if this value were absent or + // empty, ignoring its semantic content for the non-Profiling signal. + // + // Status: [Alpha] + // + // [Since v1.10.0] + int32 key_strindex = 3; } // InstrumentationScope is a message representing the instrumentation scope information // such as the fully qualified name and version. message InstrumentationScope { + // A name denoting the Instrumentation scope. // An empty instrumentation scope name means the name is unknown. string name = 1; + + // Defines the version of the instrumentation scope. + // An empty instrumentation scope version means the version is unknown. string version = 2; + + // Additional attributes that describe the scope. [Optional]. + // Attribute keys MUST be unique (it is not allowed to have more than one + // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated KeyValue attributes = 3; + + // The number of attributes that were discarded. Attributes + // can be discarded because their keys are too long or because there are too many + // attributes. If this value is 0, then no attributes were dropped. uint32 dropped_attributes_count = 4; } + +// A reference to an Entity. +// Entity represents an object of interest associated with produced telemetry: e.g spans, metrics, profiles, or logs. +// +// Status: [Development] +// +// [Since v1.6.0] +message EntityRef { + // The Schema URL, if known. This is the identifier of the Schema that the entity data + // is recorded in. To learn more about Schema URL see + // https://opentelemetry.io/docs/specs/otel/schemas/#schema-url + // + // This schema_url applies to the data in this message and to the Resource attributes + // referenced by id_keys and description_keys. + // TODO: discuss if we are happy with this somewhat complicated definition of what + // the schema_url applies to. + // + // This field obsoletes the schema_url field in ResourceMetrics/ResourceSpans/ResourceLogs. + string schema_url = 1; + + // Defines the type of the entity. MUST not change during the lifetime of the entity. + // For example: "service" or "host". This field is required and MUST not be empty + // for valid entities. + string type = 2; + + // Attribute Keys that identify the entity. + // MUST not change during the lifetime of the entity. The Id must contain at least one attribute. + // These keys MUST exist in the containing {message}.attributes. + repeated string id_keys = 3; + + // Descriptive (non-identifying) attribute keys of the entity. + // MAY change over the lifetime of the entity. MAY be empty. + // These attribute keys are not part of entity's identity. + // These keys MUST exist in the containing {message}.attributes. + repeated string description_keys = 4; +} \ No newline at end of file diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/logs/v1/logs.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/logs/v1/logs.proto index 9d0e376cad93..612bdadeacf2 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/logs/v1/logs.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/logs/v1/logs.proto @@ -55,6 +55,10 @@ message ResourceLogs { // A list of ScopeLogs that originate from a resource. repeated ScopeLogs scope_logs = 2; + // The Schema URL, if known. This is the identifier of the Schema that the resource data + // is recorded in. Notably, the last part of the URL path is the version number of the + // schema: http[s]://server[:port]/path/. To learn more about Schema URL see + // https://opentelemetry.io/docs/specs/otel/schemas/#schema-url // This schema_url applies to the data in the "resource" field. It does not apply // to the data in the "scope_logs" field which have their own schema_url field. string schema_url = 3; @@ -70,13 +74,17 @@ message ScopeLogs { // A list of log records. repeated LogRecord log_records = 2; - // This schema_url applies to all logs in the "logs" field. + // The Schema URL, if known. This is the identifier of the Schema that the log data + // is recorded in. Notably, the last part of the URL path is the version number of the + // schema: http[s]://server[:port]/path/. To learn more about Schema URL see + // https://opentelemetry.io/docs/specs/otel/schemas/#schema-url + // This schema_url applies to the data in the "scope" field and all logs in the + // "log_records" field. string schema_url = 3; } // Possible values for LogRecord.SeverityNumber. enum SeverityNumber { - // UNSPECIFIED is the default SeverityNumber, it MUST NOT be used. SEVERITY_NUMBER_UNSPECIFIED = 0; SEVERITY_NUMBER_TRACE = 1; SEVERITY_NUMBER_TRACE2 = 2; @@ -104,10 +112,23 @@ enum SeverityNumber { SEVERITY_NUMBER_FATAL4 = 24; } -// Masks for LogRecord.flags field. +// LogRecordFlags represents constants used to interpret the +// LogRecord.flags field, which is protobuf 'fixed32' type and is to +// be used as bit-fields. Each non-zero value defined in this enum is +// a bit-mask. To extract the bit-field, for example, use an +// expression like: +// +// (logRecord.flags & LOG_RECORD_FLAGS_TRACE_FLAGS_MASK) +// enum LogRecordFlags { - LOG_RECORD_FLAG_UNSPECIFIED = 0; - LOG_RECORD_FLAG_TRACE_FLAGS_MASK = 0x000000FF; + // The zero value for the enum. Should not be used for comparisons. + // Instead use bitwise "and" with the appropriate mask as shown above. + LOG_RECORD_FLAGS_DO_NOT_USE = 0; + + // Bits 0-7 are used for trace flags. + LOG_RECORD_FLAGS_TRACE_FLAGS_MASK = 0x000000FF; + + // Bits 8-31 are reserved for future use. } // A log record according to OpenTelemetry Log Data Model: @@ -153,6 +174,7 @@ message LogRecord { // Additional attributes that describe the specific event occurrence. [Optional]. // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 6; uint32 dropped_attributes_count = 7; @@ -160,18 +182,47 @@ message LogRecord { // defined in W3C Trace Context specification. 24 most significant bits are reserved // and must be set to 0. Readers must not assume that 24 most significant bits // will be zero and must correctly mask the bits when reading 8-bit trace flag (use - // flags & TRACE_FLAGS_MASK). [Optional]. + // flags & LOG_RECORD_FLAGS_TRACE_FLAGS_MASK). [Optional]. fixed32 flags = 8; // A unique identifier for a trace. All logs from the same trace share - // the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes - // is considered invalid. Can be set for logs that are part of request processing - // and have an assigned trace id. [Optional]. + // the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes OR + // of length other than 16 bytes is considered invalid (empty string in OTLP/JSON + // is zero-length and thus is also invalid). + // + // This field is optional. + // + // The receivers SHOULD assume that the log record is not associated with a + // trace if any of the following is true: + // - the field is not present, + // - the field contains an invalid value. bytes trace_id = 9; // A unique identifier for a span within a trace, assigned when the span - // is created. The ID is an 8-byte array. An ID with all zeroes is considered - // invalid. Can be set for logs that are part of a particular processing span. - // If span_id is present trace_id SHOULD be also present. [Optional]. + // is created. The ID is an 8-byte array. An ID with all zeroes OR of length + // other than 8 bytes is considered invalid (empty string in OTLP/JSON + // is zero-length and thus is also invalid). + // + // This field is optional. If the sender specifies a valid span_id then it SHOULD also + // specify a valid trace_id. + // + // The receivers SHOULD assume that the log record is not associated with a + // span if any of the following is true: + // - the field is not present, + // - the field contains an invalid value. bytes span_id = 10; + + // A unique identifier of event category/type. + // All events with the same event_name are expected to conform to the same + // schema for both their attributes and their body. + // + // Recommended to be fully qualified and short (no longer than 256 characters). + // + // Presence of event_name on the log record identifies this record + // as an event. + // + // [Optional]. + // + // [Since v1.5.0] + string event_name = 12; } diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/metrics/v1/metrics.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/metrics/v1/metrics.proto index 101c8ccbbf56..3f06bb83d161 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/metrics/v1/metrics.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/metrics/v1/metrics.proto @@ -29,6 +29,24 @@ option go_package = "go.opentelemetry.io/proto/otlp/metrics/v1"; // storage, OR can be embedded by other protocols that transfer OTLP metrics // data but do not implement the OTLP protocol. // +// MetricsData +// └─── ResourceMetrics +// ├── Resource +// ├── SchemaURL +// └── ScopeMetrics +// ├── Scope +// ├── SchemaURL +// └── Metric +// ├── Name +// ├── Description +// ├── Unit +// └── data +// ├── Gauge +// ├── Sum +// ├── Histogram +// ├── ExponentialHistogram +// └── Summary +// // The main difference between this message and collector protocol is that // in this message there will not be any "control" or "metadata" specific to // OTLP protocol. @@ -55,6 +73,10 @@ message ResourceMetrics { // A list of metrics that originate from a resource. repeated ScopeMetrics scope_metrics = 2; + // The Schema URL, if known. This is the identifier of the Schema that the resource data + // is recorded in. Notably, the last part of the URL path is the version number of the + // schema: http[s]://server[:port]/path/. To learn more about Schema URL see + // https://opentelemetry.io/docs/specs/otel/schemas/#schema-url // This schema_url applies to the data in the "resource" field. It does not apply // to the data in the "scope_metrics" field which have their own schema_url field. string schema_url = 3; @@ -70,7 +92,12 @@ message ScopeMetrics { // A list of metrics that originate from an instrumentation library. repeated Metric metrics = 2; - // This schema_url applies to all metrics in the "metrics" field. + // The Schema URL, if known. This is the identifier of the Schema that the metric data + // is recorded in. Notably, the last part of the URL path is the version number of the + // schema: http[s]://server[:port]/path/. To learn more about Schema URL see + // https://opentelemetry.io/docs/specs/otel/schemas/#schema-url + // This schema_url applies to the data in the "scope" field and all metrics in the + // "metrics" field. string schema_url = 3; } @@ -79,7 +106,6 @@ message ScopeMetrics { // // https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/data-model.md // -// // The data model and relation between entities is shown in the // diagram below. Here, "DataPoint" is the term used to refer to any // one of the specific data point value types, and "points" is the term used @@ -91,7 +117,7 @@ message ScopeMetrics { // - DataPoint contains timestamps, attributes, and one of the possible value type // fields. // -// Metric +// Metric // +------------+ // |name | // |description | @@ -162,18 +188,18 @@ message ScopeMetrics { message Metric { reserved 4, 6, 8; - // name of the metric, including its DNS name prefix. It must be unique. + // The name of the metric. string name = 1; - // description of the metric, which can be used in documentation. + // A description of the metric, which can be used in documentation. string description = 2; - // unit in which the metric value is reported. Follows the format - // described by http://unitsofmeasure.org/ucum.html. + // The unit in which the metric value is reported. Follows the format + // described by https://ucum.org/ucum and https://units-of-measurement.org/ string unit = 3; // Data determines the aggregation type (if any) of the metric, what is the - // reported value type for the data points, as well as the relatationship to + // reported value type for the data points, as well as the relationship to // the time interval over which they are reported. oneof data { Gauge gauge = 5; @@ -182,6 +208,18 @@ message Metric { ExponentialHistogram exponential_histogram = 10; Summary summary = 11; } + + // Additional metadata attributes that describe the metric. [Optional]. + // Attributes are non-identifying. + // Consumers SHOULD NOT need to be aware of these attributes. + // These attributes MAY be used to encode information allowing + // for lossless roundtrip translation to / from another data model. + // Attribute keys MUST be unique (it is not allowed to have more than one + // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. + // + // [Since v1.2.0] + repeated opentelemetry.proto.common.v1.KeyValue metadata = 12; } // Gauge represents the type of a scalar metric that always exports the @@ -194,25 +232,31 @@ message Metric { // AggregationTemporality is not included. Consequently, this also means // "StartTimeUnixNano" is ignored for all data points. message Gauge { + // The time series data points. + // Note: Multiple time series may be included (same timestamp, different attributes). repeated NumberDataPoint data_points = 1; } // Sum represents the type of a scalar metric that is calculated as a sum of all // reported measurements over a time interval. message Sum { + // The time series data points. + // Note: Multiple time series may be included (same timestamp, different attributes). repeated NumberDataPoint data_points = 1; // aggregation_temporality describes if the aggregator reports delta changes // since last report time, or cumulative changes since a fixed start time. AggregationTemporality aggregation_temporality = 2; - // If "true" means that the sum is monotonic. + // Represents whether the sum is monotonic. bool is_monotonic = 3; } // Histogram represents the type of a metric that is calculated by aggregating // as a Histogram of all reported measurements over a time interval. message Histogram { + // The time series data points. + // Note: Multiple time series may be included (same timestamp, different attributes). repeated HistogramDataPoint data_points = 1; // aggregation_temporality describes if the aggregator reports delta changes @@ -223,6 +267,8 @@ message Histogram { // ExponentialHistogram represents the type of a metric that is calculated by aggregating // as a ExponentialHistogram of all reported double measurements over a time interval. message ExponentialHistogram { + // The time series data points. + // Note: Multiple time series may be included (same timestamp, different attributes). repeated ExponentialHistogramDataPoint data_points = 1; // aggregation_temporality describes if the aggregator reports delta changes @@ -232,11 +278,16 @@ message ExponentialHistogram { // Summary metric data are used to convey quantile summaries, // a Prometheus (see: https://prometheus.io/docs/concepts/metric_types/#summary) -// and OpenMetrics (see: https://github.com/OpenObservability/OpenMetrics/blob/4dbf6075567ab43296eed941037c12951faafb92/protos/prometheus.proto#L45) +// and OpenMetrics (see: https://github.com/prometheus/OpenMetrics/blob/4dbf6075567ab43296eed941037c12951faafb92/protos/prometheus.proto#L45) // data type. These data points cannot always be merged in a meaningful way. // While they can be useful in some applications, histogram data points are // recommended for new applications. +// Summary metrics do not have an aggregation temporality field. This is +// because the count and sum fields of a SummaryDataPoint are assumed to be +// cumulative values. message Summary { + // The time series data points. + // Note: Multiple time series may be included (same timestamp, different attributes). repeated SummaryDataPoint data_points = 1; } @@ -316,15 +367,17 @@ enum AggregationTemporality { // enum is a bit-mask. To test the presence of a single flag in the flags of // a data point, for example, use an expression like: // -// (point.flags & FLAG_NO_RECORDED_VALUE) == FLAG_NO_RECORDED_VALUE +// (point.flags & DATA_POINT_FLAGS_NO_RECORDED_VALUE_MASK) == DATA_POINT_FLAGS_NO_RECORDED_VALUE_MASK // enum DataPointFlags { - FLAG_NONE = 0; + // The zero value for the enum. Should not be used for comparisons. + // Instead use bitwise "and" with the appropriate mask as shown above. + DATA_POINT_FLAGS_DO_NOT_USE = 0; // This DataPoint is valid but has no recorded value. This value // SHOULD be used to reflect explicitly missing data in a series, as // for an equivalent to the Prometheus "staleness marker". - FLAG_NO_RECORDED_VALUE = 1; + DATA_POINT_FLAGS_NO_RECORDED_VALUE_MASK = 1; // Bits 2-31 are reserved for future use. } @@ -338,6 +391,7 @@ message NumberDataPoint { // where this point belongs. The list may be empty (may contain 0 elements). // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 7; // StartTimeUnixNano is optional but strongly encouraged, see the @@ -386,6 +440,7 @@ message HistogramDataPoint { // where this point belongs. The list may be empty (may contain 0 elements). // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 9; // StartTimeUnixNano is optional but strongly encouraged, see the @@ -413,7 +468,7 @@ message HistogramDataPoint { // events, and is assumed to be monotonic over the values of these events. // Negative events *can* be recorded, but sum should not be filled out when // doing so. This is specifically to enforce compatibility w/ OpenMetrics, - // see: https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#histogram + // see: https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#histogram optional double sum = 5; // bucket_counts is an optional field contains the count values of histogram @@ -422,7 +477,9 @@ message HistogramDataPoint { // The sum of the bucket_counts must equal the value in the count field. // // The number of elements in bucket_counts array must be by one greater than - // the number of elements in explicit_bounds array. + // the number of elements in explicit_bounds array. The exception to this rule + // is when the length of bucket_counts is 0, then the length of explicit_bounds + // must also be 0. repeated fixed64 bucket_counts = 6; // explicit_bounds specifies buckets with explicitly defined bounds for values. @@ -438,6 +495,9 @@ message HistogramDataPoint { // Histogram buckets are inclusive of their upper boundary, except the last // bucket where the boundary is at infinity. This format is intentionally // compatible with the OpenMetrics histogram definition. + // + // If bucket_counts length is 0 then explicit_bounds length must also be 0, + // otherwise the data point is invalid. repeated double explicit_bounds = 7; // (Optional) List of exemplars collected from @@ -465,6 +525,7 @@ message ExponentialHistogramDataPoint { // where this point belongs. The list may be empty (may contain 0 elements). // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 1; // StartTimeUnixNano is optional but strongly encouraged, see the @@ -480,21 +541,21 @@ message ExponentialHistogramDataPoint { // 1970. fixed64 time_unix_nano = 3; - // count is the number of values in the population. Must be + // The number of values in the population. Must be // non-negative. This value must be equal to the sum of the "bucket_counts" // values in the positive and negative Buckets plus the "zero_count" field. fixed64 count = 4; - // sum of the values in the population. If count is zero then this field + // The sum of the values in the population. If count is zero then this field // must be zero. // // Note: Sum should only be filled out when measuring non-negative discrete // events, and is assumed to be monotonic over the values of these events. // Negative events *can* be recorded, but sum should not be filled out when // doing so. This is specifically to enforce compatibility w/ OpenMetrics, - // see: https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#histogram + // see: https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#histogram optional double sum = 5; - + // scale describes the resolution of the histogram. Boundaries are // located at powers of the base, where: // @@ -512,7 +573,7 @@ message ExponentialHistogramDataPoint { // values depend on the range of the data. sint32 scale = 6; - // zero_count is the count of values that are either exactly zero or + // The count of values that are either exactly zero or // within the region considered zero by the instrumentation at the // tolerated degree of precision. This bucket stores values that // cannot be expressed using the standard exponential formula as @@ -531,14 +592,14 @@ message ExponentialHistogramDataPoint { // Buckets are a set of bucket counts, encoded in a contiguous array // of counts. message Buckets { - // Offset is the bucket index of the first entry in the bucket_counts array. - // + // The bucket index of the first entry in the bucket_counts array. + // // Note: This uses a varint encoding as a simple form of compression. sint32 offset = 1; - // Count is an array of counts, where count[i] carries the count - // of the bucket at index (offset+i). count[i] is the count of - // values greater than base^(offset+i) and less or equal to than + // An array of count values, where bucket_counts[i] carries + // the count of the bucket at index (offset+i). bucket_counts[i] is the count + // of values greater than base^(offset+i) and less than or equal to // base^(offset+i+1). // // Note: By contrast, the explicit HistogramDataPoint uses @@ -546,7 +607,7 @@ message ExponentialHistogramDataPoint { // especially zeros, so uint64 has been selected to ensure // varint encoding. repeated uint64 bucket_counts = 2; - } + } // Flags that apply to this specific data point. See DataPointFlags // for the available flags and their meaning. @@ -556,15 +617,24 @@ message ExponentialHistogramDataPoint { // measurements that were used to form the data point repeated Exemplar exemplars = 11; - // min is the minimum value over (start_time, end_time]. + // The minimum value over (start_time, end_time]. optional double min = 12; - // max is the maximum value over (start_time, end_time]. + // The maximum value over (start_time, end_time]. optional double max = 13; + + // ZeroThreshold may be optionally set to convey the width of the zero + // region. Where the zero region is defined as the closed interval + // [-ZeroThreshold, ZeroThreshold]. + // When ZeroThreshold is 0, zero count bucket stores values that cannot be + // expressed using the standard exponential formula as well as values that + // have been rounded to zero. + double zero_threshold = 14; } // SummaryDataPoint is a single data point in a timeseries that describes the -// time-varying values of a Summary metric. +// time-varying values of a Summary metric. The count and sum fields represent +// cumulative values. message SummaryDataPoint { reserved 1; @@ -572,6 +642,7 @@ message SummaryDataPoint { // where this point belongs. The list may be empty (may contain 0 elements). // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 7; // StartTimeUnixNano is optional but strongly encouraged, see the @@ -597,7 +668,7 @@ message SummaryDataPoint { // events, and is assumed to be monotonic over the values of these events. // Negative events *can* be recorded, but sum should not be filled out when // doing so. This is specifically to enforce compatibility w/ OpenMetrics, - // see: https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#summary + // see: https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#summary double sum = 5; // Represents the value at a given quantile of a distribution. diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/resource/v1/resource.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/resource/v1/resource.proto index 6637560bc354..118bfed175d5 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/resource/v1/resource.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/resource/v1/resource.proto @@ -29,9 +29,19 @@ message Resource { // Set of attributes that describe the resource. // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 1; - // dropped_attributes_count is the number of dropped attributes. If the value is 0, then + // The number of dropped attributes. If the value is 0, then // no attributes were dropped. uint32 dropped_attributes_count = 2; + + // Set of entities that participate in this Resource. + // + // Note: keys in the references MUST exist in attributes of this message. + // + // Status: [Development] + // + // [Since v1.6.0] + repeated opentelemetry.proto.common.v1.EntityRef entity_refs = 3; } diff --git a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/trace/v1/trace.proto b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/trace/v1/trace.proto index 5903550742dc..235b54e8e78b 100644 --- a/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/trace/v1/trace.proto +++ b/test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/trace/v1/trace.proto @@ -55,6 +55,10 @@ message ResourceSpans { // A list of ScopeSpans that originate from a resource. repeated ScopeSpans scope_spans = 2; + // The Schema URL, if known. This is the identifier of the Schema that the resource data + // is recorded in. Notably, the last part of the URL path is the version number of the + // schema: http[s]://server[:port]/path/. To learn more about Schema URL see + // https://opentelemetry.io/docs/specs/otel/schemas/#schema-url // This schema_url applies to the data in the "resource" field. It does not apply // to the data in the "scope_spans" field which have their own schema_url field. string schema_url = 3; @@ -70,7 +74,12 @@ message ScopeSpans { // A list of Spans that originate from an instrumentation scope. repeated Span spans = 2; - // This schema_url applies to all spans and span events in the "spans" field. + // The Schema URL, if known. This is the identifier of the Schema that the span data + // is recorded in. Notably, the last part of the URL path is the version number of the + // schema: http[s]://server[:port]/path/. To learn more about Schema URL see + // https://opentelemetry.io/docs/specs/otel/schemas/#schema-url + // This schema_url applies to the data in the "scope" field and all spans and span + // events in the "spans" field. string schema_url = 3; } @@ -79,21 +88,17 @@ message ScopeSpans { // The next available field id is 17. message Span { // A unique identifier for a trace. All spans from the same trace share - // the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes - // is considered invalid. - // - // This field is semantically required. Receiver should generate new - // random trace_id if empty or invalid trace_id was received. + // the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes OR + // of length other than 16 bytes is considered invalid (empty string in OTLP/JSON + // is zero-length and thus is also invalid). // // This field is required. bytes trace_id = 1; // A unique identifier for a span within a trace, assigned when the span - // is created. The ID is an 8-byte array. An ID with all zeroes is considered - // invalid. - // - // This field is semantically required. Receiver should generate new - // random span_id if empty or invalid span_id was received. + // is created. The ID is an 8-byte array. An ID with all zeroes OR of length + // other than 8 bytes is considered invalid (empty string in OTLP/JSON + // is zero-length and thus is also invalid). // // This field is required. bytes span_id = 2; @@ -107,6 +112,31 @@ message Span { // field must be empty. The ID is an 8-byte array. bytes parent_span_id = 4; + // Flags, a bit field. + // + // Bits 0-7 (8 least significant bits) are the trace flags as defined in W3C Trace + // Context specification. To read the 8-bit W3C trace flag, use + // `flags & SPAN_FLAGS_TRACE_FLAGS_MASK`. + // + // See https://www.w3.org/TR/trace-context-2/#trace-flags for the flag definitions. + // + // Bits 8 and 9 represent the 3 states of whether a span's parent + // is remote. The states are (unknown, is not remote, is remote). + // To read whether the value is known, use `(flags & SPAN_FLAGS_CONTEXT_HAS_IS_REMOTE_MASK) != 0`. + // To read whether the span is remote, use `(flags & SPAN_FLAGS_CONTEXT_IS_REMOTE_MASK) != 0`. + // + // When creating span messages, if the message is logically forwarded from another source + // with an equivalent flags fields (i.e., usually another OTLP span message), the field SHOULD + // be copied as-is. If creating from a source that does not have an equivalent flags field + // (such as a runtime representation of an OpenTelemetry span), the high 22 bits MUST + // be set to zero. + // Readers MUST NOT assume that bits 10-31 (22 most significant bits) will be zero. + // + // [Optional]. + // + // [Since v1.1.0] + fixed32 flags = 16; + // A description of the span's operation. // // For example, the name can be a qualified method name or a file name @@ -155,7 +185,7 @@ message Span { // and `SERVER` (callee) to identify queueing latency associated with the span. SpanKind kind = 6; - // start_time_unix_nano is the start time of the span. On the client side, this is the time + // The start time of the span. On the client side, this is the time // kept by the local machine where the span execution starts. On the server side, this // is the time when the server's application handler starts running. // Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970. @@ -163,7 +193,7 @@ message Span { // This field is semantically required and it is expected that end_time >= start_time. fixed64 start_time_unix_nano = 7; - // end_time_unix_nano is the end time of the span. On the client side, this is the time + // The end time of the span. On the client side, this is the time // kept by the local machine where the span execution ends. On the server side, this // is the time when the server application handler stops running. // Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970. @@ -171,21 +201,20 @@ message Span { // This field is semantically required and it is expected that end_time >= start_time. fixed64 end_time_unix_nano = 8; - // attributes is a collection of key/value pairs. Note, global attributes - // like server name can be set using the resource API. Examples of attributes: + // A collection of key/value pairs. Note, global attributes + // like service name can be set using the resource API. Examples of attributes: // - // "/http/user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" - // "/http/server_latency": 300 - // "abc.com/myattribute": true - // "abc.com/score": 10.239 + // "user_agent.original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" + // "http.response.status_code": 200 + // "com.example.myattribute": true + // "com.example.score": 10.239 // - // The OpenTelemetry API specification further restricts the allowed value types: - // https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/README.md#attribute // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 9; - // dropped_attributes_count is the number of attributes that were discarded. Attributes + // The number of attributes that were discarded. Attributes // can be discarded because their keys are too long or because there are too many // attributes. If this value is 0, then no attributes were dropped. uint32 dropped_attributes_count = 10; @@ -193,27 +222,28 @@ message Span { // Event is a time-stamped annotation of the span, consisting of user-supplied // text description and key-value pairs. message Event { - // time_unix_nano is the time the event occurred. + // The time the event occurred. fixed64 time_unix_nano = 1; - // name of the event. + // The name of the event. // This field is semantically required to be set to non-empty string. string name = 2; - // attributes is a collection of attribute key/value pairs on the event. + // A collection of attribute key/value pairs on the event. // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 3; - // dropped_attributes_count is the number of dropped attributes. If the value is 0, + // The number of dropped attributes. If the value is 0, // then no attributes were dropped. uint32 dropped_attributes_count = 4; } - // events is a collection of Event items. + // A collection of Event items. repeated Event events = 11; - // dropped_events_count is the number of dropped events. If the value is 0, then no + // The number of dropped events. If the value is 0, then no // events were dropped. uint32 dropped_events_count = 12; @@ -232,21 +262,43 @@ message Span { // The trace_state associated with the link. string trace_state = 3; - // attributes is a collection of attribute key/value pairs on the link. + // A collection of attribute key/value pairs on the link. // Attribute keys MUST be unique (it is not allowed to have more than one // attribute with the same key). + // The behavior of software that receives duplicated keys can be unpredictable. repeated opentelemetry.proto.common.v1.KeyValue attributes = 4; - // dropped_attributes_count is the number of dropped attributes. If the value is 0, + // The number of dropped attributes. If the value is 0, // then no attributes were dropped. uint32 dropped_attributes_count = 5; + + // Flags, a bit field. + // + // Bits 0-7 (8 least significant bits) are the trace flags as defined in W3C Trace + // Context specification. To read the 8-bit W3C trace flag, use + // `flags & SPAN_FLAGS_TRACE_FLAGS_MASK`. + // + // See https://www.w3.org/TR/trace-context-2/#trace-flags for the flag definitions. + // + // Bits 8 and 9 represent the 3 states of whether the link is remote. + // The states are (unknown, is not remote, is remote). + // To read whether the value is known, use `(flags & SPAN_FLAGS_CONTEXT_HAS_IS_REMOTE_MASK) != 0`. + // To read whether the link is remote, use `(flags & SPAN_FLAGS_CONTEXT_IS_REMOTE_MASK) != 0`. + // + // Readers MUST NOT assume that bits 10-31 (22 most significant bits) will be zero. + // When creating new spans, bits 10-31 (most-significant 22-bits) MUST be zero. + // + // [Optional]. + // + // [Since v1.1.0] + fixed32 flags = 6; } - // links is a collection of Links, which are references from this span to a span + // A collection of Links, which are references from this span to a span // in the same or different trace. repeated Link links = 13; - // dropped_links_count is the number of dropped links after the maximum size was + // The number of dropped links after the maximum size was // enforced. If this value is 0, then no links were dropped. uint32 dropped_links_count = 14; @@ -278,3 +330,39 @@ message Status { // The status code. StatusCode code = 3; } + +// SpanFlags represents constants used to interpret the +// Span.flags field, which is protobuf 'fixed32' type and is to +// be used as bit-fields. Each non-zero value defined in this enum is +// a bit-mask. To extract the bit-field, for example, use an +// expression like: +// +// (span.flags & SPAN_FLAGS_TRACE_FLAGS_MASK) +// +// See https://www.w3.org/TR/trace-context-2/#trace-flags for the flag definitions. +// +// Note that Span flags were introduced in version 1.1 of the +// OpenTelemetry protocol. Older Span producers do not set this +// field, consequently consumers should not rely on the absence of a +// particular flag bit to indicate the presence of a particular feature. +// +// [Since v1.1.0] +enum SpanFlags { + // The zero value for the enum. Should not be used for comparisons. + // Instead use bitwise "and" with the appropriate mask as shown above. + SPAN_FLAGS_DO_NOT_USE = 0; + + // Bits 0-7 are used for trace flags. + SPAN_FLAGS_TRACE_FLAGS_MASK = 0x000000FF; + + // Bits 8 and 9 are used to indicate that the parent span or link span is remote. + // Bit 8 (`HAS_IS_REMOTE`) indicates whether the value is known. + // Bit 9 (`IS_REMOTE`) indicates whether the span or link is remote. + // + // [Since v1.2.0] + SPAN_FLAGS_CONTEXT_HAS_IS_REMOTE_MASK = 0x00000100; + // [Since v1.2.0] + SPAN_FLAGS_CONTEXT_IS_REMOTE_MASK = 0x00000200; + + // Bits 10-31 are reserved for future use. +}