MpTelemetry 2.0 - Distributed Tracing and Metrics API for epic #27108

[yasmin-aumeeruddy]

Feature epic details

• For the title of this issue, type: Documentation, Development epic name
• Link to development epic: MP70: Implement MicroProfile Telemetry 2.0 #27108
• Target GA release: 24.0.0.9

Operating systems

Does the documentation apply to all operating systems?

• Yes
• No; specify operating systems: ______

Summary

As an update to MicroProfile Telemetry 1.1, the MicroProfile Telemetry-2.0 feature provides the later version of the OpenTelemetry SDK, providing version 1.39 of OpenTelemetry. In addition, the feature provides additional functionality:

• Access to the OpenTelemetry Metrics API. [Provided in this issue]

• HTTP metrics Provided in 7459

• JVM runtime environment metrics Provided in 7459

• Runtime level logs Provided in 7466

• Application logs Provided in 7466

Configuring Open Liberty to use MicroProfile Telemetry

To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your server.xml file and specify system properties or environment variables to configure how MicroProfile Telemetry collects and exports traces, metrics and logs.

Add a version of the MicroProfile Telemetry feature to your server.xml file.

Enable tracing, metrics and logs and configure respective storage systems by specifying system properties or environment variables.

You can configure how MicroProfile Telemetry collects and exports traces, metrics and logs by specifying configuration properties in environment variables or system properties. If you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the otel.sdk.disabled=false property is equivalent to the OTEL_SDK_DISABLED=false environment variable.

OpenTelemetry is disabled by default. To enable the generation of traces, metrics and logs set the otel.sdk.disabled=false property.

Configure a storage system by specifying an exporter definition that includes the exporter type and the endpoint to connect to.

For example, to use a Jaeger server to send your traces, you might add configuration similar to the following example to your server.env file:

OTEL_SDK_DISABLED=false
OTEL_TRACES_EXPORTER=otlp
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317/

Alternatively, to use a Zipkin server, you might add configuration similar to the following example to your bootstrap.properties file:

OTEL_SDK_DISABLED=false
OTEL_TRACES_EXPORTER=zipkin
OTEL_EXPORTER_ZIPKIN_ENDPOINT=http://localhost:9411/api/v2/spans

If you want to export metrics to Open Liberty log files, set the following variable:

OTEL.TRACES.EXPORTER=logging

Depending on how you choose to instrument your application code for metrics, further configuration might be required.

If your runtime hosts multiple applications that require different configuration, you should use other MicroProfile Configuration sources. Be aware that this would not allow for runtime level metrics or logs. (See 7459 and 7466 )

Manual metrics

By default, all metric data is exported to OTLP. You can change each exporter with the following properties:

otel.metrics.exporter/OTEL_METRICS_EXPORTER you might add configuration similar to the following example to your server.env file:

OTEL_SDK_DISABLED=false
OTEL_METRICS_EXPORTER=otlp
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317/

Custom metrics can be defined by utilizing the metrics API as following example demonstrates:

class WithCounter {
    @Inject
    Meter meter;

    private LongCounter counter;

    @PostConstruct
    public void init() {
        counter = meter
                    .counterBuilder("new_subscriptions")
                    .setDescription("Number of new subscriptions")
                    .setUnit("1")
                    .build();
    }

    void subscribe(String plan) {
        counter.add(1,
            Attributes.of(AttributeKey.stringKey("plan"), plan));
    }
}

In this example Meter is used to define an instrument, in this case a Counter and application code then can record measurement values along with additional attributes. Measurement aggregations are computed separately for each unique combination of attributes.

For a full list of available metrics, see:

https://opentelemetry.io/docs/specs/otel/metrics/api/#meter-operations

Troubleshooting

You receive either of the CWMOT5006W or CWMOT5007 warning message that there is conflicting configuration for otel.sdk.disabled

Specify the settings to enable or disable OpenTelemetry instances by using either environment variables and system properties or other MP Config sources but not both. (If you see these warnings, the other MP Config source to look at is your server.xml file)

[dmuelle]

Hi @yasmin-aumeeruddy - I've revised the "Enable distributed tracing with MicroProfile Telemetry" page to be Enable observability with MicroProfile Telemetry and cover metrics logs and traces. I also add the troubleshooting item from your comment and updated MicroProfile Config properties: MicroProfile Telemetry, as well as other pages per #7459 and #7466.

When you have a chance, would you review Enable observability with MicroProfile Telemetry](https://docs-draft-openlibertyio.mqj6zf7jocq.us-south.codeengine.appdomain.cloud/docs/latest/microprofile-telemetry.html) and let me know if any further changes are needed? When you're satisfied with the update, you can add the technical reviewed label to this issue. Thanks!

[yasmin-aumeeruddy]

Hi @dmuelle

• MicroProfile Telemetry helps you collect n and analyze data on should be MicroProfile Telemetry helps you collect and analyze data on
• OpenTelemetry is disabled by default. To enable the generation of traces, set the otel.sdk.disabled=false property. should be OpenTelemetry is disabled by default. To enable the generation of traces, metrics and logs, set the otel.sdk.disabled=false property.
• For example, if you want to export traces to Open Liberty log files, set the following property: otel.traces.exporter=logging should now be For example, if you want to export traces to Open Liberty log files, set the following property: otel.traces.exporter=console

[dmuelle]

Hi @yasmin-aumeeruddy - thanks for reviewing. All correction made, though I did revise this section somewhat per Don's feedback in #7459.

Enable observability with MicroProfile Telemetry

Let me know if any further changes are needed- thanks!

[ramkumar-k-9286]

This option also supports configuration-compatibility with MicroProfile Telemetry 1.1 and earlier, which created only application-level SDK instances.
->
Should we be linking MicroProfile Telemetry 1.1

---

Also: configuration-compatibility -> configuration compatibility

---

set the otel.sdk.disabled=false property -> set the otel.sdk.disabled=false property
See it a couple of times

---

At runtime initialization, if the otel.sdk.disabled property is set to false
->
At runtime initialization, if the otel.sdk.disabled property is set to false

---

This configuration creates all telemetry from Application 1 with the service name A1, and from Application 2 with the service name A2. It omits all runtime-level telemetry.
->
This configuration creates all telemetry from Application 1 with the service name A1, and from Application 2 with the service name A2. It omits all runtime-level telemetry.

---

By default, the SimpleLogRecordProcessor is enabled, so the records are sent immediately.
->
By default, the SimpleLogRecordProcessor is enabled, so the records are sent immediately.

---

By default, metric data is exported at an interval of 60 seconds.
->
By default, metric data is exported at an interval of 60 seconds.

---