This example demonstrates how to instrument a Java Application on Azure App Service using OpenTelemetry, and export traces, metrics, and logs to a collector running in Azure, which will then export that data to Splunk.
We'll show how the splunk-otel-javaagent.jar file can be uploaded to Azure and added to the application startup command.
The example application is based on the default Spring Boot web application template, and the deployment steps are based on Quickstart: Create a Java app on Azure App Service.
The following tools are required to build and deploy the Java application to Azure:
- Java 17+ JDK
- Maven
- An Azure account with permissions to create and execute Azure web applications
- Azure CLI
For this example, we deployed the Splunk Distribution of the OpenTelemetry Collector onto a virtual machine in Azure using Gateway mode, and ensured it's accessible to our Azure function.
We configured it with the SPLUNK_HEC_TOKEN and SPLUNK_HEC_URL environment variables, so that it
exports logs to our Splunk Cloud instance.
Please refer to Install the Collector using packages and deployment tools for collector installation instructions.
Open a command line terminal and use the following command to authenticate with Azure:
az loginWe'll use the Maven plugin to deploy our Java application to Azure. In order to do this, we'll need to configure it first with the following command:
mvn com.microsoft.azure:azure-webapp-maven-plugin:2.13.0:configWhen prompted, provide the following inputs:
- Create new configuration:
Y - Define value for OS:
2(for Linux) - Define value for javaVersion:
1(for Java 17) - Define value for pricingTier:
3(for P1v2) - Confirm:
Y
Let's build the application with Maven:
mvn clean packageAnd then deploy it to Azure:
mvn azure-webapp:deployPoint your browser to http://<appName>.azurewebsites.net/greeting to test the application.
Note: you'll need to substitute your application name into the above URL, which was provided as an output from the earlier Maven command used to deploy the application.
You should see output such as the following:
{"id":1,"content":"Hello, World!"}
Let's download the latest splunk-otel-javaagent.jar file, which we'll use to instrument our Java application:
curl -L https://github.com/signalfx/splunk-otel-java/releases/latest/download/splunk-otel-javaagent.jar \
-o splunk-otel-javaagent.jar
To instrument our Java application running in Azure with OpenTelemetry, we'll
need to upload the splunk-otel-javaagent.jar file to Azure with the following
command:
az webapp deploy --src-path splunk-otel-javaagent.jar \
--type static \
--target-path /home/site/wwwroot/splunk-otel-javaagent.jar \
--resource-group <resource group name> \
--name <app name>Note: you'll need to substitute the resource group name and app name for your application before running the above command.
Next, we'll set the following environment variables, which tells splunk-otel-javaagent.jar
what data to collect and where to send it:
JAVA_OPTS:-javaagent:/home/site/wwwroot/splunk-otel-javaagent.jarOTEL_SERVICE_NAME:azure-java-web-appOTEL_EXPORTER_OTLP_ENDPOINT:http(s)://<collector URL>:4318OTEL_RESOURCE_ATTRIBUTES:deployment.environment=testSPLUNK_PROFILER_ENABLED:trueSPLUNK_PROFILER_MEMORY_ENABLED:true
Note: remember to substitute the URL for your collector
The environment variables should look like the following in Azure:
The approach used to upload splunk-otel-javaagent.jar to Azure and activate it by setting the JAVA_OPTS environment
variable is based on the steps in Manually upgrade the monitoring extension/agent.
Important: Setting
JAVA_OPTSrequires Application Insights to be disabled in the Azure portal for the application being monitored. If this is not possible, then follow the steps in Manually upgrade the monitoring extension/agent to include-javaagent:/home/site/wwwroot/splunk-otel-javaagent.jarin the Startup Command instead.
Note that we've enabled both the CPU and memory profiler, so we can utilize the AlwaysOn Profiling capabilities.
Access the application by navigating your web browser to the following URL:
http://<appName>.azurewebsites.net/greeting
As before, you should see output such as the following:
{"id":1,"content":"Hello, World!"}
After a minute or so, you should start to see traces for the Java application appearing in Splunk Observability Cloud:
You should also see profiling data appear:
Metrics are collected by splunk-otel-javaagent.jar automatically. For example,
the jvm.memory.used metric shows us the amount of memory used in the JVM
by type of memory:
The Splunk Distribution of OpenTelemetry Java automatically adds trace context to logs. However, it doesn't add this context to the actual log file (unless you explicitly configure the logging framework to do so). Instead, the trace context is added behind the scenes to the log events exported to the OpenTelemetry Collector.
For example, if we add the debug exporter to the logs pipeline of the collector, we can see that the trace_id and span_id have been added to the following log event for our application:
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: ScopeLogs #0
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: ScopeLogs SchemaURL:
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: InstrumentationScope org.springframework.web.servlet.DispatcherServlet
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: LogRecord #0
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: ObservedTimestamp: 2025-02-10 16:15:59.063905628 +0000 UTC
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Timestamp: 2025-02-10 16:15:59.063822027 +0000 UTC
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: SeverityText: INFO
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: SeverityNumber: Info(9)
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Body: Str(Initializing Servlet 'dispatcherServlet')
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Trace ID: 551fa951d47e609c046e23dbef393c03
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Span ID: c62ad3bd39da28ec
Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Flags: 1
The OpenTelemetry Collector can be configured to export log data to Splunk platform using the Splunk HEC exporter. The logs can then be made available to Splunk Observability Cloud using Log Observer Connect. This will provide full correlation between spans generated by Java instrumentation with metrics and logs.