INVALID_ARGUMENT: One or more TimeSeries could not be written: Points must be written in order. #247
Comments
|
Hi @joao-melo-ingka, could you clarify what exactly you mean by 'this extension' ? Also, could you share your custom configuration ? I'm assuming your overall goal here is to auto-instrument your application using opentelemetry java agent and export the telemetry data to Google Cloud. |
|
Closing as obsolete. If you are still impacted, feel free to reopen and respond to #247 (comment) |
|
I faced a similar error with the Library Instrumentation for HikariCP:
In our case, exporter is running on Cloud Run written in Koltin/Ktor, and collector is also running on Cloud Run but using different service. The javaagent for OpenTelemetry auto-instrumentation is running in the service that exports HikariCP metrics. We use receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
send_batch_size: 8192
timeout: 10s
exporters:
googlecloud:
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [googlecloud]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [googlecloud]Here is diagram regarding relation among services: flowchart LR
subgraph CloudRun
subgraph exporting service
instance
another-instance
end
another-instance --> collector
instance --> collector
end
collector --> monitoring[Cloud Monitoring]
I expected that adding instance ID to metrics attributes solves this issue, so introduced Google Cloud official Resource Detectors, but it caused another That's all what I know in this timing. I'll update some when I found some progresses. |
|
@KengoTODA which monitored resource are your metrics being written to (the ones that succeed)? |
This means it is missing either service.name or service.instance.id. Are those both set? |
Probably not, because the instrumentation for HikariCP is not Google Cloud specific. And when I set them by Google Cloud official resource provider, I found the following error:
|
|
It seems that my current configuration is not enough to use the Google Cloud official Resource Detectors. Unlike Auto-Configuration for OpenTelemetry in Google Cloud, Resource Detectors are not designed as extension for javaagent, so I cannot enable them via As a trial, I enabled the Auto-Configuration for OpenTelemetry in Google Cloud as a javaagent extension, and it worked but metrics had no instance ID nor service ID in their attributes yet: I guess that what I need is the interaction regarding how to use Google Cloud official Resource Detectors with OpenTelemetry javaagent. Please share if you have some pointers or references. Thanks in advance! |
|
I'd like to share the current progress on my side:
I initially considered using the OpenTelemetry Java Agent but decided against it due to the following reasons:
Instead, I have opted for manual instrumentation to use resource detectors and the necessary instrumentations. I have confirmed that instance-specific metadata have already been set to the resource in metrics, and provided to the collector running on other Cloud Run service. For example: // Koin module for OpenTelemetry
private val openTelemetryModule = module {
single<OpenTelemetry> {
if (isDevelopmentEnv()) {
return@single OpenTelemetry.noop()
}
val resource = GCPResource().createResource(DefaultConfigProperties.create(mapOf()))
logger.debug("Detected attributes are: {}", resource.attributes)
return@single AutoConfiguredOpenTelemetrySdk.builder()
.addMeterProviderCustomizer { providerBuilder, _ ->
providerBuilder.addResource(resource)
}
// ...
.build()
.openTelemetrySdk
}
} |
I've been experimenting with this and found the following issue.
To reproduce I've just injected the opentelemetry agent + this extension + custom configuration to setup specific resource (global).
The text was updated successfully, but these errors were encountered: