Skip to content

Replace BanyanDB Java client with native implementation. #13655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wu-sheng merged 11 commits into from
Jan 9, 2026
+10,570 −403
Merged

Replace BanyanDB Java client with native implementation. #13655

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
30a84a9
move banyandb-java-client to OAP
wankai123 Dec 29, 2025
dce737a
remove metadata cache
wankai123 Jan 5, 2026
5c25ac1
Merge branch 'master' of https://github.com/apache/skywalking into by…
wankai123 Jan 7, 2026
291fa5b
Replace BanyanDB Java client with native implementation
wankai123 Jan 7, 2026
637e253
fix sub-module
wankai123 Jan 7, 2026
7815895
fix java docs
wankai123 Jan 7, 2026
86ca168
fix java docs
wankai123 Jan 7, 2026
2783c26
fix e2e
wankai123 Jan 9, 2026
7e78cdf
remove bydb.dependencies.properties
wankai123 Jan 9, 2026
11662ad
Fix trace profiling query time range condition.
wankai123 Jan 9, 2026
ad81929
fix ut
wankai123 Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@
[submodule "skywalking-ui"]
path = skywalking-ui
url = https://github.com/apache/skywalking-booster-ui.git
[submodule "oap-server/server-library/library-banyandb-client/src/main/proto"]
path = oap-server/server-library/library-banyandb-client/src/main/proto
url = https://github.com/apache/skywalking-banyandb-client-proto
1 change: 1 addition & 0 deletions docs/en/changes/changes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
* Bump up netty to 4.2.9.Final.
* Add support for OpenSearch/ElasticSearch client certificate authentication.
* Fix BanyanDB logs paging query.
* Replace BanyanDB Java client with native implementation.

#### UI
* Fix the missing icon in new native trace view.
Expand Down
35 changes: 18 additions & 17 deletions docs/en/setup/backend/configuration-vocabulary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,23 +352,24 @@ The following table lists the configuration items:
### Global Configuration
The global settings for the whole BanyanDB:

| Settings | Value(s) and Explanation | System Environment Variable¹ | Default |
|---------------------------------|-------------------------------------------------------------------------------------------------------|---------------------------------------------------------|-----------------|
| targets | Hosts with ports of the BanyanDB. | SW_STORAGE_BANYANDB_TARGETS | 127.0.0.1:17912 |
| maxBulkSize | The maximum size of write entities in a single batch write call. | SW_STORAGE_BANYANDB_MAX_BULK_SIZE | 10000 |
| flushInterval | Period of flush interval. In the timeunit of seconds. | SW_STORAGE_BANYANDB_FLUSH_INTERVAL | 15 |
| flushTimeout | The timeout seconds of a bulk flush. | SW_STORAGE_BANYANDB_FLUSH_TIMEOUT | 10 |
| concurrentWriteThreads | Concurrent consumer threads for batch writing. | SW_STORAGE_BANYANDB_CONCURRENT_WRITE_THREADS | 15 |
| profileTaskQueryMaxSize | Max size of ProfileTask to be fetched. | SW_STORAGE_BANYANDB_PROFILE_TASK_QUERY_MAX_SIZE | 200 |
| resultWindowMaxSize | The maximum size of dataset when the OAP loads cache, such as network aliases. | SW_STORAGE_BANYAND_QUERY_MAX_WINDOW_SIZE | 10000 |
| metadataQueryMaxSize | The maximum size of metadata per query. | SW_STORAGE_BANYAND_QUERY_MAX_SIZE | 10000 |
| segmentQueryMaxSize | The maximum size of trace segments per query. | SW_STORAGE_BANYAND_QUERY_SEGMENT_SIZE | 200 |
| asyncProfilerTaskQueryMaxSize | Max size of AsyncProfilerTask to be fetched. | SW_STORAGE_BANYANDB_ASYNC_PROFILER_TASK_QUERY_MAX_SIZE | 200 |
| profileDataQueryScrollBatchSize | The batch size of query profiling data. | SW_STORAGE_BANYAND_QUERY_PROFILE_DATA_BATCH_SIZE | 100 |
| user | The username for BanyanDB authentication. | SW_STORAGE_BANYANDB_USER | - |
| password | The password for BanyanDB authentication. | SW_STORAGE_BANYANDB_PASSWORD | - |
| sslTrustCAPath | If the BanyanDB server is configured with TLS, config the TLS cert file path and open tls connection. | SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH | - |
| cleanupUnusedTopNRules | Cleanup TopN rules in BanyanDB server that are not configured in the bydb-topn.yml config. | SW_STORAGE_BANYANDB_CLEANUP_UNUSED_TOPN_RULES | true |
| Settings | Value(s) and Explanation | System Environment Variable¹ | Default |
|---------------------------------|-------------------------------------------------------------------------------------------------------|--------------------------------------------------------|-------------------|
| targets | Hosts with ports of the BanyanDB. | SW_STORAGE_BANYANDB_TARGETS | 127.0.0.1:17912 |
| maxBulkSize | The maximum size of write entities in a single batch write call. | SW_STORAGE_BANYANDB_MAX_BULK_SIZE | 10000 |
| flushInterval | Period of flush interval. In the timeunit of seconds. | SW_STORAGE_BANYANDB_FLUSH_INTERVAL | 15 |
| flushTimeout | The timeout seconds of a bulk flush. | SW_STORAGE_BANYANDB_FLUSH_TIMEOUT | 10 |
| concurrentWriteThreads | Concurrent consumer threads for batch writing. | SW_STORAGE_BANYANDB_CONCURRENT_WRITE_THREADS | 15 |
| profileTaskQueryMaxSize | Max size of ProfileTask to be fetched. | SW_STORAGE_BANYANDB_PROFILE_TASK_QUERY_MAX_SIZE | 200 |
| resultWindowMaxSize | The maximum size of dataset when the OAP loads cache, such as network aliases. | SW_STORAGE_BANYAND_QUERY_MAX_WINDOW_SIZE | 10000 |
| metadataQueryMaxSize | The maximum size of metadata per query. | SW_STORAGE_BANYAND_QUERY_MAX_SIZE | 10000 |
| segmentQueryMaxSize | The maximum size of trace segments per query. | SW_STORAGE_BANYAND_QUERY_SEGMENT_SIZE | 200 |
| asyncProfilerTaskQueryMaxSize | Max size of AsyncProfilerTask to be fetched. | SW_STORAGE_BANYANDB_ASYNC_PROFILER_TASK_QUERY_MAX_SIZE | 200 |
| profileDataQueryScrollBatchSize | The batch size of query profiling data. | SW_STORAGE_BANYAND_QUERY_PROFILE_DATA_BATCH_SIZE | 100 |
| user | The username for BanyanDB authentication. | SW_STORAGE_BANYANDB_USER | - |
| password | The password for BanyanDB authentication. | SW_STORAGE_BANYANDB_PASSWORD | - |
| sslTrustCAPath | If the BanyanDB server is configured with TLS, config the TLS cert file path and open tls connection. | SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH | - |
| cleanupUnusedTopNRules | Cleanup TopN rules in BanyanDB server that are not configured in the bydb-topn.yml config. | SW_STORAGE_BANYANDB_CLEANUP_UNUSED_TOPN_RULES | true |
| namespace | The namespace in BanyanDB to store the data of OAP. | SW_NAMESPACE | default |

### Group Configuration
The settings for each group:
Expand Down
7 changes: 6 additions & 1 deletion docs/en/setup/backend/storages/banyandb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,11 @@ global:
sslTrustCAPath: ${SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH:""}
# Cleanup TopN rules in BanyanDB server that are not configured in the bydb-topn.yml config.
cleanupUnusedTopNRules: ${SW_STORAGE_BANYANDB_CLEANUP_UNUSED_TOPN_RULES:true}
# The namespace in BanyanDB to store the data of OAP, if not set, the default is "sw".
# OAP will create BanyanDB Groups using the format of "{namespace}_{group name}", such as "sw_records".
namespace: ${SW_NAMESPACE:"sw"}
# The compatible server API versions of BanyanDB.
compatibleServerApiVersions: ${SW_STORAGE_BANYANDB_COMPATIBLE_SERVER_API_VERSIONS:"0.9"}

groups:
# The group settings of record.
Expand Down Expand Up @@ -393,4 +398,4 @@ docker run -d \
- **Cluster Mode**: Suitable for large-scale deployments.
- **Configuration**: `targets` is the IP address/hostname and port of the `liaison` nodes, separated by commas. `Liaison` nodes are the entry points of the BanyanDB cluster.

For more details, refer to the documentation of [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/latest/readme/) and the [BanyanDB Java Client](https://github.com/apache/skywalking-banyandb-java-client) subprojects.
For more details, refer to the documentation of [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/latest/readme/).
143 changes: 143 additions & 0 deletions oap-server/server-library/library-banyandb-client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>server-library</artifactId>
<groupId>org.apache.skywalking</groupId>
<version>${revision}</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>library-banyandb-client</artifactId>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-handler</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec-http2</artifactId>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
</dependency>
<dependency>
<groupId>build.buf.protoc-gen-validate</groupId>
<artifactId>protoc-gen-validate</artifactId>
<version>${bufbuild.protoc-gen-validate.version}</version>
<type>pom</type>
</dependency>
<dependency>
<groupId>build.buf.protoc-gen-validate</groupId>
<artifactId>pgv-java-stub</artifactId>
<version>${bufbuild.protoc-gen-validate.version}</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-testing</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>server-testing</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>library-integration-test</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>${os-maven-plugin.version}</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>detect</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>${protobuf-maven-plugin.version}</version>
<configuration>
<!--
The version of protoc must match protobuf-java. If you don't depend on
protobuf-java directly, you will be transitively depending on the
protobuf-java version that grpc depends on.
-->
<protocArtifact>
com.google.protobuf:protoc:${com.google.protobuf.protoc.version}:exe:${os.detected.classifier}
</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>
io.grpc:protoc-gen-grpc-java:${protoc-gen-grpc-java.plugin.version}:exe:${os.detected.classifier}
</pluginArtifact>
<protoSourceRoot>${project.basedir}/src/main/proto/proto</protoSourceRoot>
<includes>
<include>**/*.proto</include>
</includes>
</configuration>
<executions>
<execution>
<id>grpc-build</id>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
<execution>
<id>protoc-java-pgv</id>
<goals>
<goal>compile-custom</goal>
</goals>
<configuration>
<pluginParameter>lang=java</pluginParameter>
<pluginId>java-pgv</pluginId>
<pluginArtifact>
build.buf.protoc-gen-validate:protoc-gen-validate:${bufbuild.protoc-gen-validate.version}:exe:${os.detected.classifier}
</pluginArtifact>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
31 changes: 31 additions & 0 deletions ...ient/src/main/java/org/apache/skywalking/library/banyandb/v1/client/AbstractCriteria.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/

package org.apache.skywalking.library.banyandb.v1.client;

import org.apache.skywalking.banyandb.model.v1.BanyandbModel;

public abstract class AbstractCriteria {
public abstract BanyandbModel.Criteria build();
}

abstract class LogicalExpression extends AbstractCriteria {
abstract AbstractCriteria left();

abstract AbstractCriteria right();
}
Loading
Loading