Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@
[submodule "skywalking-ui"]
path = skywalking-ui
url = https://github.com/apache/skywalking-booster-ui.git
[submodule "oap-server/server-library/library-banyandb-client/src/main/proto"]
path = oap-server/server-library/library-banyandb-client/src/main/proto
url = https://github.com/apache/skywalking-banyandb-client-proto
1 change: 0 additions & 1 deletion apm-dist/src/main/assembly/binary.xml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@
<include>metadata-service-mapping.yaml</include>
<include>trace-sampling-policy-settings.yml</include>
<include>hierarchy-definition.yml</include>
<include>bydb.dependencies.properties</include>
<include>bydb.yml</include>
<include>bydb-topn.yml</include>
<include>oal/*.oal</include>
Expand Down
3 changes: 3 additions & 0 deletions docs/en/changes/changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@
* Bump up netty to 4.2.9.Final.
* Add support for OpenSearch/ElasticSearch client certificate authentication.
* Fix BanyanDB logs paging query.
* Replace BanyanDB Java client with native implementation.
* Remove `bydb.dependencies.properties` and set the compatible BanyanDB API version number in `${SW_STORAGE_BANYANDB_COMPATIBLE_SERVER_API_VERSIONS}`
* Fix trace profiling query time range condition.

#### UI
* Fix the missing icon in new native trace view.
Expand Down
7 changes: 6 additions & 1 deletion docs/en/setup/backend/backend-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,13 @@ You should set up the database ready before starting the backend. We recommend t
If you want to use other databases, please read the [storage document](backend-storage.md).

Use the docker mode to run BanyanDB containerized.
The compatible BanyanDB API version number could be found in `/config/bydb.yml`
```
${SW_STORAGE_BANYANDB_COMPATIBLE_SERVER_API_VERSIONS}
```
The compatible BanyanDB Server version number can get through the [API versions mapping](https://skywalking.apache.org/docs/skywalking-banyandb/latest/installation/versions/).

```shell
# The compatible version number could be found in /config/bydb.dependencies.properties
export BYDB_VERSION=xxx

docker pull apache/skywalking-banyandb:${BYDB_VERSION}
Expand Down
35 changes: 18 additions & 17 deletions docs/en/setup/backend/configuration-vocabulary.md
Original file line number Diff line number Diff line change
Expand Up @@ -352,23 +352,24 @@ The following table lists the configuration items:
### Global Configuration
The global settings for the whole BanyanDB:

| Settings | Value(s) and Explanation | System Environment Variable¹ | Default |
|---------------------------------|-------------------------------------------------------------------------------------------------------|---------------------------------------------------------|-----------------|
| targets | Hosts with ports of the BanyanDB. | SW_STORAGE_BANYANDB_TARGETS | 127.0.0.1:17912 |
| maxBulkSize | The maximum size of write entities in a single batch write call. | SW_STORAGE_BANYANDB_MAX_BULK_SIZE | 10000 |
| flushInterval | Period of flush interval. In the timeunit of seconds. | SW_STORAGE_BANYANDB_FLUSH_INTERVAL | 15 |
| flushTimeout | The timeout seconds of a bulk flush. | SW_STORAGE_BANYANDB_FLUSH_TIMEOUT | 10 |
| concurrentWriteThreads | Concurrent consumer threads for batch writing. | SW_STORAGE_BANYANDB_CONCURRENT_WRITE_THREADS | 15 |
| profileTaskQueryMaxSize | Max size of ProfileTask to be fetched. | SW_STORAGE_BANYANDB_PROFILE_TASK_QUERY_MAX_SIZE | 200 |
| resultWindowMaxSize | The maximum size of dataset when the OAP loads cache, such as network aliases. | SW_STORAGE_BANYAND_QUERY_MAX_WINDOW_SIZE | 10000 |
| metadataQueryMaxSize | The maximum size of metadata per query. | SW_STORAGE_BANYAND_QUERY_MAX_SIZE | 10000 |
| segmentQueryMaxSize | The maximum size of trace segments per query. | SW_STORAGE_BANYAND_QUERY_SEGMENT_SIZE | 200 |
| asyncProfilerTaskQueryMaxSize | Max size of AsyncProfilerTask to be fetched. | SW_STORAGE_BANYANDB_ASYNC_PROFILER_TASK_QUERY_MAX_SIZE | 200 |
| profileDataQueryScrollBatchSize | The batch size of query profiling data. | SW_STORAGE_BANYAND_QUERY_PROFILE_DATA_BATCH_SIZE | 100 |
| user | The username for BanyanDB authentication. | SW_STORAGE_BANYANDB_USER | - |
| password | The password for BanyanDB authentication. | SW_STORAGE_BANYANDB_PASSWORD | - |
| sslTrustCAPath | If the BanyanDB server is configured with TLS, config the TLS cert file path and open tls connection. | SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH | - |
| cleanupUnusedTopNRules | Cleanup TopN rules in BanyanDB server that are not configured in the bydb-topn.yml config. | SW_STORAGE_BANYANDB_CLEANUP_UNUSED_TOPN_RULES | true |
| Settings | Value(s) and Explanation | System Environment Variable¹ | Default |
|---------------------------------|-------------------------------------------------------------------------------------------------------|--------------------------------------------------------|-------------------|
| targets | Hosts with ports of the BanyanDB. | SW_STORAGE_BANYANDB_TARGETS | 127.0.0.1:17912 |
| maxBulkSize | The maximum size of write entities in a single batch write call. | SW_STORAGE_BANYANDB_MAX_BULK_SIZE | 10000 |
| flushInterval | Period of flush interval. In the timeunit of seconds. | SW_STORAGE_BANYANDB_FLUSH_INTERVAL | 15 |
| flushTimeout | The timeout seconds of a bulk flush. | SW_STORAGE_BANYANDB_FLUSH_TIMEOUT | 10 |
| concurrentWriteThreads | Concurrent consumer threads for batch writing. | SW_STORAGE_BANYANDB_CONCURRENT_WRITE_THREADS | 15 |
| profileTaskQueryMaxSize | Max size of ProfileTask to be fetched. | SW_STORAGE_BANYANDB_PROFILE_TASK_QUERY_MAX_SIZE | 200 |
| resultWindowMaxSize | The maximum size of dataset when the OAP loads cache, such as network aliases. | SW_STORAGE_BANYAND_QUERY_MAX_WINDOW_SIZE | 10000 |
| metadataQueryMaxSize | The maximum size of metadata per query. | SW_STORAGE_BANYAND_QUERY_MAX_SIZE | 10000 |
| segmentQueryMaxSize | The maximum size of trace segments per query. | SW_STORAGE_BANYAND_QUERY_SEGMENT_SIZE | 200 |
| asyncProfilerTaskQueryMaxSize | Max size of AsyncProfilerTask to be fetched. | SW_STORAGE_BANYANDB_ASYNC_PROFILER_TASK_QUERY_MAX_SIZE | 200 |
| profileDataQueryScrollBatchSize | The batch size of query profiling data. | SW_STORAGE_BANYAND_QUERY_PROFILE_DATA_BATCH_SIZE | 100 |
| user | The username for BanyanDB authentication. | SW_STORAGE_BANYANDB_USER | - |
| password | The password for BanyanDB authentication. | SW_STORAGE_BANYANDB_PASSWORD | - |
| sslTrustCAPath | If the BanyanDB server is configured with TLS, config the TLS cert file path and open tls connection. | SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH | - |
| cleanupUnusedTopNRules | Cleanup TopN rules in BanyanDB server that are not configured in the bydb-topn.yml config. | SW_STORAGE_BANYANDB_CLEANUP_UNUSED_TOPN_RULES | true |
| namespace | The namespace in BanyanDB to store the data of OAP. | SW_NAMESPACE | default |

### Group Configuration
The settings for each group:
Expand Down
24 changes: 11 additions & 13 deletions docs/en/setup/backend/storages/banyandb.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,12 @@

[BanyanDB](https://github.com/apache/skywalking-banyandb) is a dedicated storage implementation developed by the SkyWalking Team and the community. Activate BanyanDB as the storage by setting the storage provider to **banyandb**.

The BanyanDB server compatibility relies on API and release versions, which are defined in `bydb.dependencies.properties`
```shell
# BanyanDB version is the version number of BanyanDB Server release.
# This is the bundled and tested BanyanDB release version
bydb.version=x.y
# BanyanDB API version is the version number of the BanyanDB query APIs
# OAP server has bundled implementation of BanyanDB Java client.
# Please check BanyanDB documentation for the API version compatibility.
# https://skywalking.apache.org/docs/skywalking-banyandb/next/installation/versions
# Each `bydb.api.version` could have multiple compatible release version(`bydb.version`).
bydb.api.version=x.y
The BanyanDB server compatibility relies on API and release versions,
The compatible BanyanDB API version number could be found in `/config/bydb.yml`
```
${SW_STORAGE_BANYANDB_COMPATIBLE_SERVER_API_VERSIONS}
```
The compatible BanyanDB Server version number can get through the [API versions mapping](https://skywalking.apache.org/docs/skywalking-banyandb/latest/installation/versions/).

If the BanyanDB server API version is not compatible with the OAP server, the OAP server will not start, and the following error message will be displayed:
```shell
Expand Down Expand Up @@ -68,6 +62,11 @@ global:
sslTrustCAPath: ${SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH:""}
# Cleanup TopN rules in BanyanDB server that are not configured in the bydb-topn.yml config.
cleanupUnusedTopNRules: ${SW_STORAGE_BANYANDB_CLEANUP_UNUSED_TOPN_RULES:true}
# The namespace in BanyanDB to store the data of OAP, if not set, the default is "sw".
# OAP will create BanyanDB Groups using the format of "{namespace}_{group name}", such as "sw_records".
namespace: ${SW_NAMESPACE:"sw"}
# The compatible server API versions of BanyanDB.
compatibleServerApiVersions: ${SW_STORAGE_BANYANDB_COMPATIBLE_SERVER_API_VERSIONS:"0.9"}

groups:
# The group settings of record.
Expand Down Expand Up @@ -365,7 +364,6 @@ BanyanDB Server supports two installation modes:

Use the docker mode to run BanyanDB containerized.
```shell
# The compatible version number could be found in /config/bydb.dependencies.properties
export BYDB_VERSION=xxx

docker pull apache/skywalking-banyandb:${BYDB_VERSION}
Expand Down Expand Up @@ -393,4 +391,4 @@ docker run -d \
- **Cluster Mode**: Suitable for large-scale deployments.
- **Configuration**: `targets` is the IP address/hostname and port of the `liaison` nodes, separated by commas. `Liaison` nodes are the entry points of the BanyanDB cluster.

For more details, refer to the documentation of [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/latest/readme/) and the [BanyanDB Java Client](https://github.com/apache/skywalking-banyandb-java-client) subprojects.
For more details, refer to the documentation of [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/latest/readme/).
Original file line number Diff line number Diff line change
Expand Up @@ -191,17 +191,12 @@ public ProfileAnalyzation getProfileAnalyze(final List<SegmentProfileAnalyzeQuer
public List<SegmentRecord> getTaskSegments(String taskId) throws IOException {
List<ProfileThreadSnapshotRecord> records = getProfileThreadSnapshotQueryDAO().queryRecords(taskId);
List<String> profiledSegmentIdList = records.stream().map(ProfileThreadSnapshotRecord::getSegmentId).collect(Collectors.toList());
long endTimestamp = 0;
for (ProfileThreadSnapshotRecord record : records) {
if (record.getDumpTime() > endTimestamp) {
endTimestamp = record.getDumpTime();
}
}
long startTimestamp = endTimestamp - 5 * 60 * 1000; // 5 minutes
ProfileTask profileTask = getProfileTaskDAO().getById(taskId);
long endTimestamp = profileTask.getStartTime() + (long) profileTask.getDuration() * 60 * 1000; // duration minute to ms
Duration duration = new Duration();
duration.setStep(Step.SECOND);
DateTime endTime = new DateTime(endTimestamp);
DateTime startTime = new DateTime(startTimestamp);
DateTime endTime = new DateTime(endTimestamp + 15 * 60 * 1000); // 15 minutes later to cover delay
DateTime startTime = new DateTime(endTimestamp - 10 * 60 * 1000); // 10 minutes before end time
duration.setStart(startTime.toString("yyyy-MM-dd HHmmss"));
duration.setEnd(endTime.toString("yyyy-MM-dd HHmmss"));
return getTraceQueryDAO().queryBySegmentIdList(profiledSegmentIdList, duration);
Expand Down
Loading
Loading