Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
**/target/

# Pre-extracted release dirs / archives
apache-hugegraph-*/
**/apache-hugegraph-*/
**/*.tar
**/*.tar.gz
**/*.zip
Expand Down
53 changes: 52 additions & 1 deletion .github/workflows/docker-build-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@ on:
paths:
- '**/Dockerfile*'
- '.dockerignore'
- '.github/workflows/docker-build-ci.yml'
- 'pom.xml'
- 'hugegraph-pd/hg-pd-grpc/pom.xml'
- 'hugegraph-server/pom.xml'
- 'hugegraph-server/hugegraph-core/pom.xml'
- 'hugegraph-server/hugegraph-rocksdb/pom.xml'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: run the architecture smoke for RocksDB source changes

The new runtime matrix builds and executes the RocksDB/JNI module, but this path filter matches only its pom.xml. A PR that changes hugegraph-server/hugegraph-rocksdb/src/** without touching one of the listed infrastructure files will not schedule the amd64/arm64/riscv64 smoke, leaving native-loading and architecture regressions untested. Please cover the module path (for example, hugegraph-server/hugegraph-rocksdb/**) or otherwise add an equivalent trigger for the runtime code exercised by this job.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the maintainer-requested scope reduction. The Docker runtime matrix and its path filter were removed. RISC-V validation now runs as an independent reusable job invoked from the existing server-ci.yml, so every Server CI pull_request event, including hugegraph-rocksdb source-only changes, schedules the RISC-V build and runtime gate. Exact-head org job 89155333088 passed the clean build, JNI, REST/Gremlin, restart persistence, shutdown, and cleanup flow.

- 'hugegraph-server/hugegraph-dist/pom.xml'
- 'hugegraph-server/hugegraph-dist/docker/**'
- 'hugegraph-server/hugegraph-dist/src/main/resources/backend.properties'
- 'hugegraph-server/hugegraph-dist/src/assembly/static/**'
- 'hugegraph-server/hugegraph-dist/src/assembly/travis/check-rocksdb-only-dist.sh'
- 'hugegraph-server/hugegraph-dist/src/assembly/travis/*smoke*'

jobs:
docker-build:
Expand All @@ -36,7 +48,6 @@ jobs:
dockerfile:
- hugegraph-pd/Dockerfile
- hugegraph-store/Dockerfile
- hugegraph-server/Dockerfile
- hugegraph-server/Dockerfile-hstore

steps:
Expand All @@ -50,3 +61,43 @@ jobs:
HC=$(docker inspect --format='{{json .Config.Healthcheck}}' "$IMAGE_ID")
echo "Healthcheck: $HC"
[[ "$HC" != "null" ]] || { echo "ERROR: HEALTHCHECK missing in ${{ matrix.dockerfile }}"; exit 1; }

server-runtime-smoke:
runs-on: ubuntu-24.04
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
arch: [amd64, arm64, riscv64]

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up QEMU
if: matrix.arch != 'amd64'
uses: docker/setup-qemu-action@v3
Comment thread
imbajin marked this conversation as resolved.
Outdated
with:
platforms: ${{ matrix.arch }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build HugeGraph Server for linux/${{ matrix.arch }}
Comment thread
imbajin marked this conversation as resolved.
Outdated
run: |
docker buildx build \
--platform linux/${{ matrix.arch }} \
--load \
--tag hugegraph-server-smoke:${{ matrix.arch }} \
--file hugegraph-server/Dockerfile .

- name: Check healthcheck and run runtime smoke
env:
IMAGE: hugegraph-server-smoke:${{ matrix.arch }}
ARCH: ${{ matrix.arch }}
run: |
HC=$(docker inspect --format='{{json .Config.Healthcheck}}' "$IMAGE")
echo "Healthcheck: $HC"
[[ "$HC" != "null" ]] || { echo "ERROR: HEALTHCHECK missing"; exit 1; }
hugegraph-server/hugegraph-dist/src/assembly/travis/run-docker-runtime-smoke-test.sh \
"$IMAGE" "$ARCH"
37 changes: 36 additions & 1 deletion hugegraph-pd/hg-pd-grpc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@


<properties>
<os.plugin.version>1.6.0</os.plugin.version>
<os.plugin.version>1.7.1</os.plugin.version>
<grpc.version>1.39.0</grpc.version>
<protoc.version>3.17.2</protoc.version>
<protobuf.plugin.version>0.6.1</protobuf.plugin.version>
Expand Down Expand Up @@ -127,4 +127,39 @@
</plugin>
</plugins>
</build>

<profiles>
<profile>
<id>riscv64-protobuf-tools</id>
<activation>
<os>
<arch>riscv64</arch>
</os>
</activation>
<properties>
<!-- Compatible with the distro generators and Server runtime -->
<grpc.version>1.47.0</grpc.version>
<riscv64.protobuf.java.version>3.21.7</riscv64.protobuf.java.version>
</properties>
<dependencies>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>${riscv64.protobuf.java.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<configuration>
<protocExecutable>/usr/bin/protoc</protocExecutable>
<pluginExecutable>/usr/bin/grpc_java_plugin</pluginExecutable>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
69 changes: 58 additions & 11 deletions hugegraph-server/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,67 @@
#

# Dockerfile for HugeGraph Server
# 1st stage: build source code
ARG TARGETARCH

# Prepare the RISC-V Java runtime on the native build platform
FROM --platform=$BUILDPLATFORM maven:3.9.0-eclipse-temurin-11 AS riscv64-java

ARG TARGETARCH
ARG DRAGONWELL_RISCV64_URL="https://github.com/dragonwell-project/dragonwell11/releases/download/dragonwell-extended-11.0.31.28_jdk-11.0.31-ga/Alibaba_Dragonwell_Extended_11.0.31.28.11_riscv64_linux.tar.gz"
ARG DRAGONWELL_RISCV64_SHA256="7df2d308f0dca7a779d2854e6da19214f99cd77aa6d4982c3bf981a77266a79b"

RUN mkdir -p /opt/dragonwell \
&& if [ "$TARGETARCH" = "riscv64" ]; then \
curl --fail --location --retry 3 --show-error \
"$DRAGONWELL_RISCV64_URL" --output /tmp/dragonwell.tar.gz; \
echo "$DRAGONWELL_RISCV64_SHA256 /tmp/dragonwell.tar.gz" | sha256sum -c -; \
tar -xzf /tmp/dragonwell.tar.gz --strip-components=1 -C /opt/dragonwell; \
rm /tmp/dragonwell.tar.gz; \
fi

# Build source code
FROM --platform=$BUILDPLATFORM maven:3.9.0-eclipse-temurin-11 AS build

WORKDIR /pkg

COPY . .

ARG MAVEN_ARGS
ARG TARGETARCH

RUN --mount=type=cache,target=/root/.m2 \
mvn package $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
&& rm ./hugegraph-server/*.tar.gz ./hugegraph-pd/*.tar.gz ./hugegraph-store/*.tar.gz
if [ "$TARGETARCH" = "riscv64" ]; then \
mvn package $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true \
-Dmaven.javadoc.skip=true -Drocksdb-only \
-pl hugegraph-server/hugegraph-dist -am \
&& hugegraph-server/hugegraph-dist/src/assembly/travis/check-rocksdb-only-dist.sh \
hugegraph-server/apache-hugegraph-server-*/; \
else \
mvn package $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true \
-Dmaven.javadoc.skip=true \
&& rm ./hugegraph-server/*.tar.gz ./hugegraph-pd/*.tar.gz \
./hugegraph-store/*.tar.gz; \
fi

# 2nd stage: runtime env
# Keep the existing Temurin 11 runtime for every architecture it supports.
FROM --platform=linux/amd64 eclipse-temurin:11-jre-jammy AS runtime-amd64
FROM --platform=linux/arm/v7 eclipse-temurin:11-jre-jammy AS runtime-arm
FROM --platform=linux/arm64 eclipse-temurin:11-jre-jammy AS runtime-arm64
FROM --platform=linux/ppc64le eclipse-temurin:11-jre-jammy AS runtime-ppc64le
FROM --platform=linux/s390x eclipse-temurin:11-jre-jammy AS runtime-s390x

# Temurin 11 has no RISC-V image, so use Ubuntu with the verified Dragonwell archive.
FROM ubuntu:24.04 AS runtime-riscv64
COPY --from=riscv64-java /opt/dragonwell/ /opt/dragonwell/
ENV JAVA_HOME="/opt/dragonwell" \
PATH="/opt/dragonwell/bin:${PATH}"

# Runtime environment
# Note: ZGC (The Z Garbage Collector) is only supported on ARM-Mac with java > 13
FROM eclipse-temurin:11-jre-jammy
FROM runtime-${TARGETARCH} AS runtime

ARG TARGETARCH
ARG DEBIAN_FRONTEND=noninteractive

COPY --from=build /pkg/hugegraph-server/apache-hugegraph-server-*/ /hugegraph-server/
LABEL maintainer="HugeGraph Docker Maintainers <dev@hugegraph.apache.org>"
Expand All @@ -47,12 +92,14 @@ WORKDIR /hugegraph-server/

# 1. Install runtime dependencies and configure server
RUN apt-get -q update \
&& apt-get -q install -y --no-install-recommends --no-install-suggests \
dumb-init \
procps \
curl \
lsof \
vim \
&& runtime_packages="dumb-init procps curl lsof vim" \
&& if [ "$TARGETARCH" = "riscv64" ]; then \
runtime_packages="$runtime_packages ca-certificates libatomic1 libstdc++6 libgcc-s1"; \
fi \
&& apt-get -q install -y --no-install-recommends --no-install-suggests $runtime_packages \
&& if [ "$TARGETARCH" = "riscv64" ]; then \
test -x /opt/dragonwell/bin/java; \
fi \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& sed -i "s/^restserver.url.*$/restserver.url=http:\/\/0.0.0.0:8080/g" ./conf/rest-server.properties
Expand Down
75 changes: 75 additions & 0 deletions hugegraph-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,78 @@ HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml
```

See [docker/README.md](../docker/README.md) for the full setup guide.

## RISC-V Development and Testing

The supported RISC-V target is 64-bit `linux/riscv64` LP64D with glibc 2.30 or newer.
This target covers HugeGraph Server with the embedded RocksDB backend only. It does not
cover musl/Alpine, 32-bit RISC-V, PD, Store, HStore, or other storage backends.

The RISC-V image uses the checksum-pinned Alibaba Dragonwell 11 Extended Server VM from
the [Dockerfile](Dockerfile). The runtime also provides `libstdc++6`, `libgcc-s1`, and
`libatomic.so.1`, which are required by the packaged RocksDB JNI library.

### Docker and QEMU

Docker Desktop includes emulation support. On Linux, install the RISC-V emulator only if
it is not already registered:

```bash
docker run --privileged --rm tonistiigi/binfmt --install riscv64
```

From the repository root, build and run the complete runtime smoke test:

```bash
IMAGE=hugegraph-server:riscv64-test
docker buildx build --platform linux/riscv64 --load --tag "$IMAGE" \
--file hugegraph-server/Dockerfile .
hugegraph-server/hugegraph-dist/src/assembly/travis/run-docker-runtime-smoke-test.sh \
"$IMAGE" riscv64
docker image rm "$IMAGE"
```

The smoke helper removes its test containers, anonymous volumes, and temporary files. It
keeps the input image and does not create or remove a Buildx builder or host emulator. If
you created those resources specifically for this test, remove them separately:

```bash
docker buildx rm hugegraph-riscv64
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-riscv64
```

Do not uninstall Docker Desktop's built-in emulator. QEMU is substantially slower than
native RISC-V and is intended for correctness testing, not performance measurements.

### Native RISC-V

Use Dragonwell 11 Extended `11.0.31.28.11` and verify its archive with the URL and SHA-256
declared in the Dockerfile. On a Debian-derived glibc system, install the native tools and
runtime libraries:

```bash
sudo apt-get update
sudo apt-get install -y maven protobuf-compiler \
protobuf-compiler-grpc-java-plugin libatomic1 libstdc++6 libgcc-s1 curl jq
```

After selecting Dragonwell as `JAVA_HOME`, build and verify the RocksDB-only distribution:

```bash
test "$(uname -m)" = riscv64
java -XshowSettings:vm -version
mvn clean package -Drocksdb-only -pl hugegraph-server/hugegraph-dist -am \
-Dmaven.test.skip=true -Dmaven.javadoc.skip=true
hugegraph-server/hugegraph-dist/src/assembly/travis/check-rocksdb-only-dist.sh \
hugegraph-server/apache-hugegraph-server-*/
hugegraph-server/hugegraph-dist/src/assembly/travis/run-native-runtime-smoke-test.sh \
hugegraph-server/apache-hugegraph-server-*/
```

Run the existing Core and API gates with the same RocksDB-only dependency boundary:

```bash
hugegraph-server/hugegraph-dist/src/assembly/travis/run-core-test.sh rocksdb
hugegraph-server/hugegraph-dist/src/assembly/travis/run-api-test.sh \
rocksdb target/riscv64-api-report
```
24 changes: 23 additions & 1 deletion hugegraph-server/hugegraph-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.5.0.Final</version>
<version>1.7.1</version>
</extension>
</extensions>

Expand Down Expand Up @@ -408,4 +408,26 @@
</plugin>
</plugins>
</build>

<profiles>
<profile>
<id>riscv64-protobuf-tools</id>
<activation>
<os>
<arch>riscv64</arch>
</os>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<configuration>
<protocExecutable>/usr/bin/protoc</protocExecutable>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
3 changes: 2 additions & 1 deletion hugegraph-server/hugegraph-dist/docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ else
log "HugeGraph initialization already done. Skipping re-init..."
fi

./bin/start-hugegraph.sh -j "${JAVA_OPTS:-}" -t 120
./bin/start-hugegraph.sh -j "${JAVA_OPTS:-}" \
-t "${HG_SERVER_STARTUP_TIMEOUT:-120}"

# Post-startup cluster stabilization check (hstore only — rocksdb has no partitions)
ACTUAL_BACKEND=$(grep -E '^[[:space:]]*backend[[:space:]]*=' "${GRAPH_CONF}" | head -n 1 | sed 's/.*=//' | tr -d '[:space:]' || true)
Expand Down
Loading
Loading