Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building and using native-image #2

Merged
merged 6 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/continuous-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
strategy:
fail-fast: false
matrix:
java: ['17', '21']
java: ['21']

steps:
- name: Git Checkout
Expand Down
34 changes: 34 additions & 0 deletions .github/workflows/native-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: GraalVM Native Image builds

on:
workflow_dispatch:
inputs:
TAG:
description: 'The tag to be built'
required: true
type: string

jobs:
build:
name: Optimize-to-xes on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
steps:
- uses: actions/checkout@v4

- uses: graalvm/setup-graalvm@v1
with:
java-version: '21'
distribution: 'graalvm'
github-token: ${{ secrets.GITHUB_TOKEN }}
native-image-job-reports: 'true'
- name: Build native-image
run: mvn -Pnative native:compile

- name: Upload binary
uses: actions/upload-artifact@v3
with:
name: optimize-to-xes-${{ matrix.os }}
path: target/optimize-to-xes*
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ target/
.settings
.springBeans
.sts4-cache
.DS_Store

### IntelliJ IDEA ###
.idea
Expand All @@ -35,3 +36,4 @@ build/
### Others ###
application-local.yaml
xes-output.xml
native-image/**
66 changes: 60 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,27 @@ and convert it to [XES](https://xes-standard.org/) for Predictive Process Monito
* [✨Features](#features)
* [📤 Camunda Optimize to XES](#-camunda-optimize-to-xes)
* [📊 XES - eXtensible Event Stream](#-xes---extensible-event-stream)
* [🚀🔜 Coming Soon... 🌟🎉👀](#-coming-soon-)
* [🚀 Getting Started](#-getting-started)
* [Execute native-image](#execute-native-image)
* [Configuring `application.yaml`](#configuring-applicationyaml)
* [👨‍💻 Developer's Guide](#-developers-guide)
* [Building a Native Image](#building-a-native-image)

# ✨Features

## 📤 Camunda Optimize to XES

Here's the scoop on this project: It's your ticket to export your raw process data from Camunda Optimize using the
mighty [Data Export API](https://docs.camunda.io/optimize/apis-tools/optimize-api/report/get-data-export/). 🚀
🔍 For the smoothest experience, consider applying a filter to show only completed instances. 🌟 This helps streamline your view and focus on what's done! ✅🚀
🔍 For the smoothest experience, consider applying a filter to show only completed instances. 🌟 This helps streamline
your view and focus on what's done! ✅🚀

Afterward, we work our magic to transform it into the fantastic world of [XES](#-xes---extensible-event-stream). 🪄✨

🔮 Dive into the world of possibilities! Utilize XES for Predictive Process Monitoring, or easily import your XES file
into a Process Mining tool like [Disco](https://fluxicon.com/disco/) 🚀. Uncover insights and let the magic of data
unfold! 🌟💼

## 📊 XES - eXtensible Event Stream

We rely on [XES](https://www.xes-standard.org/openxes/start), which is like a superhero cape for event data! 🦸‍♂️
Expand All @@ -31,11 +38,24 @@ treasure map for uncovering insights in the world of processes and workflows!

For our Java-powered adventures, we especially use this [OpenXES](http://code.deckfour.org/xes/) implementation. 🧑‍💻

## 🚀🔜 Coming Soon... 🌟🎉👀
# 🚀 Getting Started

![Optimize CCR PPM Cycle](./assets/ppm-cycle.png)
## Execute native-image

# 🚀 Getting Started
You can simply download the latest native-image from the [releases here](https://github.com/envite-consulting/optimize-to-xes/releases/latest) and
execute it with the following options:

```shell
./optimize-to-xes \
--optimize.base-url='optimize_base_url' \
--optimize.report-id='report_id' \
--optimize.client-id='client_id' \
--optimize.client-secret='client_secret' \
--xes-mapping.base-path='<optional-path-to-output-dir>'
```

If you want to provide a static Bearer Token you could ignore `client-id` and `client-secret` and
add `bearer-token` instead.

## Configuring `application.yaml`

Expand All @@ -46,6 +66,9 @@ To set up your configuration in `application.yaml`, follow these steps:
3. Add the Client ID and Secret.
4. *Optionally:* Specify the base bath for the resulting XML(s).

If you want to provide a static Bearer Token you could ignore `client-id` and `client-secret` and
add `bearer-token` instead.

Once you've completed these configurations, you'll be prepared to retrieve the data and convert it to XES format 🎉

To install all dependencies, run the following command:
Expand All @@ -58,4 +81,35 @@ To install and start the commandline runner, use the command below:

```shell
$ ./mvnw spring-boot:run
```
```

# 👨‍💻 Developer's Guide

## Building a Native Image

> **🚀📚 Requirements:**
> * Java 21

Usually, running `./mvnw clean native:compile -Pnative` should be all you need to create the native image. 🚀

However, there's a little twist 🌀 – due to some reflection magic happening in the OpenXES Library, we'll need to
catch and pass all those reflections to the native image building process. It's like adding a touch of wizardry to
your development journey! 🧙‍♂️✨🏗️

```shell
# Build the native image
./mvnw clean native:compile -Pnative

# Find all reflection usages: A native-image folder will be places in the root of the project.
java -Dspring.aot.enabled=true \
-agentlib:native-image-agent=config-output-dir=./src/main/resources/META-INF/native-image \
-Doptimize.base-url='<base_url>' \
-Doptimize.report-d='<report_id' \
-Doptimize.client-id='<client_id>' \
-Doptimize.client-secret='client_secret' \
-Dxes-mapping.base-path='target' \
-jar target/optimize-to-xes-<version>.jar

# Build the native image again with the extended information on the relfection
./mvnw clean native:compile -Pnative
```
16 changes: 12 additions & 4 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,27 @@
</repository>
</distributionManagement>

<repositories>
<repository>
<id>local</id>
<url>file://${project.basedir}/src/main/resources/openxes-dependencies</url>
</repository>
</repositories>

<dependencies>
<dependency>
<groupId>org.deckfour</groupId>
<artifactId>openxes</artifactId>
<version>1.1</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/main/resources/OpenXES.jar</systemPath>
</dependency>

<dependency>
<groupId>org.deckfour</groupId>
<artifactId>spex</artifactId>
<version>1.0-RC2</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/main/resources/Spex.jar</systemPath>
</dependency>


<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
Expand Down Expand Up @@ -133,6 +137,10 @@
<sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
<testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.graalvm.buildtools</groupId>
<artifactId>native-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,5 @@ internal data class OptimizeClientProperties(
var clientId: String? = null,
var clientSecret: String? = null,
var tokenBaseUrl: String? = null,
var bearerToken: String? = null,
)
Original file line number Diff line number Diff line change
Expand Up @@ -2,50 +2,39 @@ package de.envite.greenbpm.optimzetoxes.optimizeexport.adapter.out.optimize

import de.envite.greenbpm.optimzetoxes.Logging
import de.envite.greenbpm.optimzetoxes.log
import de.envite.greenbpm.optimzetoxes.optimizeexport.adapter.out.optimize.token.OptimizeBearerTokenService
import de.envite.greenbpm.optimzetoxes.optimizeexport.domain.model.OptimizeData
import de.envite.greenbpm.optimzetoxes.optimizeexport.usecase.out.RawDataQuery
import org.springframework.beans.factory.annotation.Qualifier
import org.springframework.aot.hint.annotation.RegisterReflectionForBinding
import org.springframework.boot.context.properties.EnableConfigurationProperties
import org.springframework.http.HttpHeaders
import org.springframework.http.HttpStatusCode
import org.springframework.stereotype.Service
import org.springframework.web.reactive.function.BodyInserters
import org.springframework.web.reactive.function.client.WebClient
import reactor.core.publisher.Mono

@Service
@EnableConfigurationProperties(OptimizeClientProperties::class)
@RegisterReflectionForBinding(OptimizeData::class)
internal class OptimizeRawDataQueryService(
@Qualifier("optimize-client")
private val webClientOptimize: WebClient,
@Qualifier("login-client")
private val webClientLogin: WebClient,
private val optimizeClientProperties: OptimizeClientProperties
private val optimizeClientProperties: OptimizeClientProperties,
private val optimizeBearerTokenService: OptimizeBearerTokenService,
): RawDataQuery, Logging {

@Throws(DataQueryException::class)
private fun queryToken(): TokenResponse =
webClientLogin
.post()
.body(BodyInserters.fromValue(TokenRequest(optimizeClientProperties.clientId!!, optimizeClientProperties.clientSecret!!)))
.retrieve()
.onStatus(HttpStatusCode::isError) { _ ->
Mono.error { throw DataQueryException("Could not fetch Bearer Token from optimize") }
}
.bodyToMono(TokenResponse::class.java)
.blockOptional()
.orElseThrow { DataQueryException("Could not parse Bearer Token from optimize") }

@Throws(DataQueryException::class)
override fun queryData(): OptimizeData {
val token = queryToken()
val token = optimizeClientProperties.bearerToken ?: optimizeBearerTokenService.queryToken()

val uri = "/api/public/export/report/${optimizeClientProperties.reportId}/result/json?paginationTimeout=60&limit=2"

log().info("Fetch Camunda Optimize Raw Data Export from URI {}'", uri)

return webClientOptimize
return WebClient.builder()
.baseUrl(optimizeClientProperties.baseUrl!!)
.build()
.get()
.uri(uri)
.header(HttpHeaders.AUTHORIZATION, "Bearer ${token.access_token}")
.header(HttpHeaders.AUTHORIZATION, "Bearer $token")
.retrieve()
.onStatus(HttpStatusCode::isError) { _ ->
Mono.error { throw DataQueryException("Could not fetch data from optimize") }
Expand All @@ -55,14 +44,3 @@ internal class OptimizeRawDataQueryService(
.orElseThrow { DataQueryException("Could not parse data from optimize") }
}
}

private class TokenResponse (
var access_token: String? = null
)

private data class TokenRequest(
val client_id: String,
val client_secret: String,
val audience: String = "optimize.camunda.io",
val grant_type: String = "client_credentials"
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package de.envite.greenbpm.optimzetoxes.optimizeexport.adapter.out.optimize.token

import de.envite.greenbpm.optimzetoxes.Logging
import de.envite.greenbpm.optimzetoxes.optimizeexport.adapter.out.optimize.DataQueryException
import de.envite.greenbpm.optimzetoxes.optimizeexport.adapter.out.optimize.OptimizeClientProperties
import org.springframework.aot.hint.annotation.RegisterReflectionForBinding
import org.springframework.boot.context.properties.EnableConfigurationProperties
import org.springframework.http.HttpStatusCode
import org.springframework.stereotype.Service
import org.springframework.web.reactive.function.BodyInserters
import org.springframework.web.reactive.function.client.WebClient
import reactor.core.publisher.Mono


@Service
@EnableConfigurationProperties(OptimizeClientProperties::class)
@RegisterReflectionForBinding(TokenRequest::class, TokenResponse::class)
internal class OptimizeBearerTokenService(
private val optimizeClientProperties: OptimizeClientProperties
) : Logging {

@Throws(DataQueryException::class)
fun queryToken(): String? = WebClient.builder()
.baseUrl(optimizeClientProperties.tokenBaseUrl!!)
.build()
.post()
.body(
BodyInserters.fromValue(
TokenRequest(
optimizeClientProperties.clientId!!,
optimizeClientProperties.clientSecret!!
)
)
)
.retrieve()
.onStatus(HttpStatusCode::isError) { _ ->
Mono.error { throw DataQueryException("Could not fetch Bearer Token from optimize") }
}
.bodyToMono(TokenResponse::class.java)
.map { it.access_token }
.blockOptional()
.orElseThrow { DataQueryException("Could not parse Bearer Token from optimize") }
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package de.envite.greenbpm.optimzetoxes.optimizeexport.adapter.out.optimize.token

internal class TokenRequest(
val client_id: String,
val client_secret: String,
val audience: String = "optimize.camunda.io",
val grant_type: String = "client_credentials"
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package de.envite.greenbpm.optimzetoxes.optimizeexport.adapter.out.optimize.token

internal class TokenResponse(
var access_token: String? = null
)
Loading