Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve api #1

Merged
merged 4 commits into from
Oct 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 27 additions & 20 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
@@ -1,32 +1,39 @@
name: "Build"

on: [pull_request, push]

env:
JAVA_OPTS: -Xms1g -Xmx3g
GRADLE_OPTS: "-Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true -Dorg.gradle.jvmargs=-Xmx3g -XX:MaxPermSize=2048m -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8"
on:
push:
paths-ignore:
- 'docs/**'
- '*.md'
branches:
- main

jobs:

build:
check:
runs-on: ubuntu-latest
timeout-minutes: 30

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Restore Gradle cache
id: cache
uses: actions/[email protected]
- uses: actions/setup-java@v3
with:
distribution: 'zulu'
java-version: 17

- uses: gradle/gradle-build-action@v2
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
~/.konan
key: ubuntu-latest-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
ubuntu-latest-gradle-
arguments: build --scan --full-stacktrace

- name: Bundle the build report
if: failure()
run: find . -type d -name 'reports' | zip -@ -r build-reports.zip

- name: Build
run: ./gradlew build
- name: Upload the build report
if: failure()
uses: actions/upload-artifact@master
with:
name: error-report
path: build-reports.zip
25 changes: 25 additions & 0 deletions .github/workflows/githubpages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: githubpages

on:
release:
types: [published]

jobs:
githubpages:
runs-on: ubuntu-latest
timeout-minutes: 20

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: gradle/gradle-build-action@v2
with:
arguments: -Pversion=${{ github.event.release.tag_name }} dokkaHtml

- name: Deploy to gh-pages
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./build/dokka/htmlMultiModule
37 changes: 37 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: "Build main"

on:
pull_request:
paths-ignore:
- 'docs/**'
- '*.md'

jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 30

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-java@v3
with:
distribution: 'zulu'
java-version: 17

- uses: gradle/gradle-build-action@v2
with:
arguments: build --scan --full-stacktrace

- name: Bundle the build report
if: failure()
run: find . -type d -name 'reports' | zip -@ -r build-reports.zip

- name: Upload the build report
if: failure()
uses: actions/upload-artifact@master
with:
name: error-report
path: build-reports.zip
47 changes: 47 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: "Publish library"

on:
workflow_dispatch:
branches: [main]
inputs:
version:
description: 'Version'
required: true
type: string

env:
ORG_GRADLE_PROJECT_mavenCentralUsername: '${{ secrets.SONATYPE_USER }}'
ORG_GRADLE_PROJECT_mavenCentralPassword: '${{ secrets.SONATYPE_PWD }}'
ORG_GRADLE_PROJECT_signingInMemoryKeyId: '${{ secrets.SIGNING_KEY_ID }}'
ORG_GRADLE_PROJECT_signingInMemoryKey: '${{ secrets.SIGNING_KEY }}'
ORG_GRADLE_PROJECT_signingInMemoryKeyPassword: '${{ secrets.SIGNING_KEY_PASSPHRASE }}'

jobs:
publish:
timeout-minutes: 30
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-java@v3
with:
distribution: 'zulu'
java-version: 11

- uses: gradle/gradle-build-action@v2
with:
arguments: assemble -Pversion=${{ inputs.version }}

- name: Upload reports
if: failure()
uses: actions/upload-artifact@v3
with:
name: 'reports-${{ matrix.os }}'
path: '**/build/reports/**'

- name: Publish final version
uses: gradle/gradle-build-action@v2
with:
arguments: -Pversion=${{ inputs.version }} publishAllPublicationsToMavenCentralRepository
68 changes: 67 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,69 @@
# Kotlin GCP PubSub

A KotlinX (Flow) binding for Google Cloud PubSub.
Google Cloud PubSub made easy! Kotlin GCP PubSub offers idiomatic KotlinX & Ktor integration for GCP.

```kotlin
@Serializable
data class Event(val key: String, val message: String)

fun Application.pubSubApp() {
pubSub(ProjectId("my-project")) {
subscribe<Event>(SubscriptionId("my-subscription")) { (event, record) ->
println("event.key: ${event.key}, event.message: ${event.message}")
record.ack()
}
}

routing {
post("/publish/{key}/{message}") {
val event = Event(call.parameters["key"]!!, call.parameters["message"]!!)

pubSub()
.publisher(ProjectId("my-project"))
.publish(TopicId("my-topic"), event)

call.respond(HttpStatusCode.Accepted)
}
}
}
```

## Modules

- [PubSub Ktor plugin](pubsub-ktor/README.MD) to conveniently consume messages from GCP PubSub, and publish messages to
GCP PubSub
- [PubSub Ktor KotlinX Serialization Json](pubsub-ktor-kotlinx-serialization-json/README.MD) to conveniently consume
messages from GCP PubSub, and publish messages to GCP PubSub using KotlinX Serialization Json
- [PubSub test](pubsub-test/README.MD) one-line testing support powered by testcontainers
- [GCP PubSub](pubsub/README.MD): KotlinX integration for `TopicAdminClient`, `SubscriptionAdminClient`, `Susbcriber`
and `Publisher`.
- [PubSub Ktor KotlinX Serialization Json](pubsub-kotlinx-serialization-json/README.MD) to conveniently consume messages
from GCP PubSub, and publish messages to GCP PubSub
- [Google Common API](api-core/README.MD): KotlinX integration for `ApiFuture`

## Using in your projects

### Gradle

Add dependencies (you can also add other modules that you need):

```kotlin
dependencies {
implementation("io.github.nomisrev:gcp-pubsub-ktor:1.0.0")
implementation("io.github.nomisrev:gcp-pubsub-ktor-kotlinx-serialization-json:1.0.0")
testImplementation("io.github.nomisrev:gcp-pubsub-test:1.0.0")
}
```

### Maven

Add dependencies (you can also add other modules that you need):

```xml

<dependency>
<groupId>io.github.nomisrev</groupId>
<artifactId>gcp-pubsub-ktor</artifactId>
<version>1.0.0</version>
</dependency>
```
56 changes: 39 additions & 17 deletions build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,27 +1,49 @@
import org.jetbrains.dokka.gradle.DokkaTaskPartial
import org.jetbrains.kotlin.gradle.dsl.KotlinProjectExtension

plugins {
kotlin("jvm") version "1.7.21"
id("io.kotest.multiplatform") version "5.5.4"
base
alias(libs.plugins.kotlin.jvm)
alias(libs.plugins.kotest.multiplatform)
alias(libs.plugins.dokka)
alias(libs.plugins.kover)
alias(libs.plugins.publish)
alias(libs.plugins.knit)
alias(libs.plugins.spotless)
}

group = "org.example"
version = "1.0-SNAPSHOT"

repositories {
mavenCentral()
}

tasks.withType<Test> {
useJUnitPlatform()
}
subprojects {
group = "io.github.nomisrev"

[email protected]<DokkaTaskPartial>().configureEach {
[email protected]<KotlinProjectExtension>()?.sourceSets?.forEach { kotlinSourceSet ->
dokkaSourceSets.named(kotlinSourceSet.name) {
includes.from("README.MD")
perPackageOption {
matchingRegex.set(".*\\.internal.*")
suppress.set(true)
}
externalDocumentationLink("https://kotlinlang.org/api/kotlinx.coroutines/")
skipDeprecated.set(true)
reportUndocumented.set(false)
val baseUrl: String = checkNotNull(properties["pom.smc.url"]?.toString())

dependencies {
implementation(kotlin("stdlib"))
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4")
implementation("com.google.cloud:google-cloud-pubsub:1.122.1")
kotlinSourceSet.kotlin.srcDirs.filter { it.exists() }.forEach { srcDir ->
sourceLink {
localDirectory.set(srcDir)
remoteUrl.set(uri("$baseUrl/blob/main/${srcDir.relativeTo(rootProject.rootDir)}").toURL())
remoteLineSuffix.set("#L")
}
}
}
}
}

testImplementation("io.arrow-kt:arrow-fx-coroutines:1.1.3")
testImplementation("io.kotest:kotest-property:5.5.4")
testImplementation("io.kotest:kotest-assertions-core:5.5.4")
testImplementation("io.kotest:kotest-runner-junit5:5.5.4")
testImplementation("org.testcontainers:gcloud:1.17.6")
tasks.withType<AbstractPublishToMaven> {
dependsOn(tasks.withType<Sign>())
}
}
65 changes: 65 additions & 0 deletions common-api/README.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Module common-api

KotlinX integration module for Google's api-core module. If you need to work with
Google's [`ApiFuture`](https://cloud.google.com/java/docs/reference/api-common/latest/com.google.api.core.ApiFutures).

Extension functions:

| **Name** | **Description**
|--------------------------|---------------------------------------------------
| [ApiFuture.await]() | Awaits for completion of the future (cancellable)
| [ApiFuture.asDeferred]() | Converts a deferred value to the future

## Example

Given the following functions defined in some Java API based on Guava:

```java
public ApiFuture<AckResponse> ack(); // starts async acknowledging of message
```

We can consume this API from Kotlin coroutine to load two images and combine then asynchronously.
The resulting function returns `ListenableFuture<Image>` for ease of use back from Guava-based Java code.

```kotlin
suspend fun processMessage(record: PubsubRecord): Unit {
println(record.message.data.toStringUtf8())
when (val response = record.ack().await()) {
SUCCESSFUL -> println("Message was successfully acknowledged. Will not be redelivered.")
else -> println("Acknowledgment failed, message might be redelivered.")
}
}
```

Note that this module should be used only for integration with existing Java APIs based on `ApiFuture`.
Writing pure-Kotlin code that uses `ApiFuture` is highly not recommended, since the resulting APIs based
on the futures are quite error-prone. See the discussion on
[Asynchronous Programming Styles](https://github.com/Kotlin/kotlin-coroutines/blob/master/kotlin-coroutines-informal.md#asynchronous-programming-styles)
for details on general problems pertaining to any future-based API and keep in mind that `ApiFuture` exposes
a _blocking_ method [get](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html#get--) that makes
it especially bad choice for coroutine-based Kotlin code.

## Using in your projects

### Gradle

Add dependencies (you can also add other modules that you need):

```kotlin
dependencies {
implementation("io.github.nomisrev:google-common-api:1.0.0")
}
```

### Maven

Add dependencies (you can also add other modules that you need):

```xml

<dependency>
<groupId>io.github.nomisrev</groupId>
<artifactId>google-common-api</artifactId>
<version>1.0.0</version>
</dependency>
```
22 changes: 22 additions & 0 deletions common-api/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
plugins {
id(libs.plugins.kotlin.jvm.get().pluginId)
id(libs.plugins.dokka.get().pluginId)
id(libs.plugins.kover.get().pluginId)
alias(libs.plugins.knit)
alias(libs.plugins.publish)
}

repositories {
mavenCentral()
}

configure<JavaPluginExtension> {
toolchain {
languageVersion.set(JavaLanguageVersion.of(8))
}
}

dependencies {
api(libs.coroutines)
api(libs.google.api)
}
Loading
Loading