Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch from Ant to Gradle #389

Merged
merged 2 commits into from
Apr 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#
# https://help.github.com/articles/dealing-with-line-endings/
#
# Linux start script should use lf
/gradlew text eol=lf

# These are Windows script files and should use crlf
*.bat text eol=crlf

16 changes: 4 additions & 12 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,21 +24,14 @@ jobs:
include:
# the current production setup
- java: '8'
codecov: true
mongo: 'mongodb-linux-x86_64-3.6.13'
wired_tiger: 'false'
ant_test: 'test'
gradle_test: 'test'
# test all code w/ java 11
- java: '11'
codecov: true
mongo: 'mongodb-linux-x86_64-3.6.13'
wired_tiger: 'false'
ant_test: 'test'
# test the latest 3X version of mongo against just the mongo storage code
- java: '11'
mongo: 'mongodb-linux-x86_64-3.6.23'
wired_tiger: 'true'
ant_test: 'test_mongo_storage'
gradle_test: 'test'
steps:
- uses: actions/checkout@v3

Expand All @@ -59,7 +52,7 @@ jobs:
# set up jars
git clone https://github.com/kbase/jars
export JARSDIR=$(pwd)/jars/lib/jars/

# set up mongo
wget -q http://fastdl.mongodb.org/linux/${{matrix.mongo}}.tgz
tar xfz ${{matrix.mongo}}.tgz
Expand All @@ -77,10 +70,9 @@ jobs:
- name: Run tests
shell: bash
run: |
ant ${{matrix.ant_test}}
gradle ${{matrix.gradle_test}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run: ./gradlew test is sufficient? Probably don't need gradle_test: 'test' unless I am missing something important.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for now, but it's a placeholder in case we ever split out the mongo tests as noted in a TODO earlier in the file IIRC


- name: Upload coverage to Codecov
if: matrix.codecov
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
Expand Down
27 changes: 22 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,23 @@
FROM kbase/sdkbase2 as build

COPY . /tmp/groups
RUN cd /tmp \
&& git clone https://github.com/kbase/jars \
&& cd groups \
&& ant buildwar
WORKDIR /tmp/groups

# dependencies take a while to D/L, so D/L & cache before the build so code changes don't cause
# a new D/L
# can't glob *gradle because of the .gradle dir
COPY build.gradle gradlew settings.gradle /tmp/groups/
COPY gradle/ /tmp/groups/gradle/
RUN ./gradlew dependencies

# Now build the code
COPY deployment/ /tmp/groups/deployment/
COPY jettybase/ /tmp/groups/jettybase/
COPY src /tmp/groups/src/
COPY war /tmp/groups/war/
# for the git commit
COPY .git /tmp/groups/.git/
RUN ./gradlew war


FROM kbase/kb_jre:latest

Expand All @@ -15,6 +28,7 @@ ARG BRANCH=develop

COPY --from=build /tmp/groups/deployment/ /kb/deployment/
COPY --from=build /tmp/groups/jettybase/ /kb/deployment/jettybase/
COPY --from=build /tmp/groups/build/libs/groups.war /kb/deployment/jettybase/webapps/ROOT.war

# The BUILD_DATE value seem to bust the docker cache when the timestamp changes, move to
# the end
Expand All @@ -30,6 +44,9 @@ ENV KB_DEPLOYMENT_CONFIG=/kb/deployment/conf/deployment.cfg

RUN chmod -R a+rwx /kb/deployment/conf /kb/deployment/jettybase/

# TODO BUILD update to no longer use dockerize and take env vars (e.g. like Collections).
# TODO BUILD Use subsections in the ini file / switch to TOML

ENTRYPOINT [ "/kb/deployment/bin/dockerize" ]

# Here are some default params passed to dockerize. They would typically
Expand Down
4 changes: 4 additions & 0 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# KBase Groups Service release notes

## 0.1.7

* The build tool has been switched from Ant to Gradle.

## 0.1.6

### Admin notes:
Expand Down
167 changes: 167 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
/*
* This file was generated by the Gradle 'init' task.
*/

plugins {
id 'java'
id 'war'
id 'jacoco'
id 'org.ajoberstar.grgit' version '4.1.1'
}

// TODO NOW run tests from Eclipse w/o specifying classpath manually & remove sourceSets & classpath
// TODO NOW update any ant refs in docs to gradle

repositories {
mavenCentral()
}

task buildGitCommitFile {
doLast {
def commitId = grgit.head().id
// is there a variable for builddir/classes/java/main?
file("$buildDir/classes/java/main/us/kbase/groups/gitcommit").text = commitId
}
}
Comment on lines +19 to +25
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What benefits do we get for saving a commit in a file? Verification?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That commit winds up here: https://ci.kbase.us/services/groups/

so you know what code the server is running


compileJava {
// TODO BUILD remove when we no longer support java 8, use `options.release = 11` if needed
java.sourceCompatibility = JavaVersion.VERSION_1_8
java.targetCompatibility = JavaVersion.VERSION_1_8
finalizedBy buildGitCommitFile
}

test {
/*
* TODO TEST Figure out why tests fail without this and remove. Might have something to do
* with the stfuLoggers() call in many of the tests, might kill logging for tests that
* require it
* Although it seems to make Mongo start up correctly as well which is odd
*/
forkEvery = 1
/*
* TODO TEST split mongo wrapper tests out.
* Set up GHA to run the full test suite with a single version of mongo and run the
* mongo tests with matrixed mongo versions.
* Since the tests only take a minute or two this isn't high ROI.
*/
systemProperty "KBASE_GROUPS_TEST_CONFIG", "./test.cfg"
maxHeapSize = "3G"
testLogging {
exceptionFormat = 'full'
showStandardStreams = true
}
finalizedBy jacocoTestReport
}

// TODO TEST add a test that starts the server in a docker container and checks some simple cmds

jacocoTestReport {
reports {
xml.required = true
csv.required = true
}
}

war {
webXml = file('war/web.xml')
}

// Custom java project layout
sourceSets {
main {
java {
srcDirs = ["src"]
exclude '**/test/**'
}
}
test {
java {
srcDirs = ["src"]
include '**/test/**'
}
resources {
srcDirs = ["src"]
include "**/*wsjars"
}
}
}

configurations {
// can't directly access testImplementation, so extend and access
testimpl.extendsFrom testImplementation
}

def fromURL = { url, name ->
File file = new File("$buildDir/download/${name}.jar")
file.parentFile.mkdirs()
if (!file.exists()) {
new URL(url).withInputStream { downloadStream ->
file.withOutputStream { fileOut ->
fileOut << downloadStream
}
}
}
files(file.absolutePath)
}

dependencies {

compileOnly 'javax.servlet:javax.servlet-api:3.0.1'

implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/auth/kbase-auth-0.4.4.jar',
'kbase-auth-0.4.4'
)
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/workspace/WorkspaceClient-0.8.0.jar',
'WorkspaceClient-0.8.0'
)
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/catalog/kbase-catalog-client-2.1.3.jar',
'kbase-catalog-client-2.1.3'
)
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/common/kbase-common-0.0.25.jar',
'kbase-common-0.0.25'
)
// Pull from jars repo vs a maven repository to avoid the JNA dependency
// TODO DEPS Need to rework the java common logger to not use syslog4j at all since it's abandonware
// and has a ton of CVEs, even in the newer versions.
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/syslog4j/syslog4j-0.9.46.jar',
'syslog4j-0.9.46'
)
implementation 'ch.qos.logback:logback-classic:1.1.2'
implementation 'org.ini4j:ini4j:0.5.2'
implementation 'org.mongodb:mongo-java-driver:3.8.2'
implementation 'org.apache.kafka:kafka-clients:2.1.0'
implementation 'com.google.guava:guava:18.0'
implementation 'org.slf4j:slf4j-api:1.7.25'
implementation 'com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.5.4'
implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.5.4'
implementation 'org.glassfish.jersey.containers:jersey-container-servlet:2.23.2'
implementation 'org.glassfish.jersey.media:jersey-media-json-jackson:2.23.2'
implementation 'com.github.zafarkhaja:java-semver:0.9.0'

testImplementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/auth2/kbase-auth2-test-shadow-all-0.7.0.jar',
'kbase-auth2-test-shadow-all-0.7.0'
)
testImplementation 'nl.jqno.equalsverifier:equalsverifier:3.1.10'
testImplementation 'junit:junit:4.12'
testImplementation 'org.mockito:mockito-core:3.0.0'
testImplementation('org.mock-server:mockserver-netty:5.4.1') {
// otherwise a newer jackson implementation is brought in and tests make a poo poo
// TODO BUILD on upgrade switch to the no dependencies version
exclude group: "com.fasterxml.jackson.core"
}
testImplementation 'org.eclipse.jetty:jetty-servlet:9.3.11.v20160721'
}

task showTestClassPath {
doLast {
configurations.testimpl.each { println it }
}
}

Loading
Loading