Skip to content

Commit

Permalink
JPERF-831 Migrate CI from CircleCI to Github Actions. Change the pipe…
Browse files Browse the repository at this point in the history
…line config to use IAM role and OIDC token for assuming an identity on AWS.

This change allows getting rid of the access key and secret. While applying changes I ran into a problem where some ubuntu images were not available on a particular AWS region. To fix that I had to bump both aws-resources and aws-infrastructure and some other required dependencies.
  • Loading branch information
ewefie committed Jan 31, 2023
1 parent 97b23e4 commit 30e2b27
Show file tree
Hide file tree
Showing 10 changed files with 214 additions and 163 deletions.
79 changes: 0 additions & 79 deletions .circleci/config.yml

This file was deleted.

95 changes: 95 additions & 0 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: Build and test

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
schedule:
- cron: '0 0 * * 0'
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Compile
uses: gradle/gradle-build-action@v2
with:
gradle-version: wrapper
arguments: compileTestKotlin
- name: Build
uses: gradle/gradle-build-action@v2
with:
arguments: build
- name: Upload test reports
uses: actions/upload-artifact@v3
if: always()
with:
name: test-reports
path: ./build/reports/tests
- name: Upload test results
uses: actions/upload-artifact@v3
if: always()
with:
name: test-results
path: ./build/test-results
build-check:
runs-on: ubuntu-latest
needs: build
steps:
- run: echo "All build jobs successful."
test-integration:
runs-on: ubuntu-latest
needs: build-check
env:
JIRA_LICENSE: '${{ secrets.JIRA_LICENSE }}'
TERM: dumb
permissions:
id-token: write
contents: read
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Prepare Jira license
run: echo $JIRA_LICENSE > jira-license.txt
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: eu-west-1
role-duration-seconds: 14400
- name: Generate workflow URL
run: echo "WORKFLOW_URL=$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" >> $GITHUB_ENV
- name: Test integration
uses: gradle/gradle-build-action@v2
continue-on-error: true # expect it to sometimes flake
with:
gradle-version: wrapper
arguments: testIntegration
env:
USER: ${{ env.WORKFLOW_URL }}
- name: Test integration # if the last run was green, this should be quick, expect the rerun to be green
uses: gradle/gradle-build-action@v2
with:
arguments: testIntegration
env:
USER: ${{ env.WORKFLOW_URL }}
- name: Upload test reports
uses: actions/upload-artifact@v3
if: always()
with:
name: test-reports
path: ./build/reports/tests
- name: Upload test results
uses: actions/upload-artifact@v3
if: always()
with:
name: test-results
path: ./build/test-results
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
[![CircleCI](https://circleci.com/gh/atlassian/jira-hardware-exploration.svg?style=svg)](https://circleci.com/gh/atlassian/jira-hardware-exploration)

[![Build and test](https://github.com/atlassian/jira-hardware-exploration/actions/workflows/build-and-test.yml/badge.svg)](https://github.com/atlassian/jira-hardware-exploration/actions/workflows/build-and-test.yml)
### Automated Jira hardware recommendations

This runs the entire hardware recommendation.
Expand All @@ -22,17 +21,17 @@ If the Bamboo agent goes offline after 12 hours, rerun the plan. This can take 3
2. Set up AWS credentials according to [default AWS credentials]
3. Create a `jira-license.txt` file and fill it with a Jira license
4. Run `recommendHardware` Gradle task
* From terminal: `./gradlew recommendHardware`
* Or in short: `./gradlew recHar`
* Or from IntelliJ 2019+: `Run anything` (e.g. double tap `Ctrl`) and type `gradle recommendHardware`
* From terminal: `./gradlew recommendHardware`
* Or in short: `./gradlew recHar`
* Or from IntelliJ 2019+: `Run anything` (e.g. double tap `Ctrl`) and type `gradle recommendHardware`
5. Read the logs and look for results in `build/jpt-workspace`

### Caching

At the beginning of the run, the results from S3 cache (if any) is downloaded and merged with local results.
Then the local results are reused. Only the results that are missing will be run.
Use this to your advantage. If the build flakes, rerun to just fill in the missing subresults.
Naturally, [Bamboo] does not have any local results so it will always download the entire S3 cache.
Naturally, [Bamboo] does not have any local results so it will always download the entire S3 cache.

The S3 cache requires read/write permissions to the S3 bucket,
so either match AWS creds to the bucket or change the bucket in [test source].
Expand Down
2 changes: 1 addition & 1 deletion build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ dependencies {
testCompile(project(":virtual-users"))
testCompile("com.atlassian.performance.tools:jira-performance-tests:[3.3.0,4.0.0)")
testCompile("com.atlassian.performance.tools:infrastructure:[4.14.0,5.0.0)")
testCompile("com.atlassian.performance.tools:virtual-users:[3.6.2,4.0.0)")
testCompile("com.atlassian.performance.tools:virtual-users:[3.6.2,3.12.0)")
testCompile("com.atlassian.performance.tools:jira-software-actions:[1.1.0,2.0.0]")
testCompile("com.atlassian.performance.tools:aws-infrastructure:[2.15.0,3.0.0)")
testCompile("com.atlassian.performance.tools:aws-resources:[1.3.4,2.0.0)")
Expand Down
22 changes: 11 additions & 11 deletions gradle/dependency-locks/testCompile.lockfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,19 @@ com.amazonaws:aws-java-sdk-sts:1.11.817
com.amazonaws:aws-java-sdk-support:1.11.817
com.amazonaws:jmespath-java:1.11.817
com.atlassian.data:random-data:1.4.3
com.atlassian.performance.tools:aws-infrastructure:2.21.4
com.atlassian.performance.tools:aws-resources:1.6.1
com.atlassian.performance.tools:concurrency:1.1.0
com.atlassian.performance.tools:infrastructure:4.17.1
com.atlassian.performance.tools:aws-infrastructure:2.25.8
com.atlassian.performance.tools:aws-resources:1.9.1
com.atlassian.performance.tools:concurrency:1.1.2
com.atlassian.performance.tools:infrastructure:4.22.2
com.atlassian.performance.tools:io:1.2.0
com.atlassian.performance.tools:jira-actions:3.13.3
com.atlassian.performance.tools:jira-performance-tests:3.6.0
com.atlassian.performance.tools:jira-software-actions:1.3.3
com.atlassian.performance.tools:jvm-tasks:1.1.0
com.atlassian.performance.tools:report:3.8.2
com.atlassian.performance.tools:ssh:2.3.1
com.atlassian.performance.tools:jira-actions:3.18.0
com.atlassian.performance.tools:jira-performance-tests:3.6.1
com.atlassian.performance.tools:jira-software-actions:1.4.2
com.atlassian.performance.tools:jvm-tasks:1.2.3
com.atlassian.performance.tools:report:3.13.0
com.atlassian.performance.tools:ssh:2.4.3
com.atlassian.performance.tools:virtual-users:3.11.1
com.atlassian.performance.tools:workspace:2.0.1
com.atlassian.performance.tools:workspace:2.0.2
com.atlassian.performance:selenium-js:1.0.1
com.fasterxml.jackson.core:jackson-annotations:2.6.0
com.fasterxml.jackson.core:jackson-core:2.9.4
Expand Down
22 changes: 11 additions & 11 deletions gradle/dependency-locks/testCompileClasspath.lockfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,19 @@ com.amazonaws:aws-java-sdk-s3:1.11.817
com.amazonaws:aws-java-sdk-sts:1.11.817
com.amazonaws:aws-java-sdk-support:1.11.817
com.amazonaws:jmespath-java:1.11.817
com.atlassian.performance.tools:aws-infrastructure:2.21.4
com.atlassian.performance.tools:aws-resources:1.6.1
com.atlassian.performance.tools:concurrency:1.1.0
com.atlassian.performance.tools:infrastructure:4.17.1
com.atlassian.performance.tools:aws-infrastructure:2.25.8
com.atlassian.performance.tools:aws-resources:1.9.1
com.atlassian.performance.tools:concurrency:1.1.2
com.atlassian.performance.tools:infrastructure:4.22.2
com.atlassian.performance.tools:io:1.2.0
com.atlassian.performance.tools:jira-actions:3.13.3
com.atlassian.performance.tools:jira-performance-tests:3.6.0
com.atlassian.performance.tools:jira-software-actions:1.3.3
com.atlassian.performance.tools:jvm-tasks:1.1.0
com.atlassian.performance.tools:report:3.8.2
com.atlassian.performance.tools:ssh:2.3.1
com.atlassian.performance.tools:jira-actions:3.18.0
com.atlassian.performance.tools:jira-performance-tests:3.6.1
com.atlassian.performance.tools:jira-software-actions:1.4.2
com.atlassian.performance.tools:jvm-tasks:1.2.3
com.atlassian.performance.tools:report:3.13.0
com.atlassian.performance.tools:ssh:2.4.3
com.atlassian.performance.tools:virtual-users:3.11.1
com.atlassian.performance.tools:workspace:2.0.1
com.atlassian.performance.tools:workspace:2.0.2
com.atlassian.performance:selenium-js:1.0.1
com.fasterxml.jackson.core:jackson-annotations:2.6.0
com.fasterxml.jackson.core:jackson-core:2.9.4
Expand Down
22 changes: 11 additions & 11 deletions gradle/dependency-locks/testRuntime.lockfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,19 @@ com.amazonaws:aws-java-sdk-sts:1.11.817
com.amazonaws:aws-java-sdk-support:1.11.817
com.amazonaws:jmespath-java:1.11.817
com.atlassian.data:random-data:1.4.3
com.atlassian.performance.tools:aws-infrastructure:2.21.4
com.atlassian.performance.tools:aws-resources:1.6.1
com.atlassian.performance.tools:concurrency:1.1.0
com.atlassian.performance.tools:infrastructure:4.17.1
com.atlassian.performance.tools:aws-infrastructure:2.25.8
com.atlassian.performance.tools:aws-resources:1.9.1
com.atlassian.performance.tools:concurrency:1.1.2
com.atlassian.performance.tools:infrastructure:4.22.2
com.atlassian.performance.tools:io:1.2.0
com.atlassian.performance.tools:jira-actions:3.13.3
com.atlassian.performance.tools:jira-performance-tests:3.6.0
com.atlassian.performance.tools:jira-software-actions:1.3.3
com.atlassian.performance.tools:jvm-tasks:1.1.0
com.atlassian.performance.tools:report:3.8.2
com.atlassian.performance.tools:ssh:2.3.1
com.atlassian.performance.tools:jira-actions:3.18.0
com.atlassian.performance.tools:jira-performance-tests:3.6.1
com.atlassian.performance.tools:jira-software-actions:1.4.2
com.atlassian.performance.tools:jvm-tasks:1.2.3
com.atlassian.performance.tools:report:3.13.0
com.atlassian.performance.tools:ssh:2.4.3
com.atlassian.performance.tools:virtual-users:3.11.1
com.atlassian.performance.tools:workspace:2.0.1
com.atlassian.performance.tools:workspace:2.0.2
com.atlassian.performance:selenium-js:1.0.1
com.fasterxml.jackson.core:jackson-annotations:2.6.0
com.fasterxml.jackson.core:jackson-core:2.9.4
Expand Down
22 changes: 11 additions & 11 deletions gradle/dependency-locks/testRuntimeClasspath.lockfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,19 @@ com.amazonaws:aws-java-sdk-sts:1.11.817
com.amazonaws:aws-java-sdk-support:1.11.817
com.amazonaws:jmespath-java:1.11.817
com.atlassian.data:random-data:1.4.3
com.atlassian.performance.tools:aws-infrastructure:2.21.4
com.atlassian.performance.tools:aws-resources:1.6.1
com.atlassian.performance.tools:concurrency:1.1.0
com.atlassian.performance.tools:infrastructure:4.17.1
com.atlassian.performance.tools:aws-infrastructure:2.25.8
com.atlassian.performance.tools:aws-resources:1.9.1
com.atlassian.performance.tools:concurrency:1.1.2
com.atlassian.performance.tools:infrastructure:4.22.2
com.atlassian.performance.tools:io:1.2.0
com.atlassian.performance.tools:jira-actions:3.13.3
com.atlassian.performance.tools:jira-performance-tests:3.6.0
com.atlassian.performance.tools:jira-software-actions:1.3.3
com.atlassian.performance.tools:jvm-tasks:1.1.0
com.atlassian.performance.tools:report:3.8.2
com.atlassian.performance.tools:ssh:2.3.1
com.atlassian.performance.tools:jira-actions:3.18.0
com.atlassian.performance.tools:jira-performance-tests:3.6.1
com.atlassian.performance.tools:jira-software-actions:1.4.2
com.atlassian.performance.tools:jvm-tasks:1.2.3
com.atlassian.performance.tools:report:3.13.0
com.atlassian.performance.tools:ssh:2.4.3
com.atlassian.performance.tools:virtual-users:3.11.1
com.atlassian.performance.tools:workspace:2.0.1
com.atlassian.performance.tools:workspace:2.0.2
com.atlassian.performance:selenium-js:1.0.1
com.fasterxml.jackson.core:jackson-annotations:2.6.0
com.fasterxml.jackson.core:jackson-core:2.9.4
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,24 @@ object IntegrationTestRuntime {
private const val roleArn: String = "arn:aws:iam::695067801333:role/server-gdn-bamboo"
private val region = Regions.EU_WEST_1

fun prepareAws() = Aws(
credentialsProvider = AWSCredentialsProviderChain(
STSAssumeRoleSessionCredentialsProvider.Builder(
roleArn,
UUID.randomUUID().toString()
).build(),
ProfileCredentialsProvider("jpt-dev"),
EC2ContainerCredentialsProviderWrapper(),
WebIdentityTokenCredentialsProvider.builder()
.roleArn(roleArn)
.roleSessionName(UUID.randomUUID().toString())
.build(),
DefaultAWSCredentialsProviderChain()
),
region = region,
regionsWithHousekeeping = listOf(Regions.EU_WEST_1),
capacity = TextCapacityMediator(region),
batchingCloudformationRefreshPeriod = Duration.ofSeconds(20)
)
fun prepareAws() = Aws.Builder(region)
.credentialsProvider(
AWSCredentialsProviderChain(
STSAssumeRoleSessionCredentialsProvider.Builder(
roleArn,
UUID.randomUUID().toString()
).build(),
ProfileCredentialsProvider("jpt-dev"),
EC2ContainerCredentialsProviderWrapper(),
WebIdentityTokenCredentialsProvider.builder()
.roleArn(roleArn)
.roleSessionName(UUID.randomUUID().toString())
.build(),
DefaultAWSCredentialsProviderChain()
)
)
.regionsWithHousekeeping(listOf(Regions.EU_WEST_1))
.capacity(TextCapacityMediator(region))
.batchingCloudformationRefreshPeriod(Duration.ofSeconds(20))
.build()
}
Loading

0 comments on commit 30e2b27

Please sign in to comment.