Skip to content

Commit

Permalink
Migrated to Cloud Build
Browse files Browse the repository at this point in the history
  • Loading branch information
ivankorn committed Sep 26, 2019
1 parent d409acd commit 0a6ff54
Show file tree
Hide file tree
Showing 28 changed files with 466 additions and 1,007 deletions.
1 change: 0 additions & 1 deletion .ruby-version

This file was deleted.

10 changes: 9 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [3.0.1] - 2019-XX-YY

### Changed

- Migrated to Cloud Build. [#33]

## [3.0.0] - 2019-07-23

### Changed
Expand Down Expand Up @@ -45,13 +51,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Initial release of log export module.

[Unreleased]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v3.0.0...HEAD
[Unreleased]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v3.0.1...HEAD
[3.0.1]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v3.0.0...v3.0.1
[3.0.0]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v2.3.0...v3.0.0
[2.3.0]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v2.2.0...v2.3.0
[2.2.0]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v2.1.0...v2.2.0
[2.1.0]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v2.0.0...v2.1.0
[2.0.0]: https://github.com/terraform-google-modules/terraform-google-log-export/compare/v1.0.0...v2.0.0
[1.0.0]: https://github.com/terraform-google-modules/terraform-google-log-export/releases/tag/v1.0.0
[#33]: https://github.com/terraform-google-modules/terraform-google-log-export/pull/33
[#22]: https://github.com/terraform-google-modules/terraform-google-log-export/pull/22
[#19]: https://github.com/terraform-google-modules/terraform-google-log-export/pull/19
[#18]: https://github.com/terraform-google-modules/terraform-google-log-export/pull/18
Expand Down
101 changes: 101 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Contributing

This document provides guidelines for contributing to the module.

## Dependencies

The following dependencies must be installed on the development system:

- [Docker Engine][docker-engine]
- [Google Cloud SDK][google-cloud-sdk]
- [make]

## Generating Documentation for Inputs and Outputs

The Inputs and Outputs tables in the READMEs of the root module,
submodules, and example modules are automatically generated based on
the `variables` and `outputs` of the respective modules. These tables
must be refreshed if the module interfaces are changed.

### Execution

Run `make generate_docs` to generate new Inputs and Outputs tables.

## Integration Testing

Integration tests are used to verify the behaviour of the root module,
submodules, and example modules. Additions, changes, and fixes should
be accompanied with tests.

The integration tests are run using [Kitchen][kitchen],
[Kitchen-Terraform][kitchen-terraform], and [InSpec][inspec]. These
tools are packaged within a Docker image for convenience.

The general strategy for these tests is to verify the behaviour of the
[example modules](./examples/), thus ensuring that the root module,
submodules, and example modules are all functionally correct.

### Test Environment
The easiest way to test the module is in an isolated test project. The setup for such a project is defined in [test/setup](./test/setup/) directory.

To use this setup, you need a service account with Project Creator access on a folder. Export the Service Account credentials to your environment like so:

```
export SERVICE_ACCOUNT_JSON=$(< credentials.json)
```

You will also need to set a few environment variables:
```
export TF_VAR_project_id="project_id_of_test_project"
export TF_VAR_parent_resource_project="project_id_of_test_project"
export TF_VAR_parent_resource_folder="folder_id_of_test_folder"
export TF_VAR_parent_resource_organization="org_id_of_test_organization"
export TF_VAR_parent_resource_billing_account="billing_account_id_of_test_billing_account"
```

With these settings in place, you can prepare a test project using Docker:
```
make docker_test_prepare
```

### Noninteractive Execution

Run `make docker_test_integration` to test all of the example modules
noninteractively, using the prepared test project.

### Interactive Execution

1. Run `make docker_run` to start the testing Docker container in
interactive mode.

1. Run `kitchen_do create <EXAMPLE_NAME>` to initialize the working
directory for an example module.

1. Run `kitchen_do converge <EXAMPLE_NAME>` to apply the example module.

1. Run `kitchen_do verify <EXAMPLE_NAME>` to test the example module.

1. Run `kitchen_do destroy <EXAMPLE_NAME>` to destroy the example module
state.

## Linting and Formatting

Many of the files in the repository can be linted or formatted to
maintain a standard of quality.

### Execution

Run `make docker_test_lint`.

[docker-engine]: https://www.docker.com/products/docker-engine
[flake8]: http://flake8.pycqa.org/en/latest/
[gofmt]: https://golang.org/cmd/gofmt/
[google-cloud-sdk]: https://cloud.google.com/sdk/install
[hadolint]: https://github.com/hadolint/hadolint
[inspec]: https://inspec.io/
[kitchen-terraform]: https://github.com/newcontext-oss/kitchen-terraform
[kitchen]: https://kitchen.ci/
[make]: https://en.wikipedia.org/wiki/Make_(software)
[shellcheck]: https://www.shellcheck.net/
[terraform-docs]: https://github.com/segmentio/terraform-docs
[terraform]: https://terraform.io/
19 changes: 0 additions & 19 deletions Gemfile

This file was deleted.

196 changes: 53 additions & 143 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,162 +18,72 @@
# Make will use bash instead of sh
SHELL := /usr/bin/env bash

# Docker build config variables
CREDENTIALS_PATH ?= /cft/workdir/credentials.json
DOCKER_TAG_VERSION_DEVELOPER_TOOLS := 0.1.0
DOCKER_IMAGE_DEVELOPER_TOOLS := cft/developer-tools
REGISTRY_URL := gcr.io/cloud-foundation-cicd


# Docker build config variables
DOCKER_ORG := gcr.io/cloud-foundation-cicd
DOCKER_TAG_BASE_KITCHEN_TERRAFORM ?= 2.1.0
DOCKER_REPO_BASE_KITCHEN_TERRAFORM := ${DOCKER_ORG}/cft/kitchen-terraform:${DOCKER_TAG_BASE_KITCHEN_TERRAFORM}

# All is the first target in the file so it will get picked up when you just run 'make' on its own
all: check generate_docs

# Run all available linters
check: check_shell check_python check_golang check_terraform check_docker check_base_files test_check_headers check_headers check_trailing_whitespace

# The .PHONY directive tells make that this isn't a real target and so
# the presence of a file named 'check_shell' won't cause this target to stop
# working
.PHONY: check_shell
check_shell:
@source test/make.sh && check_shell

.PHONY: check_python
check_python:
@source test/make.sh && check_python

.PHONY: check_golang
check_golang:
@source test/make.sh && golang

.PHONY: check_terraform
check_terraform:
@source test/make.sh && check_terraform

.PHONY: check_docker
check_docker:
@source test/make.sh && docker

.PHONY: check_base_files
check_base_files:
@source test/make.sh && basefiles

.PHONY: check_trailing_whitespace
check_trailing_whitespace:
@source test/make.sh && check_trailing_whitespace

.PHONY: test_check_headers
test_check_headers:
@echo "Testing the validity of the header check"
@python test/test_verify_boilerplate.py

.PHONY: check_headers
check_headers:
@source test/make.sh && check_headers

# Integration tests
.PHONY: test_integration
test_integration:
test/ci_integration.sh

.PHONY: generate_docs
generate_docs:
@source test/make.sh && generate_docs

# Versioning
.PHONY: version
version:
@source helpers/version-repo.sh

# Run docker
# Enter docker container for local development
.PHONY: docker_run
docker_run:
docker run --rm -it \
-e PROJECT_ID \
-e PARENT_RESOURCE_PROJECT \
-e PARENT_RESOURCE_FOLDER \
-e PARENT_RESOURCE_BILLING_ACCOUNT \
-e PARENT_RESOURCE_ORGANIZATION \
-e SERVICE_ACCOUNT_JSON \
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-e SUITE \
-v "$(CURDIR)":/cft/workdir \
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
/bin/bash -c "source test/ci_integration.sh && setup_environment && exec /bin/bash"

.PHONY: docker_create
docker_create:
-e TF_VAR_project_id \
-e TF_VAR_parent_resource_project \
-e TF_VAR_parent_resource_folder \
-e TF_VAR_parent_resource_billing_account \
-e TF_VAR_parent_resource_organization \
-v $(CURDIR):/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/bin/bash

# Execute prepare tests within the docker container
.PHONY: docker_test_prepare
docker_test_prepare:
docker run --rm -it \
-e PROJECT_ID \
-e PARENT_RESOURCE_PROJECT \
-e PARENT_RESOURCE_FOLDER \
-e PARENT_RESOURCE_BILLING_ACCOUNT \
-e PARENT_RESOURCE_ORGANIZATION \
-e SERVICE_ACCOUNT_JSON \
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-e SUITE \
-v "$(CURDIR)":/cft/workdir \
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
/bin/bash -c "source test/ci_integration.sh && setup_environment && kitchen create"

.PHONY: docker_converge
docker_converge:
-e TF_VAR_project_id \
-e TF_VAR_parent_resource_project \
-e TF_VAR_parent_resource_folder \
-e TF_VAR_parent_resource_billing_account \
-e TF_VAR_parent_resource_organization \
-v $(CURDIR):/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/execute_with_credentials.sh prepare_environment

# Clean up test environment within the docker container
.PHONY: docker_test_cleanup
docker_test_cleanup:
docker run --rm -it \
-e PROJECT_ID \
-e PARENT_RESOURCE_PROJECT \
-e PARENT_RESOURCE_FOLDER \
-e PARENT_RESOURCE_BILLING_ACCOUNT \
-e PARENT_RESOURCE_ORGANIZATION \
-e SERVICE_ACCOUNT_JSON \
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-e SUITE \
-v "$(CURDIR)":/cft/workdir \
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
/bin/bash -c "source test/ci_integration.sh && setup_environment && kitchen converge"
-v $(CURDIR):/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/execute_with_credentials.sh cleanup_environment

.PHONY: docker_verify
docker_verify:
# Execute integration tests within the docker container
.PHONY: docker_test_integration
docker_test_integration:
docker run --rm -it \
-e PROJECT_ID \
-e PARENT_RESOURCE_PROJECT \
-e PARENT_RESOURCE_FOLDER \
-e PARENT_RESOURCE_BILLING_ACCOUNT \
-e PARENT_RESOURCE_ORGANIZATION \
-e SERVICE_ACCOUNT_JSON \
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-e SUITE \
-v "$(CURDIR)":/cft/workdir \
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
/bin/bash -c "source test/ci_integration.sh && setup_environment && kitchen verify"
-v $(CURDIR):/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/test_integration.sh

.PHONY: docker_destroy
docker_destroy:
# Execute lint tests within the docker container
.PHONY: docker_test_lint
docker_test_lint:
docker run --rm -it \
-e PROJECT_ID \
-e PARENT_RESOURCE_PROJECT \
-e PARENT_RESOURCE_FOLDER \
-e PARENT_RESOURCE_BILLING_ACCOUNT \
-e PARENT_RESOURCE_ORGANIZATION \
-e SERVICE_ACCOUNT_JSON \
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-e SUITE \
-v "$(CURDIR)":/cft/workdir \
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
/bin/bash -c "source test/ci_integration.sh && setup_environment && kitchen destroy"
-v $(CURDIR):/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/test_lint.sh

.PHONY: test_integration_docker
test_integration_docker:
# Generate documentation
.PHONY: docker_generate_docs
docker_generate_docs:
docker run --rm -it \
-e PROJECT_ID \
-e PARENT_RESOURCE_PROJECT \
-e PARENT_RESOURCE_FOLDER \
-e PARENT_RESOURCE_BILLING_ACCOUNT \
-e PARENT_RESOURCE_ORGANIZATION \
-e SERVICE_ACCOUNT_JSON \
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-e SUITE \
-v "$(CURDIR)":/cft/workdir \
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
make test_integration
-v $(CURDIR):/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/bin/bash -c 'source /usr/local/bin/task_helper_functions.sh && generate_docs'

# Alias for backwards compatibility
.PHONY: generate_docs
generate_docs: docker_generate_docs
Loading

0 comments on commit 0a6ff54

Please sign in to comment.