Skip to content

Latest commit

 

History

History
143 lines (117 loc) · 7.26 KB

README.md

File metadata and controls

143 lines (117 loc) · 7.26 KB
Raw

terraform-google-pubsub

This module makes it easy to create Google Cloud Pub/Sub topic and subscriptions associated with the topic.

Compatibility

This module is meant for use with Terraform 0.13+ and tested using Terraform 1.0+. If you find incompatibilities using Terraform >=0.13, please open an issue. If you haven't upgraded and need a Terraform 0.12.x-compatible version of this module, the last released version intended for Terraform 0.12.x is v1.9.0.

Usage

This is a simple usage of the module. Please see also a simple setup provided in the example directory.

module "pubsub" {
  source  = "terraform-google-modules/pubsub/google"
  version = "~> 6.0"

  topic      = "tf-topic"
  project_id = "my-pubsub-project"
  push_subscriptions = [
    {
      name                       = "push"                                               // required
      ack_deadline_seconds       = 20                                                   // optional
      push_endpoint              = "https://example.com"                                // required
      x-goog-version             = "v1beta1"                                            // optional
      oidc_service_account_email = "[email protected]"                                     // optional
      audience                   = "example"                                            // optional
      expiration_policy          = "1209600s"                                           // optional
      dead_letter_topic          = "projects/my-pubsub-project/topics/example-dl-topic" // optional
      max_delivery_attempts      = 5                                                    // optional
      maximum_backoff            = "600s"                                               // optional
      minimum_backoff            = "300s"                                               // optional
      filter                     = "attributes.domain = \"com\""                        // optional
    }
  ]
  pull_subscriptions = [
    {
      name                         = "pull"                                               // required
      ack_deadline_seconds         = 20                                                   // optional
      dead_letter_topic            = "projects/my-pubsub-project/topics/example-dl-topic" // optional
      max_delivery_attempts        = 5                                                    // optional
      maximum_backoff              = "600s"                                               // optional
      minimum_backoff              = "300s"                                               // optional
      filter                       = "attributes.domain = \"com\""                        // optional
      enable_message_ordering      = true                                                 // optional
      service_account              = "[email protected]"          // optional
      enable_exactly_once_delivery = true                                                 // optional
    }
  ]
  bigquery_subscriptions = [
    {
      name                = "bigquery"              // required
      table               = "project.dataset.table" // required
      use_topic_schema    = true                    // optional
      write_metadata      = false                   // optional
      drop_unknown_fields = false                   // optional
    }
  ]
  cloud_storage_subscriptions = [
    {
      name            = "cloud-storage"  // required
      bucket          = "example-bucket" // required
      filename_prefix = "log_events_"    // optional
      filename_suffix = ".avro"          // optional
      max_duration    = "60s"            // optional
      max_bytes       = "10000000"       // optional
      output_format   = "avro"           // optional
      write_metadata  = false            // optional
    }
  ]
}

Inputs

Name Description Type Default Required
bigquery_subscriptions The list of the Bigquery push subscriptions. list(map(string)) [] no
cloud_storage_subscriptions The list of the Cloud Storage push subscriptions. list(map(string)) [] no
create_subscriptions Specify true if you want to create subscriptions. bool true no
create_topic Specify true if you want to create a topic. bool true no
grant_token_creator Specify true if you want to add token creator role to the default Pub/Sub SA. bool true no
message_storage_policy A map of storage policies. Default - inherit from organization's Resource Location Restriction policy. map(any) {} no
project_id The project ID to manage the Pub/Sub resources. string n/a yes
pull_subscriptions The list of the pull subscriptions. list(map(string)) [] no
push_subscriptions The list of the push subscriptions. list(map(string)) [] no
schema Schema for the topic. 
object({
    name       = string
    type       = string
    definition = string
    encoding   = string
  })
 null no
subscription_labels A map of labels to assign to every Pub/Sub subscription. map(string) {} no
topic The Pub/Sub topic name. string n/a yes
topic_kms_key_name The resource name of the Cloud KMS CryptoKey to be used to protect access to messages published on this topic. string null no
topic_labels A map of labels to assign to the Pub/Sub topic. map(string) {} no
topic_message_retention_duration The minimum duration in seconds to retain a message after it is published to the topic. string null no

Outputs

Name Description
id The ID of the Pub/Sub topic
subscription_names The name list of Pub/Sub subscriptions
subscription_paths The path list of Pub/Sub subscriptions
topic The name of the Pub/Sub topic
topic_labels Labels assigned to the Pub/Sub topic
uri The URI of the Pub/Sub topic

Requirements

Installation Dependencies

Configure a Service Account

In order to execute this module you must have a Service Account with the following:

Roles

  • roles/pubsub.admin

Enable APIs

In order to operate with the Service Account you must activate the following APIs on the project where the Service Account was created:

  • Cloud Pub/Sub API

Service Account Credentials

You can pass the service account credentials into this module by setting the following environment variables:

  • GOOGLE_CREDENTIALS
  • GOOGLE_CLOUD_KEYFILE_JSON
  • GCLOUD_KEYFILE_JSON

See more details.