-
Notifications
You must be signed in to change notification settings - Fork 824
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add terraform code to deploy patch release notification service #6853
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
# See the OWNERS docs at https://go.k8s.io/owners | ||
|
||
approvers: | ||
- release-engineering-approvers | ||
- sig-k8s-infra-leads | ||
reviewers: | ||
- release-engineering-reviewers |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# patch-release-notification service | ||
|
||
This terraform code deploys the code to notify the K8s community about the cherry pick deadline for the patch releases. | ||
|
||
The patch-release-notification code can be found in `cmd/patch-release-notification` | ||
|
||
Right now, the terraform is applied manually by the release managers that have access to the AWS account for the SIG-release. | ||
|
||
# Deploy | ||
|
||
To deploy will require to have both repositories cloned: | ||
|
||
- https://github.com/kubernetes/release/ | ||
- https://github.com/kubernetes/k8s.io | ||
|
||
from https://github.com/kubernetes/k8s.io | ||
|
||
``` | ||
# loging to the AWS ECR | ||
$ aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 433650573627.dkr.ecr.us-west-2.amazonaws.com | ||
|
||
$ cd k8s.io/infra/aws/terraform/sig-release/patch-release-notification | ||
|
||
$ terraform init | ||
|
||
$ terraform plan -out=plan.out | ||
|
||
$ terraform apply "plan.out" | ||
``` | ||
|
||
_note_: you will need to configure your AWS credentials before. | ||
_note2_: this is setup to run in the AWS SIG-Release account. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
/* | ||
Copyright 2024 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
provider "aws" { | ||
region = var.region | ||
} | ||
|
||
resource "aws_sesv2_email_identity" "sig_release_email_identity" { | ||
email_identity = var.email_identity | ||
} | ||
|
||
resource "aws_iam_role" "lambda_ses_role" { | ||
name = "lambda_ses_role" | ||
assume_role_policy = jsonencode({ | ||
Version = "2012-10-17", | ||
Statement = [ | ||
{ | ||
Action = "sts:AssumeRole", | ||
Effect = "Allow", | ||
Principal = { | ||
Service = "lambda.amazonaws.com" | ||
} | ||
} | ||
] | ||
}) | ||
} | ||
|
||
resource "aws_iam_policy" "lambda_ses_policy" { | ||
name = "lambda_ses_policy" | ||
description = "IAM policy for Lambda to access SES" | ||
policy = jsonencode({ | ||
Version = "2012-10-17", | ||
Statement = [ | ||
{ | ||
Action = [ | ||
"ses:SendEmail", | ||
"ses:SendRawEmail" | ||
], | ||
Effect = "Allow", | ||
Resource = "*" | ||
} | ||
] | ||
}) | ||
} | ||
|
||
resource "aws_ecr_repository" "repo" { | ||
name = var.repository | ||
image_tag_mutability = "MUTABLE" | ||
|
||
image_scanning_configuration { | ||
scan_on_push = false | ||
} | ||
} | ||
|
||
resource "aws_ecr_repository" "cherry_pick_notification_repo" { | ||
name = "${var.repository}/patch-release-notification" | ||
image_tag_mutability = "MUTABLE" | ||
|
||
image_scanning_configuration { | ||
scan_on_push = false | ||
} | ||
} | ||
|
||
resource "ko_build" "cherry_pick_notification_image" { | ||
repo = aws_ecr_repository.cherry_pick_notification_repo.repository_url | ||
base_image = "public.ecr.aws/lambda/provided:al2023" | ||
working_dir = "${path.module}/../../../../../../release/cmd/patch-release-notification" | ||
importpath = "k8s.io/release/cmd/patch-release-notification" | ||
} | ||
|
||
resource "aws_iam_role_policy_attachment" "lambda_ses_policy_attachment" { | ||
role = aws_iam_role.lambda_ses_role.name | ||
policy_arn = aws_iam_policy.lambda_ses_policy.arn | ||
} | ||
|
||
resource "aws_lambda_function" "cherry_pick_notification" { | ||
function_name = "patch-release-notification" | ||
role = aws_iam_role.lambda_ses_role.arn | ||
image_uri = ko_build.cherry_pick_notification_image.image_ref | ||
package_type = "Image" | ||
|
||
environment { | ||
variables = { | ||
FROM_EMAIL = var.email_identity | ||
TO_EMAIL = var.to_email | ||
SCHEDULE_PATH = var.schedule_path | ||
DAYS_TO_ALERT = var.days_to_alert | ||
NO_MOCK = var.no_mock | ||
AWS_REGION = var.region | ||
} | ||
} | ||
} | ||
|
||
resource "aws_cloudwatch_event_rule" "trigger_lambda_cron" { | ||
name = "trigger-patch-release-notification-cron" | ||
description = "Trigger Lambda function on a schedule" | ||
schedule_expression = "cron(0 16 * * ? *)" # Example cron expression to run at 16:00 PM UTC every day | ||
} | ||
|
||
resource "aws_cloudwatch_event_target" "trigger_lambda_target" { | ||
rule = aws_cloudwatch_event_rule.trigger_lambda_cron.name | ||
target_id = "send_email_lambda" | ||
arn = aws_lambda_function.cherry_pick_notification.arn | ||
} | ||
|
||
resource "aws_lambda_permission" "allow_cloudwatch_to_invoke_lambda" { | ||
statement_id = "AllowExecutionFromCloudWatch" | ||
action = "lambda:InvokeFunction" | ||
function_name = aws_lambda_function.cherry_pick_notification.function_name | ||
principal = "events.amazonaws.com" | ||
source_arn = aws_cloudwatch_event_rule.trigger_lambda_cron.arn | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
/* | ||
Copyright 2024 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
output "email_identity_arn" { | ||
value = aws_sesv2_email_identity.sig_release_email_identity.arn | ||
} |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should also commit There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will commit, but as we discussed here, I will need to refactor not to build the image here and use a prebuilt image that will be done in another job. but, I need to know how to push the image to the AWS account and more specifically to the aws account we are interested in, how that can be done? cc @upodroid There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cpanato I would try the following approach if it's okay with @upodroid:
There might some additional steps between 1 and 2 in terms of configuring the AWS account, but we have done that before. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If y'all are okay with that, I can take care of setting up everything because I think I have access to all the resources/accounts that we need for this. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we will build it using cloudbuild? or can i use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You should be able to use |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
/* | ||
Copyright 2024 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
variable "region" { | ||
description = "The AWS region to deploy the resources" | ||
type = string | ||
} | ||
|
||
variable "email_identity" { | ||
description = "The email address or domain to verify" | ||
type = string | ||
} | ||
|
||
variable "to_email" { | ||
description = "The email address to send the notification" | ||
type = string | ||
} | ||
|
||
variable "no_mock" { | ||
description = "if will send the message to [email protected] or just internal" | ||
type = bool | ||
} | ||
|
||
variable "days_to_alert" { | ||
description = "when to send the notification" | ||
type = number | ||
} | ||
|
||
variable "schedule_path" { | ||
description = "path where we can find the schedule.yaml" | ||
type = string | ||
} | ||
|
||
variable "repository" { | ||
description = "The ECR repository to use for the image" | ||
type = string | ||
default = "" | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
/* | ||
Copyright 2024 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
terraform { | ||
backend "s3" { | ||
bucket = "tf-state-sig-release" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this bucket manually created? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, we need to bootstrap some how :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What I have been doing before for both SIG Release and SIG K8s Infra is using CloudFormation: https://github.com/kubernetes/k8s.io/blob/main/infra/aws/cloudformation/iam/cdn.packages.k8s.io/cloudformation.yaml Maybe we can take the same approach here if that's okay for y'all. It still requires manual bootstrapping (i.e. running that CloudFormation script), but all the resources are created automatically. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yikes! cloudformation, i thought that did not exist anymore LOL but sounds good to me, i can add that, and delete the bucket that I've create before. |
||
key = "cherry-pick-notification" | ||
region = "us-west-2" | ||
} | ||
|
||
required_version = "1.8.0" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we relax the version if we don't have any dependency on 1.8.0? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sure |
||
|
||
required_providers { | ||
ko = { | ||
source = "ko-build/ko" | ||
} | ||
|
||
aws = { | ||
source = "hashicorp/aws" | ||
version = "5.51.1" | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not a big fan of doing this because I think it's going to be very hard to automate the Terraform execution in the future. @upodroid Maybe you can help here, I'm thinking out loud, but can we setup SA with IRSA in EKS build cluster that would allow pushing to this registry, and then we run a ProwJob that's going to use that SA to build the image and push it to the registry?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the main idea here is to use terraform to build the image when is needed and deploy the lambda, then everytime we run that and if there are changes in the code it will generate a new image
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but it is not hard to automate, in the automation we clone both repos, we do this in several places already for other things