Add a Terraform configuration to deploy lnt.llvm.org #128
Conversation
|
CC @lukel97 I went ahead and gave this a shot, I was curious to understand the whole pipeline |
|
@boomanaiden154 I also created the appropriately-named secrets in the Github Actions of this repository, however they all have fake values at the moment. |
lukel97
left a comment
lukel97
left a comment
There was a problem hiding this comment.
Thanks for fleshing this out, I'm not sure if you've tried deploying this to a test AWS account yet but it looks like it's missing a security group/ingress rules etc., so the web server won't be reachable by any public traffic IIUC.
I've also got some terraform files written here, it would be good to collaborate on this. I don't want us to step on each others toes so I'll just leave review comments for now but let me know if you'd rather have me just commit directly to the branch.
Please feel free to commit directly to the branch. Sorry, I didn't know you had started on this already. I did try to deploy an EC2 instance in my personal account, however that account is blocked right now (IDK why) so I haven't gotten very far. This was intended to be a starting point. Feel free to push whatever changes you have to the branch. |
|
|
||
| LNT_DB_PASSWORD=${__db_password__} | ||
| LNT_AUTH_TOKEN=${__auth_token__} | ||
| docker compose --file compose.yaml up |
There was a problem hiding this comment.
IIUC these user data scripts are only called when the instance is first initialized, but not e.g. rebooted. So we probably want to change the docker-compose restart policy to be unless-stopped so the containers get relaunched on a reboot
There was a problem hiding this comment.
This depends upon how we set it up. I was thinking it might be better to setup the machine to be a clean slate on every boot, and mount a persistent volume that actually contains the DB. That makes it super easy to change system software inside TF.
Hah, my AWS account was also blocked, I'm currently waiting for AWS support to verify my identity. I feel your pain :) |
|
|
||
| LNT_DB_PASSWORD=${__db_password__} | ||
| LNT_AUTH_TOKEN=${__auth_token__} | ||
| docker compose --file compose.yaml up |
There was a problem hiding this comment.
This depends upon how we set it up. I was thinking it might be better to setup the machine to be a clean slate on every boot, and mount a persistent volume that actually contains the DB. That makes it super easy to change system software inside TF.
docker/lnt.llvm.org/main.tf
Outdated
| provider "aws" { | ||
| region = "us-west-2" | ||
| } | ||
|
|
There was a problem hiding this comment.
We also need a way to set the terraform state. We use a GCS bucket in the premerge cluster to do this. https://github.com/llvm/llvm-zorg/blob/87d07e600970abf419046d2ab6083b2d64240bce/premerge/main.tf#L31
Otherwise state isn't saved across checkouts, which means things won't work.
docker/lnt.llvm.org/main.tf
Outdated
| region = "us-west-2" | ||
| } | ||
|
|
||
| variable "lnt_db_password" { |
There was a problem hiding this comment.
These should probably be data resources that reference secrets stored inside AWS's secret manager.
https://github.com/llvm/llvm-zorg/blob/87d07e600970abf419046d2ab6083b2d64240bce/premerge/main.tf#L113 is how we set this up for premerge. Not sure exactly how to do this for AWS.
There was a problem hiding this comment.
My preference would actually be to leave it as-is and to store these secrets in the Github Actions secrets, unless you see a problem with that. That makes it easier to reuse this Terraform configuration by using their own AWS keys -- that's almost the only thing required.
|
Just to avoid duplicate work, @lukel97 my AWS account is functional now and I am working on addressing the PR comments. LMK if you had also made progress on your end before I invest too much time into it. |
ldionne
force-pushed
the
review/terraform
branch
3 times, most recently
from
f01d36e to
c591f0b
Compare
This patch adds a Terraform configuration file that should allow deploying to an EC2 instance. It requires a few secrets to be made available to Github Actions.
ldionne
force-pushed
the
review/terraform
branch
from
c591f0b to
469b91d
Compare
|
@lukel97 @boomanaiden154 This should be ready for review now. With this Terraform configuration, I'm able to deploy a server in my personal AWS account. The only remaining steps would be:
Also CC @petrhosek as member of the infrastructure group. |
3 participants
This patch adds a Terraform configuration file that should allow deploying to an EC2 instance. It requires a few secrets to be made available to Github Actions.