Active Directory with Linux EC2 join (from ASG)

CHANGELOG.md

@@ -11,6 +11,7 @@
 ### Examples
 
 * `ad-ec2`: Demonstrate how an Windows EC2 instance seamlessly joins an Active directory when it gets newly spawned.
+* `ad-linux-asg`: Demonstrate how an Linux EC2 instance (from an ASG) joins an Active directory when it gets newly spawned.
 
 # v0.9.3
 

examples/ad-linux-asg/Makefile

@@ -0,0 +1,74 @@
+.PHONY: init ssh-key plan-vpc plan-subnets plan-gateway plan apply destroy clean
+
+.DEFAULT_GOAL = help
+
+# Hardcoding value of 3 minutes when we check if the plan file is stale
+STALE_PLAN_FILE := `find "tf.out" -mmin -3 | grep -q tf.out`
+
+## Check if tf.out is stale (Older than 2 minutes)
+check-plan-file:
+	@if ! ${STALE_PLAN_FILE} ; then \
+		echo "ERROR: Stale tf.out plan file (older than 3 minutes)!"; \
+		exit 1; \
+	fi
+
+## Runs terraform get and terraform init for env
+init:
+	@terraform get
+	@terraform init
+
+## Create ssh key
+ssh-key:
+	@ssh-keygen -q -N "" -b 4096 -C "SSH key for vpc-scenario-1 example" -f ./id_rsa
+
+## use 'terraform plan' to 'target' the vpc in the vpc module
+plan-vpc:
+	@terraform plan \
+		-target="module.vpc.module.vpc" \
+		-out=tf.out
+
+## use 'terraform plan' to 'target' the public subnets in the vpc module
+plan-subnets:
+	@terraform plan \
+		-target="module.vpc.module.public-subnets" \
+		-out=tf.out
+
+## use 'terraform plan' to 'target' the public gateway in the vpc module
+plan-gateway:
+	@terraform plan \
+		-target="module.vpc.module.public-gateway" \
+		-out=tf.out
+
+## use 'terraform plan' to map out updates to apply
+plan:
+	@terraform plan -out=tf.out
+
+## use 'terraform apply' to apply updates in a 'tf.out' plan file
+apply: check-plan-file
+	@terraform apply tf.out
+
+## use 'terraform destroy' to remove all resources from AWS
+destroy:
+	@terraform destroy
+
+## rm -rf all files and state
+clean:
+	@rm -f tf.out
+	@rm -f id_rsa
+	@rm -f id_rsa.pub
+	@rm -f terraform.tfvars
+	@rm -f terraform.*.backup
+	@rm -f terraform.tfstate
+
+## Show help screen.
+help:
+	@echo "Please use \`make <target>' where <target> is one of\n\n"
+	@awk '/^[a-zA-Z\-\_0-9]+:/ { \
+		helpMessage = match(lastLine, /^## (.*)/); \
+		if (helpMessage) { \
+			helpCommand = substr($$1, 0, index($$1, ":")); \
+			helpMessage = substr(lastLine, RSTART + 3, RLENGTH); \
+			printf "%-30s %s\n", helpCommand, helpMessage; \
+		} \
+	} \
+	{ lastLine = $$0 }' $(MAKEFILE_LIST)

examples/ad-linux-asg/README.md

@@ -0,0 +1,130 @@
+# Active Directory with Linux EC2 join (from ASG)
+
+The terraform code is built on top of
+[vpc-scenario1](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario1.html)
+with two additional private subnets and a NAT gateway on a public
+subnet. This example demonstrate how an Linux EC2 instance present in
+[ASG](https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html)
+joins an Active directory when it gets newly spawned. The only
+difference between this example and the [ad-asg](../ad-asg/) is
+operating system.
+
+We use a custom AMI to have active directory related softwares
+available in our instance.
+
+## AMI Creation
+
+``` shellsession
+$ cd packer
+$ make build
+....
+    amazon-ebs: Stopping instance
+==> amazon-ebs: Waiting for the instance to stop...
+==> amazon-ebs: Creating AMI centos7-server-ami-1569914218 from instance i-074b5b3a18dde102b
+    amazon-ebs: AMI: ami-0473c8e0ea159ff34
+==> amazon-ebs: Waiting for AMI to become ready...
+==> amazon-ebs: Terminating the source AWS instance...
+==> amazon-ebs: Cleaning up any extra volumes...
+==> amazon-ebs: Destroying volume (vol-0ba5fc0a8d3e4d180)...
+==> amazon-ebs: Deleting temporary security group...
+==> amazon-ebs: Deleting temporary keypair...
+Build 'amazon-ebs' finished.
+
+==> Builds finished. The artifacts of successful builds are:
+--> amazon-ebs: AMIs were created:
+us-east-2: ami-0473c8e0ea159ff34
+```
+
+Note the new ami-id built from the above command. You would need to
+put that in the [variables.tf](./variables.tf) for the `ami_id`
+variable.
+
+## Environment creation and deployment
+
+To use this example set up AWS credentials and then run the commands in the 
+following order:
+
+```
+make ssh-key
+make init
+make plan-vpc
+make apply
+make plan-subnets
+make apply
+make plan-gateway
+make apply
+make plan
+make apply
+```
+
+## Execution
+
+Once you run the above commands, you will get an output like this:
+
+``` shellsession
+...
+module.nat-gateway.aws_route_table_association.private-rta[0]: Refreshing state... [id=rtbassoc-0be4f2c71ef12e768]
+module.nat-gateway.aws_route_table_association.private-rta[1]: Refreshing state... [id=rtbassoc-08a1f878abab73841]
+aws_ssm_association.associate_ssm: Refreshing state... [id=996ff9a8-0931-4000-85aa-d01ef536f5a7]
+
+
+Outputs:
+
+asg-name = test-ad-project-asg-cluster20190919093341776000000005
+microsoft-ad_dns_ip_addresses = [
+  "10.23.21.134",
+  "10.23.22.45",
+]
+microsoft-ad_dns_name = dev.fpcomplete.local
+```
+
+## Testing
+
+You need to test that the Linux EC2 instance from the ASG actually
+joined the Active directory. Let's SSH into our instance and verify
+it:
+
+* Find the public IP address of your EC2 instance either via the `aws`
+  cli or through the console.
+* SSH into the instance.
+* Verify if the instance has actually joined the AD domain:
+
+``` shellsession
+[centos@ip-10-23-11-81 ~]$ realm list
+dev.fpcomplete.local
+  type: kerberos
+  realm-name: DEV.FPCOMPLETE.LOCAL
+  domain-name: dev.fpcomplete.local
+  configured: kerberos-member
+  server-software: active-directory
+  client-software: sssd
+  required-package: oddjob
+  required-package: oddjob-mkhomedir
+  required-package: sssd
+  required-package: adcli
+  required-package: samba-common-tools
+  login-formats: %[email protected]
+  login-policy: allow-realm-logins
+```
+
+## Destruction
+
+To destroy the test environment run the following commands:
+
+```
+$ make destroy
+$ make clean
+```
+
+## Debugging
+
+The script execution using `user_data` is usually hard to debug. The
+execution of our [bootstrap script](./bootstrap.linux.txt) results in
+a log which can be viewed [by either SSHing the instance or through
+AWS Console](https://stackoverflow.com/q/15904095/1651941).
+
+## Reference
+
+* [AWS docs on AWS Managed Microsoft AD](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_getting_started.html)
+* [AWS docs on Joining an EC2 instance](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_join_instance.html)
+* [AWS docs on Systems manager and AD](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-systems-manager-dx-domain/)

examples/ad-linux-asg/bootstrap.linux.txt

@@ -0,0 +1,5 @@
+# cloud-config
+# Reference: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
+
+runcmd:
+ - [ sh, -c, 'echo "${ad_password}" | sudo realm join -U Admin@${ad_domain} ${ad_domain}']

examples/ad-linux-asg/locals.tf

@@ -0,0 +1,5 @@
+locals {
+  stage       = "dev"
+  base_domain = "fpcomplete.local"
+  domain      = "${local.stage}.${local.base_domain}"
+}

examples/ad-linux-asg/main.tf

@@ -0,0 +1,116 @@
+provider "aws" {
+  region = var.region
+}
+
+data "aws_availability_zones" "available" {
+}
+
+module "vpc" {
+  source      = "../../modules/vpc-scenario-1"
+  name_prefix = var.name
+  region      = var.region
+  cidr        = var.vpc_cidr
+  azs         = [data.aws_availability_zones.available.names[0]]
+  dns_servers = aws_directory_service_directory.main.dns_ip_addresses
+  extra_tags  = var.extra_tags
+
+  public_subnet_cidrs = var.public_subnet_cidrs
+}
+
+resource "aws_key_pair" "main" {
+  key_name   = var.name
+  public_key = file(var.ssh_pubkey)
+}
+
+module "web-sg" {
+  source      = "../../modules/security-group-base"
+  description = "For my-web-app instances in ${var.name}"
+  name        = "${var.name}-web"
+  vpc_id      = module.vpc.vpc_id
+}
+
+# shared security group, open egress (outbound from nodes)
+module "web-open-egress-rule" {
+  source            = "../../modules/open-egress-sg"
+  security_group_id = module.web-sg.id
+}
+
+resource "aws_security_group_rule" "ssh" {
+  type              = "ingress"
+  from_port         = 22
+  to_port           = 22
+  protocol          = "tcp"
+  cidr_blocks       = ["0.0.0.0/0"]
+  security_group_id = module.web-sg.id
+}
+
+data "template_file" "init" {
+  template = file("${path.module}/bootstrap.linux.txt")
+
+  vars = {
+    ad_password = var.active_directory_password
+    ad_domain   = local.domain
+  }
+}
+
+data "template_cloudinit_config" "config" {
+  # Main cloud-config configuration file.
+  part {
+    filename     = "init.cfg"
+    content_type = "text/cloud-config"
+    content      = "${data.template_file.init.rendered}"
+  }
+}
+
+
+module "web-asg" {
+  source             = "../../modules/asg"
+  ami                = var.ami_id
+  azs                = []
+  name_prefix        = "${var.name}-asg"
+  instance_type      = "t2.micro"
+  max_nodes          = 1
+  min_nodes          = 1
+  public_ip          = true
+  key_name           = aws_key_pair.main.key_name
+  subnet_ids         = module.vpc.public_subnet_ids
+  security_group_ids = [module.web-sg.id]
+
+  root_volume_type = "gp2"
+  root_volume_size = "40"
+
+  user_data = data.template_cloudinit_config.config.rendered
+}
+
+module "private-subnets" {
+  source      = "../../modules/subnets"
+  azs         = slice(data.aws_availability_zones.available.names, 1, 3)
+  vpc_id      = module.vpc.vpc_id
+  name_prefix = "${var.name}-private"
+  cidr_blocks = var.private_subnet_cidrs
+  public      = false
+  extra_tags  = merge(var.extra_tags, var.private_subnet_extra_tags)
+}
+
+module "nat-gateway" {
+  source             = "../../modules/nat-gateways"
+  vpc_id             = module.vpc.vpc_id
+  name_prefix        = var.name
+  nat_count          = length(var.public_subnet_cidrs)
+  public_subnet_ids  = module.vpc.public_subnet_ids
+  private_subnet_ids = module.private-subnets.ids
+  extra_tags         = merge(var.extra_tags, var.nat_gateway_extra_tags)
+}
+
+resource "aws_directory_service_directory" "main" {
+  name     = local.domain
+  password = var.active_directory_password
+  size     = "Small"
+  edition  = "Standard"
+  type     = "MicrosoftAD"
+
+  vpc_settings {
+    vpc_id     = module.vpc.vpc_id
+    subnet_ids = module.private-subnets.ids
+  }
+}

examples/ad-linux-asg/outputs.tf

@@ -0,0 +1,14 @@
+output "microsoft-ad_dns_ip_addresses" {
+  description = "Microsoft AD DNS IP Address"
+  value       = aws_directory_service_directory.main.dns_ip_addresses
+}
+
+output "microsoft-ad_dns_name" {
+  description = "Microsoft AD DNS Name"
+  value       = aws_directory_service_directory.main.name
+}
+
+output "asg-name" {
+  description = "ASG Name"
+  value       = module.web-asg.name
+}

examples/ad-linux-asg/packer/Makefile

@@ -0,0 +1,24 @@
+.PHONY: build validate
+
+.DEFAULT_GOAL = help
+
+## Build CentOS 7 AMI Image
+build:
+	packer build centos7_server.json
+
+## Validate packer template file
+validate:
+	packer validate centos7_server.json
+
+## Show help screen.
+help:
+	@echo "Please use \`make <target>' where <target> is one of\n\n"
+	@awk '/^[a-zA-Z\-\_0-9]+:/ { \
+		helpMessage = match(lastLine, /^## (.*)/); \
+		if (helpMessage) { \
+			helpCommand = substr($$1, 0, index($$1, ":")); \
+			helpMessage = substr(lastLine, RSTART + 3, RLENGTH); \
+			printf "%-30s %s\n", helpCommand, helpMessage; \
+		} \
+	} \
+	{ lastLine = $$0 }' $(MAKEFILE_LIST)