Terraform deploying Amazon EC2 Autoscaling Group and AWS Load Balancers

This is the next article about using Terraform to create EC2 autoscaling group and the different load balancing options for EC2 instances. This setup depends on my previous blog post about using Terraform to deploy a AWS VPC so please read this first. In my Github repository you will find all the needed Terraform files ec2.tf and vpc.tf to deploy the full environment.

EC2 resource overview:

Let’s start with the launch configuration and creating the autoscaling group. I am using eu-west-1 and a standard Ubuntu 16.04 AMI. The instances are created in the private subnet and don’t get a public IP address assigned but have internet access via the NAT gateway:

resource "aws_launch_configuration" "autoscale_launch" {
  image_id = "${lookup(var.aws_amis, var.aws_region)}"
  instance_type = "t2.micro"
  security_groups = ["${aws_security_group.sec_web.id}"]
  key_name = "${aws_key_pair.auth.id}"
  user_data = <<-EOF
              #!/bin/bash
              sudo apt-get -y update
              sudo apt-get -y install nginx
              EOF
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_autoscaling_group" "autoscale_group" {
  launch_configuration = "${aws_launch_configuration.autoscale_launch.id}"
  vpc_zone_identifier = ["${aws_subnet.PrivateSubnetA.id}","${aws_subnet.PrivateSubnetB.id}","${aws_subnet.PrivateSubnetC.id}"]
  load_balancers = ["${aws_elb.elb.name}"]
  min_size = 3
  max_size = 3
  tag {
    key = "Name"
    value = "autoscale"
    propagate_at_launch = true
  }
}

I also created a few security groups to allow the traffic,  please have look for more detail in the ec2.tf.

Autoscaling Group

Now the configuration for a AWS Elastic (Classic) Load Balancer:

resource "aws_elb" "elb" {
  name = "elb"
  security_groups = ["${aws_security_group.sec_lb.id}"]
  subnets            = ["${aws_subnet.PublicSubnetA.id}","${aws_subnet.PublicSubnetB.id}","${aws_subnet.PublicSubnetC.id}"]
  cross_zone_load_balancing   = true
  health_check {
    healthy_threshold = 2
    unhealthy_threshold = 2
    timeout = 3
    interval = 30
    target = "HTTP:80/"
  }
  listener {
    lb_port = 80
    lb_protocol = "http"
    instance_port = "80"
    instance_protocol = "http"
  }
}

Elastic Load Balancer (Classic LB)

Use the Application Load Balancing (ALB) for more advanced web load balancing which only support http and https protocols. You start with creating the ALB resource, afterwards creating the target group where you can define stickiness and health checks. The listener defines which protocol type the ALB uses and assigns the target group. In the end you attach the target- with the autoscaling group:

resource "aws_lb" "alb" {  
  name            = "alb"  
  subnets         = ["${aws_subnet.PublicSubnetA.id}","${aws_subnet.PublicSubnetB.id}","${aws_subnet.PublicSubnetC.id}"]
  security_groups = ["${aws_security_group.sec_lb.id}"]
  internal        = false 
  idle_timeout    = 60   
  tags {    
    Name    = "alb"    
  }   
}

resource "aws_lb_target_group" "alb_target_group" {  
  name     = "alb-target-group"  
  port     = "80"  
  protocol = "HTTP"  
  vpc_id   = "${aws_vpc.default.id}"   
  tags {    
    name = "alb_target_group"    
  }   
  stickiness {    
    type            = "lb_cookie"    
    cookie_duration = 1800    
    enabled         = true 
  }   
  health_check {    
    healthy_threshold   = 3    
    unhealthy_threshold = 10    
    timeout             = 5    
    interval            = 10    
    path                = "/"    
    port                = 80
  }
}

resource "aws_lb_listener" "alb_listener" {  
  load_balancer_arn = "${aws_lb.alb.arn}"  
  port              = 80  
  protocol          = "http"
  
  default_action {    
    target_group_arn = "${aws_lb_target_group.alb_target_group.arn}"
    type             = "forward"  
  }
}

resource "aws_autoscaling_attachment" "alb_autoscale" {
  alb_target_group_arn   = "${aws_lb_target_group.alb_target_group.arn}"
  autoscaling_group_name = "${aws_autoscaling_group.autoscale_group.id}"
}

Application Load Balancer (ALB)

ALB Target Group

The Network Load Balancing (NLB) is very similar to the configuration like the ALB only that it supports the TCP protocol which should be only used for performance because of the limited health check functionality:

resource "aws_lb" "nlb" {
  name               = "nlb"
  internal           = false
  load_balancer_type = "network"
  subnets            = ["${aws_subnet.PublicSubnetA.id}","${aws_subnet.PublicSubnetB.id}","${aws_subnet.PublicSubnetC.id}"]
  enable_cross_zone_load_balancing  = true
  tags {
    Name = "nlb"
  }
}

resource "aws_lb_target_group" "nlb_target_group" {  
  name     = "nlb-target-group"  
  port     = "80"  
  protocol = "TCP"  
  vpc_id   = "${aws_vpc.default.id}"   
  tags {    
    name = "nlb_target_group"    
  }     
}

resource "aws_lb_listener" "nlb_listener" {  
  load_balancer_arn = "${aws_lb.nlb.arn}"  
  port              = 80  
  protocol          = "TCP"
  
  default_action {    
    target_group_arn = "${aws_lb_target_group.nlb_target_group.arn}"
    type             = "forward"  
  }
}

resource "aws_autoscaling_attachment" "nlb_autoscale" {
  alb_target_group_arn   = "${aws_lb_target_group.nlb_target_group.arn}"
  autoscaling_group_name = "${aws_autoscaling_group.autoscale_group.id}"
}

Network Load Balancer (NLB)

NLB Target Group

Let’s run terraform apply:

[email protected]:~/aws-terraform$ terraform apply
data.aws_availability_zones.available: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_autoscaling_attachment.alb_autoscale
      id:                                          
      alb_target_group_arn:                        "${aws_lb_target_group.alb_target_group.arn}"
      autoscaling_group_name:                      "${aws_autoscaling_group.autoscale_group.id}"

  + aws_autoscaling_attachment.nlb_autoscale
      id:                                          
      alb_target_group_arn:                        "${aws_lb_target_group.nlb_target_group.arn}"
      autoscaling_group_name:                      "${aws_autoscaling_group.autoscale_group.id}"

...

Plan: 41 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

...

aws_lb.nlb: Creation complete after 2m53s (ID: arn:aws:elasticloadbalancing:eu-west-1:...:loadbalancer/net/nlb/235e69c61779b723)
aws_lb_listener.nlb_listener: Creating...
  arn:                               "" => ""
  default_action.#:                  "" => "1"
  default_action.0.target_group_arn: "" => "arn:aws:elasticloadbalancing:eu-west-1:552276840222:targetgroup/nlb-target-group/7b3c10cbdd411669"
  default_action.0.type:             "" => "forward"
  load_balancer_arn:                 "" => "arn:aws:elasticloadbalancing:eu-west-1:552276840222:loadbalancer/net/nlb/235e69c61779b723"
  port:                              "" => "80"
  protocol:                          "" => "TCP"
  ssl_policy:                        "" => ""
aws_lb_listener.nlb_listener: Creation complete after 0s (ID: arn:aws:elasticloadbalancing:eu-west-1:.../nlb/235e69c61779b723/dfde2530387b470f)

Apply complete! Resources: 41 added, 0 changed, 0 destroyed.

Outputs:

alb_dns_name = alb-1295224636.eu-west-1.elb.amazonaws.com
elb_dns_name = elb-611107604.eu-west-1.elb.amazonaws.com
nlb_dns_name = nlb-235e69c61779b723.elb.eu-west-1.amazonaws.com
[email protected]:~/aws-terraform$

Together with the VPC configuration from my previous article, this deploys the different load balancers and provides you the DNS names as an output and ready to use.

Over the coming weeks I will optimise the Terraform code and move some of the resource settings into the variables.tf file to make this more scaleable.

If you like this article, please share your feedback and leave a comment.

Howto Update Citrix NetScaler Firmware

Log on to the NetScaler appliance with SSH, such as PuTTY. Use the nsroot credentials to log in to the appliance.

Switch to the shell prompt.

Last login: Tue Mar  4 00:03:13 2014 from 10.49.9.110
Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994
        The Regents of the University of California.  All rights reserved.
 Done
> 
> shell
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
[email protected]#

Run the following command to change to the default installation directory:

[email protected]# cd /var/nsinstall/
[email protected]# ls
10.1nsinstall   installns_state
[email protected]# cd 10.1nsinstall/
[email protected]# ls
build_123.9
[email protected]# mkdir build_124.13

Upload new firmware in the build directory build-10.1-124.13_nc.tgz

Run the following command to extract the firmware:

[email protected]# cd build_124.13/
[email protected]# ls
build-10.1-124.13_nc.tgz
[email protected]# tar xzvf build-10.1-124.13_nc.tgz 
.ns.version
ns-10.1-124.13.gz
ns-10.1-124.13.sha2
installns
nsconfig
bootloader.tgz
help.tgz
CitrixNetScalerManagementPackSCOM2012.msi
CitrixNetScalerLoadBalancer.msi
BaltimoreCyberTrustRoot.cert
BaltimoreCyberTrustRoot_CH.cert
Citrix_Access_Gateway.dmg
macversion.txt
apidoc.tgz
NSConfig.wsdl
NSStat.wsdl
ns-10.1-124.13-gui.tar
ns-10.1-124.13-nitro-java.tgz
ns-10.1-124.13-nitro-csharp.tgz
ns-10.1-124.13-nitro-rest.tgz
ns-10.1-124.13-nitro-perl-samples.tgz
ns-10.1-124.13-nitro-python-samples.tgz
vmware-tools.tgz

Run the following command to install the software you have downloaded:

[email protected]# ./installns 

installns version (10.1-124.13) kernel (ns-10.1-124.13.gz)

  The Netscaler version 10.1-124.13 checksum file is located on 
  http://www.mycitrix.com under Support > Downloads > Citrix NetScaler.
  Select the Release 10.1-124.13 link and expand the "Show Documentation" link
  to view the SHA2 checksum file for build 10.1-124.13.

  There may be a pause of up to 3 minutes while data is written to the flash.
  Do not interrupt the installation process once it has begun.

Installation will proceed in 5 seconds, CTRL-C to abort
Installation is starting ...
VPX platform. Skipping CallHome checks.

Copying ns-10.1-124.13.gz to /flash/ns-10.1-124.13.gz ... 
.......................................................
Installing XML API documentation...
Installing NSConfig.wsdl...
Installing NSStat.wsdl...
Installing online help...
Installing SCOM Management Pack...
Installing LoadBalancer Pack...
Installing GUI...
Installing Mac binary and Mac version file...
Installing NITRO...
Installing Jazz certificate ...
Installing Call Home certificate ...
Creating after upgrade script ...

Installation has completed.

Reboot NOW? [Y/N] Y
Rebooting ...

NetScaler Exchange 2013 Load Balancing

Here an example how to configure Microsoft Exchange 2013 load balancing on Citrix NetScaler

Add Exchange Client Access (CAS) Servers

add server EXCHANGE-CAS01 10.1.0.101
add server EXCHANGE-CAS02 10.1.0.102

Create Service Groups

add serviceGroup service-EXCHANGE-OWA SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP YES -appf DISABLED

bind serviceGroup service-EXCHANGE-OWA EXCHANGE-CAS01 443 -CustomServerID “\”None\””
bind serviceGroup service-EXCHANGE-OWA EXCHANGE-CAS02 443 -CustomServerID “\”None\””
bind serviceGroup service-EXCHANGE-OWA -monitorName https

Import SSL cert

add ssl certKey mail-exchange-cas -cert “/nsconfig/ssl/mail-exchange-cas.pem” -key “/nsconfig/ssl/mail-exchange-cas.pem”

Create and Configure Virtual Server for Exchange

add lb vserver vserver-EXCHANGE-OWA SSL 192.168.0.1 443 -persistenceType SOURCEIP -cltTimeout 180
set ssl vserver vserver-EXCHANGE-OWA -tls11 DISABLED -tls12 DISABLED
bind lb vserver vserver-EXCHANGE-OWA service-EXCHANGE-OWA

Bind SSL to Service Groups and Virtual Server

bind ssl serviceGroup service-EXCHANGE-OWA -certkeyName mail-exchange-cas
bind ssl vserver vserver-EXCHANGE-OWA -certkeyName mail-exchange-cas

Some more information you find in the Citrix deployment guide for Exchange

Citrix NetScaler (Update)

Almost 3 years ago I evaluated and implemented for my ex company F5 BIG-IP 8950 load balancer; now for my new company I start implementing Citrix NetScaler VPX for our Windows infrastructre (Lync, Exchange and Citrix). Here a short overview how it is integrate in the two data centre’s:

From the first look I like the NetScaler, the CLI is a bit easier to understand from what I think but will see how it goes over the next weeks regarding balancing compare to F5 😉

I like the policy based routing implementation on the Netscaler, here a short example:

add ns pbr mgmt-access ALLOW -srcIP = 10.1.0.200 -destIP = 192.168.0.1-192.168.0.254 -nextHop 10.1.0.254 -priority 10

Access to the 10.1.0.200 will be routed to the gateway 10.1.0.254 even you have a default gateway configured what shows to another direction.

In my set-up you need to configure policy based routing if server in the Windows backend network try to access virtual server IPs in the LB-Transfer network otherwise you have asymetric routing.

add ns pbr VIP-WIND-DC02 ALLOW -srcIP = 10.2.0.1-10.2.0.100 -destIP = 10.1.0.1-10.1.0.254 -nextHop 10.2.0.254 -priority 11