KEY CONCEPTS:
- Configuration management – install and manage software
on existing servers: Ansible, Chef, Puppet
- Infrastructure Orchestration – provision servers and other
infrastructure components: Terraform, AWS CloudFormation
TERRAFORM
- free, works w multiple platforms: https://www.terraform.io/docs/providers/index.html
- single binary - https://www.terraform.io/downloads.html. On Windows – add path to the .exe to system PATH variable (Control Panel, etc.)
- Terraform on AWS: https://registry.terraform.io/providers/hashicorp/aws/latest/docs
- Example of an EC2 instance config: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance
To test this out, create a sample.tf as below:
# Configure the AWS Provider
provider "aws" {
region = "us-east-1"
access_key
= "my-access-key "
secret_key
= "my-secret-key "
}
resource "aws_instance" “my_awesome_ec2” {
ami = "ami-096fda3c22c1c990a"
instance_type = "t2.micro"
}
- “provider” defines what API you are running against
- “resource” is specific to a provider
- “terraform init” downloads the necessary plugins for the given provider
- To run, execute the following on the same folder as the above .tf:
terraform init
terraform plan
terraform apply
OR
terraform apply -auto-approve
-
NOTE: if numerous resources are being created and there is a cap
on API calls, can use “terraform plan -refresh=false” to limit an API refresh call
on every single resource when changing only a subset OR “terraform plan -refresh=false
-target=<resource_name>” to change a specific resource only
- To clean up and remove all created resources run “terraform destroy”
To remove a specific resource:
- terraform destroy -target aws_instance.my_awesome_ec2
- OR comment out a specific resource that is no longer needed and re-run “terraform plan” / “apply”
Format / validate
-
https://www.terraform.io/docs/commands/fmt.html
-
Run “terraform fmt” to format the .tf file removing extra
indentation, etc.
-
Run “terraform validate” to check if confoguration supplied
in the file is valid
State file
-
Terraform saves it’s “current view” of the deployed infrastructure
in terraform.tfstate
-
Uses this file to diff against updates in the user’s .tf
-
If changes are made to the infra manually – terraform will pick up
and highlight the differences
-
.tf = desired state of infra, terraform.tfstate = current stare of
infra
-
Run “terraform refresh” to reconcile and update the state
file
-
“terraform show” provides a more user-friendly read of the
state file
-
NOTE: only items explicitly defined in .tf (desired state) get
flagged as outstanding if manually changed in the background (ex: manually
switch a Sec Group on EC2 w/out siting it in .tf)
Plugin version
-
There is a terraform plugin running on the infra provider side for
the client tor interact with
-
It is possible to specify a version of the plugin desired; can
specify a range >=, etc.
-
By default, the latest version is used if none is specified
-
Can work with 3rd party infra providers – requires
download of a 3rd party plugin (wget <source>)
provider "aws" {
region = "us-east-1"
access_key
= "my-access-key "
secret_key
= "my-secret-key "
version = “2.7”
}
Attribute reference and output
values
-
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip
-
Can expose attributes of a newly created resource
o
print on-screen for reference after “terraform apply”
o
use those as input for resources created subsequently
-
Output values are visible in “
"outputs": {},” section in the state file
output "instance_ip_addr" {
value =
aws_instance.server.private_ip
}
Association of an ID with an
instance or a network interface
-
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip_association
resource "aws_eip_association"
"eip_assoc" {
instance_id = aws_instance.web.id
allocation_id = aws_eip.example.id
}
Variables
-
https://www.terraform.io/docs/configuration/variables.html
variable "instancetype" {
}
resource "aws_instance"
"example" {
instance_type = var.instancetype
}
-
Can specify a variable value on command line at launch:
o terraform plan -var="instancetype=t2.small"
-
if no default is set (i.e. variable "image_id" {}), terraform
will prompt for value at launch
-
BEST PRACTICE
o define variables and default values in variables.tf
o move out variable name/value pairs and default overrides into terraform.tfvars
o can optionally save variables into a custom-ly name file
- Can specify a variable value on command line at launch:
variable "instancetype" {
}
- Can store variables as lists
variable "instancetype" {
default = ["m5.large",
“m5.xlarge”, “t2.medium”]
}
Count
- Speed up creation of multiple resource by using list of variables and count
- Can apply conditional expressions: “condition ? true : false”
type = list
default = ["dev-loadbalancer", "stage-loadbalanacer","prod-loadbalancer"]
}
resource "aws_iam_user" "lb" {
name = var.elb_names[count.index]
count = 3
path = "/system/"
}
variable "istest" {}
resource "aws_instance" "prod" {
ami = "ami-082b5a644766e0e6f"
instance_type = "t2.large"
count = var.istest == false ? 1 : 0
}
Functions
- https://www.terraform.io/docs/configuration/functions.html
- A finite list of built-in functions is available – as documented above
- Can run “terraform console” to test out function code
Data sources
- https://www.terraform.io/docs/configuration/data-sources.html
- Can filter and extract data from the provider and use in configuration:
owners = ["amazon"]
filter {
name = "name"
values = ["amzn2-ami-hvm*"]
}
}
resource "aws_instance" "instance-1" {
ami = data.aws_ami.app_ami.id
instance_type = "t2.micro"
}
Debugging
- https://www.terraform.io/docs/internals/debugging.html
- Set environment variable TF_LOG to any value to enable detailed logging
- Set to TRACE (most verbose), DEBUG, INFO, WARN or ERROR to change the verbosity
- TF_LOG_PATH sends the log output to be appended to a specific file:
o set TF_LOG_PATH=./terraform.log
- Can use “terraform output <name>” to view variable values
Dynamic blocks
- Can dynamically construct repeating blocks:
default = [8200, 8201,8300, 9200, 9500]
}
resource "aws_security_group" "dynamicsg" {
name = "dynamic-sg"
description = "Ingress for Vault"
dynamic "ingress" {
for_each = var.sg_ports
iterator = port
content {
from_port = port.value
to_port = port.value
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
}
- Can introduce “iterator” – name of the element to be set. If iterator is omitted – element name is derived from the name of the dynamic block itself (“ingress” above)
Taint
- “terraform taint <ex: aws_security_group.allow_all>” can be used to manually mark a resource as tainted and thus in need of being destroyed and recreated on next apply.
- the command marks a resource as “tainted” in the .tfstate file
- careful to check the dependencies – ex: recreating an EC2 might change its IP
Splat
- [*] style expressions to represent the running and increasing total
Plan to graph or file
- “terraform graph > graph.dot” helps convert resource definition ns into a grap. Can us graphwiz tool for visual representation:
o yum install graphwiz
o cat graph.dot | dot -Tsvg > graph.svg
o open svg (on frnt end) using graphwiz
- “terraform plan -out=demoplan” saves plan into a binary file. Can subsequently do “terraform apply demofile” to load directly from the file
Provisioners
- https://www.terraform.io/docs/provisioners/index.html
- The local-exec provisioner invokes a local executable after a resource is created. Ex: run an Ansilble playbook
# ...
provisioner "local-exec" {
command = "echo ${aws_instance.web.private_ip} >> private_ips.txt"
}
}
- Can add “when = destroy” to make it an destroy-time provisioner
- The remote-exec provisioner invokes a script on a remote resource after it is created
# ...
provisioner "remote-exec" {
inline = [
"puppet apply",
"consul join ${aws_instance.web.private_ip}",
]
}
- To ensure the remote provisioner can connect to the created resource, define a connection inside the provisioner block
- https://www.terraform.io/docs/provisioners/connection.html
type = "ssh"
user = "root"
password = "${var.root_password}"
host = "${var.host}"
}
- Can tweak the connection to load a key from a local file
- Can set “on failure” to "continue" or "fail" to specify what should happen of the provisioner fails
Modules
- https://www.terraform.io/docs/configuration/modules.html
- A module is a container for multiple resources that are used together. DRY – do not repeat yourself. Ex: separate out an EC2 definition use it across multiple projects
source = "../../modules/ec2"
}
- Need to do “terraform init” when a new module is added
- Verified provider modules are available at the terraform registry:
- https://registry.terraform.io/browse/modules
- https://github.com/terraform-aws-modules
- Can pull in modules directly from the terraform registry
source = "hashicorp/consul/aws"
version = "0.0.5"
servers = 3
}
- Can pull from Git: source = "git::https://mystuff.com/stuff.git?ref=1.2.0"
- Can do git::ssh
Workspace
- https://www.terraform.io/docs/cloud/workspaces/index.html
- A way of organizing infrastructure. Ex: dev/uat/prod
- “terraform workspace”
instance_type = lookup(var.instance_type,terraform.workspace)
}
variable "instance_type" {
default = {
default = "t2.nano"
dev = "t2.micro"
prd = "t2.large"
}
}
- The above will do the instance type lookup based on the name of the current workspace
Backends
- https://www.terraform.io/docs/backends/index.html
- If storing code in github or like, bet proactive is to avoid storing .tfstate, variables etc. in a public repo to avoid exposing secrets. This can be worked around with .gitignore:
o https://www.terraform.io/docs/backends/types/index.html
- Nonpublic storage options exist:
o https://www.terraform.io/docs/backends/types/index.html
- Need to be careful as automatic state locking of files at backend is not available with all backends Example: s3 allows file overwrites, need to implement locking manually using dynamoDb:
backend "s3" {
bucket = "<remote-backend_bucket_name>"
key = "myproject.tfstate"
region = "us-east-1"
access_key = "<access_key>"
secret_key = "<secret_key>"
dynamodb_table = "<state_lock_tabel_name>"
}
}
State
- https://www.terraform.io/docs/state/index.html
- Manipulate the content of .tfstate directly - helps avoid destroying/recreating resources, etc.
o terraform state list
o terraform state pull
- useful for team collaboration
Import
- https://www.terraform.io/docs/import/index.html
- https://learn.hashicorp.com/tutorials/terraform/state-import
- Can import configuration of an existing resource created outside of terraform
- Writes into the state file only
- Need to manually define the resource first to be able map it out uniquely
Alias / Profile
- By default, terraform operates in one region per provider
- Can add alias to provider definition to introduce additional regions / availability zones
}
provider "aws" {
region = " us-east-1"
profile = "account2"
}
resource "aws_eip" "myeipEast" {
provider = "aws.awsEast"
}
- Can add a profile setting to the resource definition as well as to the credentials file if need to operate on separate provider accounts
Alias / Profile
- Can integrate with AWS STS to assume a pre-define role
- https://support.hashicorp.com/hc/en-us/articles/360041289933-Using-AWS-AssumeRole-with-the-AWS-Terraform-Provider
No comments:
Post a Comment