Getting Started with Terraform: Basics, Modules, and State

Terraform is a powerful tool that enables developers to define and manage infrastructure as code. As organizations increasingly rely on infrastructure automation, understanding Terraform becomes crucial to creating scalable, consistent, and maintainable systems. This guide will walk you through the basics of Terraform, introduce reusable modules, manage state effectively, and explain how to structure projects in preparation for using Terragrunt.

Introduction

Terraform is an open-source tool developed by HashiCorp that allows users to define cloud and on-prem infrastructure using a high-level configuration language called HashiCorp Configuration Language (HCL). By defining your infrastructure in code, you can reap the benefits of version control, consistency, and easy collaboration. This practice is commonly known as Infrastructure as Code (IaC).

A proper project structure is essential to keep your Terraform code manageable and scalable as your infrastructure evolves. This guide will explore how Terragrunt can simplify managing multiple environments and reduce the complexity of scaling your infrastructure. By the end of this post, you will have a solid understanding of setting up Terraform, using modules, managing state files, and preparing for advanced workflows using Terragrunt.

Setting Up Terraform

To get started with Terraform, you'll first need to install it on your local machine. Follow these steps:

1. Installation

To avoid including instructions that could quickly become out of date here, refer to the guide for your operating system and use the Terraform installation instructions.

Tip: Consider using tfswitch for easy version management. By using tfswitch you can easily change terraform versions depending on your project.

2. Configure Providers

Providers are responsible for managing the lifecycle of specific resources (e.g., AWS, Azure, Google Cloud). You need to define a provider block to specify which cloud platform or infrastructure provider to use. For the entire list of providers, visit the Terraform Provider Registry

Example of configuring the AWS provider:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {
  region = "us-west-2"
}

By specifying the provider version (~> 4.0), you ensure consistent behavior across different deployments, avoiding any unexpected issues caused by newer versions.

3. Setting Up Authentication

The authentication requirements can differ based on the provider. For example, to interact with AWS, you need to set up environment variables or an AWS credentials file.

Whatever authentication method you use, ensure that you don't put any credentials into your source control in plain text. One of our current favorite practices is to put all our secrets in a shared password manager and then call for passwords from the automation when they're needed.

Creating Basic Infrastructure Components

The core of Terraform is defining infrastructure using resource blocks. Resources represent individual infrastructure components, such as servers, databases, or networks. For our example, we’ll use AWS. However, there are hundreds of providers for different vendors and technologies.

Example: Creating a VPC and EC2 Instance

Here's an example of defining a simple AWS VPC and an EC2 instance:

# Create a VPC
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  
  tags = {
    Name = "main-vpc"
    Environment = "dev"
  }
}

# Create a subnet within the VPC
resource "aws_subnet" "public" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "us-west-2a"
  
  tags = {
    Name = "public-subnet"
    Environment = "dev"
  }
}

# Create an EC2 instance
resource "aws_instance" "web_server" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
  subnet_id     = aws_subnet.public.id
  
  tags = {
    Name = "web-server"
    Environment = "dev"
  }
}

By breaking infrastructure into simple components, you can ensure the initial setup remains easy to manage. As your needs grow, you can start reusing common components to avoid repetition and maintain consistency.

Planning and Applying Terraform

After writing your Terraform configuration, the next step is to apply it to create or modify your infrastructure. Terraform provides several essential commands for this purpose:

# Initialize working directory
terraform init

# Preview the changes
terraform plan -out=tfplan

# Apply the changes
terraform apply "tfplan"

# Alternatively, apply without a saved plan
terraform apply

# Destroy infrastructure when no longer needed
terraform destroy

Terraform's core workflow are these commands (init, plan, apply, and destroy). Following these steps ensures proper control over your infrastructure and avoids unintended changes. As your codebase matures, you can start putting these commands in your CI/CD Pipeline to automate your deployments

Introducing Modules for Reusable Code

Modules are reusable packages of Terraform code that help you create standardized, repeatable infrastructure. This is particularly useful as your infrastructure grows in complexity and scale.

A typical module contains three main files:

• main.tf: Defines the core resource definitions

• variables.tf: Declares variables that can be passed into the module

• outputs.tf: Defines outputs for the module, which can be used by other parts of your configuration

By putting your terraform into reusable modules, not only do you make your code more robust, you also set the stage for implementing Terragrunt and enhancing your entire Terraform IaC workflow

Example: EC2 Instance Module

modules/
└── ec2_instance/
    ├── main.tf
    ├── variables.tf
    └── outputs.tf

main.tf:

resource "aws_instance" "this" {
  ami                    = var.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  vpc_security_group_ids = var.security_group_ids
  key_name               = var.key_name
  
  associate_public_ip_address = var.associate_public_ip
  
  root_block_device {
    volume_type           = var.root_volume_type
    volume_size           = var.root_volume_size
    delete_on_termination = true
    encrypted             = var.enable_volume_encryption
  }

  # Optional additional EBS volume (See caveat below)
  dynamic "ebs_block_device" {
    for_each = var.create_data_volume ? [1] : []
    content {
      device_name           = "/dev/sdh"
      volume_type           = var.data_volume_type
      volume_size           = var.data_volume_size
      encrypted             = var.enable_volume_encryption
      delete_on_termination = true
    }
  }
  
  # User data script for initial setup
  user_data = var.user_data
  
  tags = merge(
    {
      Name = var.name
    },
    var.tags
  )

  # Wait for instance to be ready
  provisioner "remote-exec" {
    inline = ["echo 'Instance is ready'"]

    connection {
      type        = "ssh"
      user        = var.ssh_user
      private_key = var.private_key_content
      host        = self.public_ip
    }
  }
}

You might want to separate your persistent volumes, such as a data EBS volume, into different Terraform modules to limit your damage domain. The “damage domain” is what we like to call the scope of all the resources that will be touched when updating or destroying your environment. You might want to update your computing resource while maintaining the data in the attached storage, and in that case, putting the volume and computing resource in the same module is a bad idea.

variables.tf:

variable "name" {
  description = "Name to be used for the instance"
  type        = string
}

variable "ami_id" {
  description = "The AMI to use for the instance"
  type        = string
}

variable "instance_type" {
  description = "The type of instance to start"
  type        = string
  default     = "t3.micro"
}

variable "subnet_id" {
  description = "The VPC Subnet ID to launch in"
  type        = string
}

variable "security_group_ids" {
  description = "A list of security group IDs to associate with"
  type        = list(string)
  default     = []
}

variable "key_name" {
  description = "Key name of the Key Pair to use for the instance"
  type        = string
  default     = null
}

variable "associate_public_ip" {
  description = "Whether to associate a public IP address with the instance"
  type        = bool
  default     = false
}

variable "root_volume_type" {
  description = "The type of the root volume (standard, gp2, io1)"
  type        = string
  default     = "gp3"
}

variable "root_volume_size" {
  description = "The size of the root volume in gigabytes"
  type        = number
  default     = 20
}

variable "create_data_volume" {
  description = "Whether to create a separate data volume"
  type        = bool
  default     = false
}

variable "data_volume_type" {
  description = "The type of the data volume (standard, gp2, io1)"
  type        = string
  default     = "gp3"
}

variable "data_volume_size" {
  description = "The size of the data volume in gigabytes"
  type        = number
  default     = 50
}

variable "enable_volume_encryption" {
  description = "Whether to enable encryption for the volumes"
  type        = bool
  default     = true
}

variable "user_data" {
  description = "The user data to provide when launching the instance"
  type        = string
  default     = null
}

variable "ssh_user" {
  description = "SSH user name to use for remote-exec connection"
  type        = string
  default     = "ec2-user"
}

variable "private_key_content" {
  description = "The private key content to use for SSH connections"
  type        = string
  default     = ""
  sensitive   = true
}

variable "tags" {
  description = "A mapping of tags to assign to the resource"
  type        = map(string)
  default     = {}
}

outputs.tf:

output "instance_id" {
  description = "ID of the EC2 instance"
  value       = aws_instance.this.id
}

output "instance_arn" {
  description = "ARN of the EC2 instance"
  value       = aws_instance.this.arn
}

output "public_ip" {
  description = "Public IP address of the EC2 instance"
  value       = aws_instance.this.public_ip
}

output "private_ip" {
  description = "Private IP address of the EC2 instance"
  value       = aws_instance.this.private_ip
}

output "instance_state" {
  description = "State of the EC2 instance"
  value       = aws_instance.this.instance_state
}

Using the Module

Once you've created a module, you can reference it in your root configuration:

module "web_server" {
  source = "./modules/ec2_instance"

  name           = "web-server"
  ami_id         = "ami-0c55b159cbfafe1f0"  # Amazon Linux 2 AMI (example)
  instance_type  = "t3.small"
  subnet_id      = "subnet-abcdef123456"
  
  security_group_ids = ["sg-abcdef123456"]
  key_name           = "my-key-pair"
  
  associate_public_ip     = true
  root_volume_size        = 30
  enable_volume_encryption = true
  
  create_data_volume = true
  data_volume_size   = 100
  
  # Simple user data script to prepare for Ansible
  user_data = <<-EOF
    #!/bin/bash
    yum update -y
    amazon-linux-extras install -y epel
    yum install -y python3 python3-pip
    pip3 install ansible
    mkdir -p /etc/ansible
    echo "Ready for Ansible configuration"
  EOF
  
  ssh_user = "ec2-user"
  # In production, use a secrets manager instead of hardcoded values
  private_key_content = file("~/.ssh/my-key-pair.pem")
  
  tags = {
    Environment = "dev"
    Project     = "example"
    ManagedBy   = "terraform"
  }
}

# Use outputs from the module
output "server_public_ip" {
  value = module.web_server.public_ip
}

output "server_private_ip" {
  value = module.web_server.private_ip
}

Modules help in reducing redundancy and ensuring consistency across different parts of your infrastructure. They also make it easier to manage and change infrastructure because modifications can be done centrally within the module.

Managing State in Terraform

Terraform keeps track of the real-world state of your infrastructure using state files. State files are essential for keeping your infrastructure synchronized with the code, as they help Terraform determine which resources need to be created, updated, or deleted.

Local vs. Remote State

By default, Terraform stores state locally in a file called terraform.tfstate. However, this is not ideal for collaboration or production environments, as different team members could end up using outdated state information, causing conflicts.

Remote State Storage

To collaborate with others or manage production infrastructure, it is best practice to use a remote backend to store state files securely. Using AWS S3 and DynamoDB is a popular choice for this purpose:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "dev/network/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

In the above example:

• S3 Bucket: Stores the actual state file.

• DynamoDB Table: Acts as a state lock to prevent simultaneous updates to the state file, reducing the risk of corruption.

Remote state storage ensures that your team works with the latest state, reducing the chances of conflicting changes.

Structuring Terraform Projects for Terragrunt

Maintaining separate configurations for multiple environments (like development, staging, and production) becomes challenging as your infrastructure becomes more complex.

It's tempting to put everything into one Terraform module so you can build it all at once. However, this isn't a good idea because it unnecessarily broadens your damage domain. Making a small change to one piece could cause cascading changes to Terraform resources you never intended to touch. It's best to break out all the modules into smaller chunks so you can apply them separately. This approach limits the risk and makes changes more predictable.

Terragrunt is a tool that helps manage Terraform configurations, providing easier handling of environment-specific variables and configurations. We'll go into more detail on Terragrunt in an upcoming article.

Example Project Structure

A good standard practice is to set up your terraform directory structure to match your infrastructure. By organizing your terraform this way, it's easier to manage the individual pieces separately, and it sets you up to effectively use Terraform with Terragrunt:

terraform
├── live
│   ├── dev
│   │   ├── vpc
│   │   ├── database
│   │   ├── app-servers
│   └── prod
│       ├── vpc
│       ├── database
│       ├── app-servers
└── modules
    ├── vpc
    ├── database
    └── app-servers

This structure ensures that all environments use the same modules but have different configuration values, making it much easier to manage infrastructure changes across environments.

---

---

Benefits of Using Terragrunt

1. Environment Isolation
   Each environment has its configuration, which prevents accidental changes across environments.

2. DRY Principle
   Terragrunt avoids code duplication by reusing the same modules with different configurations.

3. Simplified State Management
   Terragrunt can simplify remote state configurations by managing backend settings for you.

4. Dependency Management
   Terragrunt provides built-in dependency management, ensuring modules are applied in the correct order.

5. Parallel Execution
   Terragrunt can run Terraform commands in parallel across multiple modules, significantly speeding up the process.

In our next installment, we'll use Terragrunt to deploy a fresh GitLab instance, including runners. Terragrunt allows us to limit the damage domains to a subset of resources while still running the whole thing using the run-all option.

Best Practices for Modular Terraform Code

When preparing your Terraform codebase for Terragrunt and DRY principles, follow these best practices:

1. Standardize Module Interfaces

Create consistent variable and output names across your modules:

variable "name_prefix" {
  description = "Prefix for resource names"
  type        = string
}

variable "vpc_id" {
  description = "ID of the VPC where resources will be created"
  type        = string
}

variable "tags" {
  description = "Tags to apply to all resources"
  type        = map(string)
  default     = {}
}

2. Use Conditional Resources

Make your modules flexible by using conditionals to create resources only when needed:

# Create NAT Gateway only if public subnets exist
resource "aws_nat_gateway" "this" {
  count = length(var.public_subnets) > 0 ? min(length(var.public_subnets), length(var.private_subnets)) : 0
  
  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index].id
  
  tags = merge(
    {
      "Name" = format("${var.name}-nat-%s", element(var.azs, count.index))
    },
    var.tags
  )
}

3. Implement Secure Default Values

Set secure, sensible defaults for your variables:

variable "enable_encryption" {
  description = "Whether to enable encryption"
  type        = bool
  default     = true  # Secure by default
}

variable "log_retention_days" {
  description = "Number of days to retain logs"
  type        = number
  default     = 90    # Reasonable default
}

4. Version Your Modules

Use Git tags or semantic versioning for your modules to ensure stability:

# In your terragrunt.hcl file
terraform {
  # Reference a specific git tag
  source = "git::https://github.com/my-org/terraform-modules.git//vpc?ref=v1.2.3"
}

5. Create Input Validation

Add validation rules to your variables to catch issues early:

variable "environment" {
  description = "Environment name (dev, staging, prod)"
  type        = string
  
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "The environment must be one of: dev, staging, prod."
  }
}

Summary

In this guide, we've covered the essential aspects of working with Terraform effectively:

1. Setting up Terraform - Installing and configuring Terraform with the appropriate providers and authentication methods.

2. Basic Infrastructure as Code - Creating infrastructure components using Terraform's declarative syntax.

3. Terraform Workflow - Understanding the init, plan, apply, and destroy commands for managing infrastructure.

4. Modular Architecture - Building reusable modules encapsulating specific infrastructure components like VPCs, databases, and application servers.

5. State Management - Implementing remote state storage and locking for team collaboration.

6. Project Structure - Organizing your Terraform code for multi-environment deployment with Terragrunt.

7. DRY Principles: Apply the Don't Repeat Yourself principles through standardized interfaces, conditional resources, and proper variable handling.

Adopting these practices allows you to scale your infrastructure while maintaining control and consistency across environments.

Conclusion

Infrastructure as Code has revolutionized how we deploy and manage cloud resources, and Terraform stands at the forefront of this transformation. As you've seen throughout this guide, Terraform's power grows exponentially when you implement modular, reusable code with well-structured projects.

The patterns we've shared prepare you for using Terragrunt to manage complex, multi-environment deployments without duplicating code. While Terraform handles the actual resource provisioning, our approach ensures your infrastructure is ready for tools like Ansible to handle configuration management (which we'll cover in our next post).

Remember that good Terraform code is:

• Modular - Breaking components into logical, reusable pieces

• Consistent - Using standardized interfaces and naming conventions

• Maintainable - Following DRY principles and proper documentation

• Secure - Implementing best practices for state management and secrets handling

Experiment with the concepts covered here and gradually incorporate them into your infrastructure management workflow. You'll find that the initial investment in structuring your Terraform code pays significant dividends as your infrastructure needs grow and evolve.

Resources

Official Documentation

• Terraform Documentation

• Terragrunt Documentation

• AWS Provider Documentation

Learning Resources

• Terraform Module Registry

• HashiCorp Learn: Terraform

• "Terraform: Up & Running" by Yevgeniy Brikman

Helpful Tools

• tfswitch - Manage multiple Terraform versions

• tflint - Linter for Terraform code

• checkov - Security and compliance scanning

• infracost - Cost estimation for Terraform plans

Stay tuned for our next post, where we'll explore how to use Ansible to configure the infrastructure you've deployed with Terraform, creating a complete Infrastructure as Code pipeline.

---