Building Reusable Infrastructure with Terraform Modules

Terraform modules are a powerful way to create reusable, maintainable infrastructure as code. In this comprehensive guide, we’ll explore how to create effective Terraform modules, understand best practices, and look at real-world examples.

What are Terraform Modules?

A Terraform module is a collection of Terraform resources that are grouped together in a meaningful way. Think of them as reusable building blocks for your infrastructure. Modules can represent:

Complete application stacks
Individual components (like a load balancer setup)
Cross-cutting concerns (like IAM roles and security groups)

Module Structure Best Practices

A well-organized Terraform module typically follows this structure:

module-name/
├── README.md           # Documentation
├── main.tf            # Main resource definitions
├── variables.tf       # Input variables
├── outputs.tf         # Output values
├── versions.tf        # Required providers and versions
└── examples/          # Example implementations
    └── basic/
        ├── main.tf
        └── terraform.tfvars

Creating Your First Module

Let’s create a practical example: a module that sets up an AWS S3 bucket with standardized settings and optional encryption:

# main.tf
resource "aws_s3_bucket" "this" {
  bucket = var.bucket_name
  
  tags = merge(
    var.tags,
    {
      Name = var.bucket_name
      Environment = var.environment
    }
  )
}

resource "aws_s3_bucket_versioning" "this" {
  bucket = aws_s3_bucket.this.id
  
  versioning_configuration {
    status = var.enable_versioning ? "Enabled" : "Disabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
  count = var.enable_encryption ? 1 : 0
  
  bucket = aws_s3_bucket.this.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

# variables.tf
variable "bucket_name" {
  description = "Name of the S3 bucket"
  type        = string
}

variable "environment" {
  description = "Environment name (e.g., prod, staging, dev)"
  type        = string
}

variable "enable_versioning" {
  description = "Enable versioning on the bucket"
  type        = bool
  default     = true
}

variable "enable_encryption" {
  description = "Enable server-side encryption"
  type        = bool
  default     = true
}

variable "tags" {
  description = "Additional tags for the bucket"
  type        = map(string)
  default     = {}
}

# outputs.tf
output "bucket_id" {
  description = "The name of the bucket"
  value       = aws_s3_bucket.this.id
}

output "bucket_arn" {
  description = "The ARN of the bucket"
  value       = aws_s3_bucket.this.arn
}

Using the Module

Here’s how to use the module we just created:

module "storage" {
  source = "./modules/s3-bucket"

  bucket_name       = "my-application-storage"
  environment      = "production"
  enable_versioning = true
  enable_encryption = true
  
  tags = {
    Project     = "MyApp"
    CostCenter  = "12345"
  }
}

Advanced Module Patterns

1. Conditional Resource Creation

Sometimes you want to make entire resources optional within your module:

variable "create_lifecycle_rule" {
  description = "Whether to create a lifecycle rule"
  type        = bool
  default     = false
}

resource "aws_s3_bucket_lifecycle_configuration" "example" {
  count = var.create_lifecycle_rule ? 1 : 0
  
  bucket = aws_s3_bucket.this.id

  rule {
    id     = "cleanup-old-files"
    status = "Enabled"

    expiration {
      days = 90
    }
  }
}

2. Dynamic Blocks

Dynamic blocks allow you to create multiple similar nested blocks based on a variable:

variable "cors_rules" {
  description = "List of CORS rules"
  type = list(object({
    allowed_headers = list(string)
    allowed_methods = list(string)
    allowed_origins = list(string)
    max_age_seconds = number
  }))
  default = []
}

resource "aws_s3_bucket_cors_configuration" "example" {
  count = length(var.cors_rules) > 0 ? 1 : 0
  
  bucket = aws_s3_bucket.this.id

  dynamic "cors_rule" {
    for_each = var.cors_rules
    content {
      allowed_headers = cors_rule.value.allowed_headers
      allowed_methods = cors_rule.value.allowed_methods
      allowed_origins = cors_rule.value.allowed_origins
      max_age_seconds = cors_rule.value.max_age_seconds
    }
  }
}

Module Composition

Modules can be composed together to create larger infrastructure units:

module "web_app" {
  source = "./modules/web-app"

  # This module might internally use other modules
  depends_on = [
    module.networking,
    module.database
  ]
}

module "networking" {
  source = "./modules/networking"
}

module "database" {
  source = "./modules/database"
}

Best Practices

Version Your Modules: Use semantic versioning for your modules and tag releases in your version control system.
Documentation: Always include:
- Required and optional variables
- Example usage
- Expected outputs
- Required providers and versions
Input Validation: Use variable validation to ensure inputs meet your requirements:

variable "environment" {
  type        = string
  description = "Environment name"
  
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be dev, staging, or prod."
  }
}

Keep Modules Focused: Each module should do one thing well. Avoid creating monolithic modules that try to do everything.
Use Data Sources: Leverage data sources to make modules more flexible and avoid hardcoding values:

data "aws_caller_identity" "current" {}
data "aws_region" "current" {}

locals {
  account_id = data.aws_caller_identity.current.account_id
  region     = data.aws_region.current.name
}

Testing Modules

Always include examples that double as test cases. Consider using tools like:

Terratest for integration testing
terraform-docs for documentation generation
tflint for static analysis
checkov for security and compliance testing

Conclusion

Terraform modules are a powerful way to create reusable, maintainable infrastructure code. By following these patterns and best practices, you can create modules that are:

Easy to understand and use
Flexible enough to handle various use cases
Maintainable over time
Well-tested and reliable

Remember that good modules evolve over time based on real usage patterns and feedback. Start simple and refactor as needed based on actual use cases.