Skip to content

Chaitanya5068/aws-multitier-terraform-automation

BranchesTags

Repository files navigation

🚀 AWS Multi-Tier Terraform Automation Platform

Production-Grade AWS DevOps Infrastructure Automation

Scalable • Automated • Observable • High Availability • Production-Ready


A production-grade multi-tier AWS infrastructure automation platform built using Terraform, featuring automated deployments, Auto Scaling, monitoring, observability, CloudWatch dashboards, SNS alerts, and event-driven frontend deployment workflows.

This project supports both DEV and PROD environments with isolated Terraform state management, reusable modular architecture, and production-style infrastructure automation.

📋 Project Overview

This repository contains a production-grade Infrastructure as Code (IaC) solution built with Terraform that automates deployment and management of a scalable AWS multi-tier architecture.

The infrastructure is fully modular, environment-aware (DEV/PROD), and designed to handle real-world production workloads with:

  • Automated deployment pipelines
  • Auto Scaling & self-healing infrastructure
  • Centralized monitoring & observability
  • CloudWatch dashboards & alarms
  • SNS notifications
  • Secure private subnet deployment
  • Event-driven frontend deployment using S3 + Lambda + SSM

Supported Environments

environments/
├── dev/
│   ├── backend.tfvars
│   └── terraform.tfvars
│
└── prod/
    ├── backend.tfvars
    └── terraform.tfvars

Key Highlights

  • ✅ Multi-Environment Architecture (DEV & PROD)
  • ✅ Fully Automated Deployment Workflow
  • ✅ Production-Grade Monitoring Stack
  • ✅ Infrastructure as Code using Terraform
  • ✅ Modular AWS Infrastructure
  • ✅ Auto Scaling & High Availability
  • ✅ Real-Time CloudWatch Observability
  • ✅ Secure & Scalable Design

🏗️ Architecture Overview

┌──────────────────────────────────────────────────────────────┐
│                        USER TRAFFIC                          │
└──────────────────────────────┬───────────────────────────────┘
                               │
                               ▼
┌──────────────────────────────────────────────────────────────┐
│         AWS CloudFront / Route 53 (Optional)                 │
└──────────────────────────────┬───────────────────────────────┘
                               │
                               ▼
┌──────────────────────────────────────────────────────────────┐
│               Application Load Balancer (ALB)                │
│  • Multi-AZ Traffic Distribution                             │
│  • Health Checks Enabled                                     │
│  • HTTPS / SSL Termination                                   │
└──────────────────────────────┬───────────────────────────────┘
                               │
                               ▼
┌──────────────────────────────────────────────────────────────┐
│                Private Subnets (Multiple AZs)                │
│                                                              │
│   ┌──────────────────────────────────────────────────────┐   │
│   │              EC2 Auto Scaling Group                  │   │
│   │                                                      │   │
│   │  • Min Capacity: 2                                   │   │
│   │  • Max Capacity: 4                                   │   │
│   │  • Desired Capacity: 2                               │   │
│   │  • ELB Health Checks                                 │   │
│   │  • Launch Template + User Data                       │   │
│   │  • Nginx Application Server                          │   │
│   │  • Self-Healing Infrastructure                       │   │
│   └──────────────────────────────────────────────────────┘   │
│                                                              │
└──────────────────────────────┬───────────────────────────────┘
                               ▲
                               │
┌──────────────────────────────────────────────────────────────┐
│             AWS Systems Manager (SSM)                        │
│                 Run Command + IAM Roles                      │
└──────────────────────────────┬───────────────────────────────┘
                               ▲
                               │
┌──────────────────────────────────────────────────────────────┐
│                      AWS Lambda                              │
│             Triggered by S3 Upload Events                    │
└──────────────────────────────┬───────────────────────────────┘
                               ▲
                               │
┌──────────────────────────────────────────────────────────────┐
│                    Amazon S3 Bucket                          │
│                 Frontend Application Assets                  │
│            (HTML, CSS, JS, Images, Static Files)             │
└──────────────────────────────────────────────────────────────┘


┌──────────────────────────────────────────────────────────────┐
│             📊 MONITORING & OBSERVABILITY LAYER             │
├──────────────────────────────────────────────────────────────┤
│                                                              │
│   ┌────────────────┐  ┌────────────────┐  ┌──────────────┐   │
│   │ CloudWatch     │  │ CloudWatch     │  │ SNS Alerts   │   │
│   │ Logs           │  │ Dashboards     │  │ Email / SMS  │   │
│   │                │  │                │  │ Notifications│   │
│   │ • Nginx Logs   │  │ • ALB Metrics  │  │              │   │
│   │ • Lambda Logs  │  │ • EC2 Metrics  │  │ • High CPU   │   │
│   │ • App Logs     │  │ • Lambda Stats │  │ • Unhealthy  │   │
│   │ • Deploy Logs  │  │ • Traffic Data │  │   Hosts      │   │
│   └────────────────┘  └────────────────┘  └──────────────┘   │
│                                                              │
└──────────────────────────────────────────────────────────────┘
image

Deployment Flow

Frontend Asset Upload
         ↓
    S3 Object
         ↓
  S3 Event Trigger
         ↓
  AWS Lambda Function
         ↓
  SSM Run Command
         ↓
 EC2 Instances (ASG)
         ↓
  File Sync & Nginx Restart
         ↓
  Live Application Updates

☁️ AWS Services Used

Service Purpose Configuration
VPC Network isolation and segmentation Multi-AZ, Public/Private subnets
EC2 Application servers Auto Scaling Group with Launch Template
ALB Load balancing and traffic distribution Multi-AZ, Health checks enabled
Auto Scaling Dynamic capacity management Min: 2, Max: 4, Scale-based policies
S3 Frontend asset storage Versioning, Event notifications
Lambda Automated deployment trigger Event-driven, IAM execution role
Systems Manager Infrastructure automation Run Command for EC2 deployment
CloudWatch Monitoring and observability Custom metrics, dashboards, alarms
SNS Alert notifications Email/SMS subscriptions
IAM Identity and access management Service roles, instance profiles
Security Groups Network access control Inbound/Outbound rules per layer

🎯 Key Features

Infrastructure as Code

  • 📦 Modular Terraform Design: Separate modules for each AWS service
  • 🔄 Reusable Components: DRY principle - shared modules across environments
  • 📝 Version Controlled: Complete infrastructure history in Git
  • 🔍 Validated: Terraform validate and plan before apply

High Availability & Scaling

  • 🌍 Multi-AZ Deployment: Services distributed across availability zones
  • 📈 Auto Scaling: Automatic scaling based on demand
  • 💪 Health Checks: Automatic instance replacement on failure
  • ⚡ Load Balancing: Efficient traffic distribution across instances

Automated Deployment

  • 🤖 CI/CD Ready: Lambda-triggered deployment pipeline
  • 📦 S3-Based Deployment: Upload and deploy frontend assets
  • 🔄 Zero-Downtime: Rolling updates via Auto Scaling
  • 📊 Deployment Tracking: CloudWatch logs for audit trail

Monitoring & Observability

  • 📊 CloudWatch Dashboards: Real-time infrastructure metrics
  • 📋 Centralized Logging: Nginx and application logs in CloudWatch
  • 🚨 Intelligent Alerts: SNS notifications for critical metrics
  • 📈 Custom Metrics: Application-specific performance tracking

Security

  • 🔐 Private Subnet Deployment: EC2 instances not directly accessible
  • 🛡️ IAM Least Privilege: Granular permissions for each service
  • 🔒 Security Groups: Network segmentation and access control
  • 📝 Audit Trails: CloudWatch Logs for compliance

🏗️ Infrastructure as Code (IaC) Architecture

Module Structure

modules/
├── vpc/                    # Virtual Private Cloud
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── security/               # Security Groups & NACLs
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── alb/                    # Application Load Balancer
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── ec2/                    # EC2 & Auto Scaling
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── iam/                    # IAM Roles & Policies
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── lambda/                 # Lambda Function
│   ├── main.tf
│   ├── variables.tf
│   ├── outputs.tf
│   └── function/
│       └── index.py        # Python deployment handler
├── s3/                     # S3 Bucket
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── cloudwatch/             # Monitoring & Logs
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── sns/                    # SNS Topic
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
└── validate_destroy/       # Destruction validations
    └── main.tf

Environment Configuration

environments/
├── dev/
│   ├── backend.tfvars      # DEV state backend
│   └── terraform.tfvars    # DEV variables
└── prod/
    ├── backend.tfvars      # PROD state backend
    └── terraform.tfvars    # PROD variables

Root Configuration Files

  • main.tf: Module orchestration and composition
  • providers.tf: AWS provider configuration
  • backend.tf: Terraform state management
  • variables.tf: Input variables definition
  • outputs.tf: Infrastructure outputs
  • versions.tf: Terraform version constraints

🚀 Automated Deployment Pipeline

How It Works

  1. Frontend Upload: Developer uploads frontend assets to S3 bucket
  2. S3 Event Trigger: S3 sends event notification on object upload
  3. Lambda Invocation: Lambda function receives S3 event payload
  4. SSM Command: Lambda triggers Systems Manager Run Command
  5. EC2 Execution: Run Command executes on all instances in ASG
  6. File Sync: Script syncs files from S3 to EC2 instances
  7. Service Restart: Nginx restarts to serve updated content
  8. Health Check: ALB verifies instance health
  9. Live Update: Users receive updated content

Deployment Script Flow

# On EC2 Instance (via SSM Run Command)
├── Download files from S3
├── Verify checksum/integrity
├── Backup previous version
├── Deploy new files
├── Validate permissions
├── Run health checks
└── Restart Nginx service

🔄 High Availability & Auto Scaling

Auto Scaling Configuration

  • Min Capacity: 2 instances
  • Max Capacity: 4 instances
  • Desired Capacity: 2 instances
  • Health Check Type: ELB
  • Termination Policy: Oldest instance first
  • Cooldown Period: 300 seconds

Scaling Policies

CPU Utilization > 70%  →  Scale Up (add instance)
CPU Utilization < 30%  →  Scale Down (remove instance)

High Availability Features

  • ✅ Multi-AZ Distribution: Instances spread across 2+ availability zones
  • ✅ Load Balancing: ALB distributes traffic evenly
  • ✅ Health Monitoring: Automatic instance replacement
  • ✅ Connection Draining: Graceful shutdown during updates
  • ✅ DNS Failover: Route 53 integration ready

📊 Monitoring & Observability

CloudWatch Dashboards

Automated dashboards display:

  • 📈 EC2 Metrics

    • CPU Utilization
    • Network In/Out
    • Disk Read/Write
    • Status Checks

  • 🔄 Auto Scaling Metrics

    • Group Size
    • Desired Capacity
    • Launch/Termination Events
    • Scaling Activity

  • 🌐 ALB Metrics

    • Target Count
    • Request Count
    • Target Response Time
    • HTTP Status Codes

  • 💾 Application Metrics

    • Memory Utilization
    • Disk Usage
    • Application Response Time

CloudWatch Alarms

Configured alarms trigger SNS notifications for:

  • ⚠️ High CPU utilization (> 80%)
  • ⚠️ High memory usage (> 85%)
  • ⚠️ ALB unhealthy targets
  • ⚠️ Failed deployments
  • ⚠️ Auto Scaling failures

SNS Notifications

Alerts sent via:

  • 📧 Email notifications
  • 📱 SMS alerts (optional)
  • 🔔 Custom webhooks

📋 Logging System

Centralized Logging

All logs aggregated in CloudWatch Logs:

/aws/ec2/application         # Application logs
/aws/ec2/nginx               # Nginx access & error logs
/aws/lambda/deployment       # Lambda function logs
/aws/autoscaling/asg         # Auto Scaling events

Log Analysis

  • 🔍 CloudWatch Insights: Query logs with SQL-like syntax
  • 📊 Log Filtering: Create custom filters and metrics
  • 📈 Log Trending: Identify patterns over time
  • 🚨 Log Alarms: Alert on error patterns

Nginx Logging

Access Logs:  /var/log/nginx/access.log
Error Logs:   /var/log/nginx/error.log
Format:       Combined with response time
Rotation:     Daily, compressed
Retention:    7 days local, 30 days CloudWatch

📂 Project Structure

aws-multitier-terraform-automation/
│
├── README.md                          # Project documentation
├── LICENSE                            # MIT License
├── DESTROY_FIXES.md                   # Destruction troubleshooting
│
├── root configuration files
├── main.tf                            # Module orchestration
├── providers.tf                       # AWS provider setup
├── backend.tf                         # State management
├── variables.tf                       # Input variables
├── outputs.tf                         # Infrastructure outputs
├── versions.tf                        # Version constraints
│
├── modules/                           # Reusable Terraform modules
│   ├── vpc/                          # VPC, Subnets, Route Tables, NAT Gateway
│   ├── security/                     # Security Groups, NACLs
│   ├── alb/                          # Application Load Balancer, Listeners
│   ├── ec2/                          # Launch Template, Auto Scaling Group
│   ├── iam/                          # IAM Roles, Policies, Instance Profile
│   ├── lambda/                       # Lambda Function, Permissions, S3 Trigger
│   │   └── function/
│   │       └── index.py              # Python deployment handler
│   ├── s3/                           # S3 Bucket, Versioning, Events
│   ├── cloudwatch/                   # CloudWatch Logs, Dashboards, Alarms
│   ├── sns/                          # SNS Topic, Subscriptions
│   └── validate_destroy/             # Destruction validations
│
├── environments/                      # Environment-specific configurations
│   ├── dev/
│   │   ├── backend.tfvars            # DEV state backend
│   │   └── terraform.tfvars          # DEV variables
│   └── prod/
│       ├── backend.tfvars            # PROD state backend
│       └── terraform.tfvars          # PROD variables
│
└── scripts/                           # Helper scripts
    └── userdata.sh                   # EC2 user data for Nginx setup

🛠️ Terraform Commands

Prerequisites

# Install Terraform
# https://www.terraform.io/downloads.html

# Verify installation
terraform version

DEV Environment Workflow

# Navigate to dev environment
cd environments/dev

# Initialize Terraform (downloads providers & modules)
terraform init

# Validate configuration syntax
terraform validate

# Preview changes
terraform plan -out=tfplan

# Apply changes (requires approval)
terraform apply tfplan

# Destroy infrastructure
terraform destroy -var-file=terraform.tfvars

PROD Environment Workflow

# Navigate to prod environment
cd environments/prod

# Initialize Terraform
terraform init

# Validate configuration
terraform validate

# Plan with production values
terraform plan -out=tfplan

# Apply production infrastructure
terraform apply tfplan

# View outputs
terraform output

# Destroy (use with caution!)
terraform destroy -var-file=terraform.tfvars

Common Terraform Commands

# Format code
terraform fmt -recursive

# Validate all modules
terraform validate

# View current state
terraform state list
terraform state show aws_instance.example

# Import existing resources
terraform import aws_instance.example i-1234567890abcdef0

# Taint resources (force recreation)
terraform taint aws_instance.example

# Get resource details
terraform show

# Refresh state
terraform refresh

# Clean up .terraform directory
rm -rf .terraform

State Management

# Backup state
cp terraform.tfstate terraform.tfstate.backup

# Remote state operations
terraform state pull > current.tfstate
terraform state push current.tfstate

# Remove resource from state
terraform state rm aws_instance.example

🚀 Frontend Deployment Steps

Step 1: Prepare Frontend Assets

# Build your frontend application
npm run build
# or
yarn build
# or for static HTML
# Ensure all assets are in a distribution folder

Step 2: Upload to S3

# Using AWS CLI
aws s3 sync ./dist s3://your-bucket-name/frontend/ \
  --delete \
  --cache-control "max-age=31536000,public"

# Or upload individual files
aws s3 cp ./dist/index.html s3://your-bucket-name/frontend/ \
  --cache-control "max-age=0,no-cache" \
  --content-type "text/html"

Step 3: Automatic Deployment Triggering

  1. S3 detects file upload
  2. Lambda function is invoked automatically
  3. Lambda executes SSM Run Command on EC2 instances
  4. Deployment script runs on all ASG instances
  5. Files are synced from S3
  6. Nginx restarts automatically

Step 4: Verify Deployment

# Check ALB endpoint
curl http://your-alb-dns-name

# Check instance logs
aws logs tail /aws/ec2/nginx --follow

# View Lambda logs
aws logs tail /aws/lambda/deployment --follow

# Check Auto Scaling Group status
aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names your-asg-name

Rollback Procedure

# If deployment fails, S3 versioning allows quick rollback
aws s3api get-object-version-id \
  --bucket your-bucket-name \
  --key frontend/index.html

# Restore previous version
aws s3api restore-object \
  --bucket your-bucket-name \
  --key frontend/index.html

📊 Monitoring & Logs

Accessing CloudWatch Dashboards

# Open AWS Console and navigate to:
# CloudWatch > Dashboards > your-project-dashboard

Dashboard Widgets:

  • Infrastructure health overview
  • EC2 performance metrics
  • ALB traffic distribution
  • Auto Scaling activity
  • Lambda execution metrics
  • Application error rates

Viewing Logs

# Using AWS CLI
aws logs tail /aws/ec2/nginx --follow
aws logs tail /aws/ec2/application --follow
aws logs tail /aws/lambda/deployment --follow

# Using AWS Console:
# CloudWatch > Log Groups > select log group

Creating Custom Alarms

# Example: Alert on high CPU
aws cloudwatch put-metric-alarm \
  --alarm-name high-cpu-alarm \
  --alarm-description "Alert when CPU > 80%" \
  --metric-name CPUUtilization \
  --namespace AWS/EC2 \
  --statistic Average \
  --period 300 \
  --threshold 80 \
  --comparison-operator GreaterThanThreshold

Performance Optimization

# Monitor key metrics:
# 1. Response Time: p50, p95, p99 latency
# 2. Throughput: Requests per second
# 3. Error Rate: 4xx and 5xx responses
# 4. Resource Utilization: CPU, Memory, Disk
# 5. Scaling Events: ASG launch/termination

🔐 Security Features

Network Security

  • 🛡️ VPC Isolation: Resources in private subnets
  • 🔒 Security Groups: Ingress/Egress rules enforced
  • 🌐 ALB: Single entry point with health checks
  • 🚫 NACLs: Network access control lists per subnet

Identity & Access Control

  • 👤 IAM Roles: Service-specific permissions
  • 🔑 Principle of Least Privilege: Minimal permissions granted
  • 📋 Instance Profile: EC2 to AWS service authentication
  • 🔐 Lambda Execution Role: Specific permissions for deployment

Data Protection

  • 🔒 Encryption at Rest: S3 and EBS encryption enabled
  • 🔐 Encryption in Transit: TLS/HTTPS support
  • 📝 Audit Logging: CloudTrail integration ready
  • 🔄 Backup & Recovery: Automated snapshots

Monitoring & Compliance

  • 📊 CloudWatch Logs: Complete audit trail
  • 🚨 Security Alarms: Unauthorized access detection
  • 📋 Compliance Reports: IAM access analyzer
  • 🔍 Vulnerability Scanning: Security group analysis

Best Practices Implemented

  • ✅ No public IP assignments for application servers
  • ✅ Bastion host ready (via SSM Session Manager)
  • ✅ Immutable infrastructure (launch templates)
  • ✅ Secure defaults for all security groups
  • ✅ Regular security group audits
  • ✅ Cross-account access prevented

🎓 Key Learning Outcomes

By deploying this infrastructure, you'll understand:

Terraform Skills

  • ✅ Modular infrastructure design patterns
  • ✅ State management and remote backends
  • ✅ Variable interpolation and dynamic blocks
  • ✅ Output values and data sources
  • ✅ Terraform best practices

AWS Skills

  • ✅ VPC design with public/private subnets
  • ✅ EC2 Auto Scaling and Launch Templates
  • ✅ Application Load Balancer configuration
  • ✅ Lambda event-driven architecture
  • ✅ IAM role design and policies
  • ✅ CloudWatch monitoring and alarms
  • ✅ S3 event notifications
  • ✅ Systems Manager Run Command

DevOps Skills

  • ✅ Infrastructure as Code principles
  • ✅ CI/CD pipeline design
  • ✅ Multi-environment management
  • ✅ Observability and monitoring
  • ✅ Automated deployment systems
  • ✅ High availability architecture
  • ✅ Scalability patterns

Production Skills

  • ✅ Blue-green deployment strategies
  • ✅ Graceful shutdowns and drain connections
  • ✅ Health check implementation
  • ✅ Centralized logging and log analysis
  • ✅ Alert fatigue management
  • ✅ Incident response automation
  • ✅ Cost optimization

🛠️ Technologies Used

Category Technology Purpose
IaC Terraform Infrastructure provisioning
Cloud AWS Cloud infrastructure
Compute EC2 Application servers
Network VPC, ALB Networking layer
Automation Lambda, SSM Deployment automation
Storage S3 Frontend asset storage
Monitoring CloudWatch Observability
Alerting SNS Notifications
Security IAM, Security Groups Access control
Web Server Nginx Application server
OS Linux (Amazon Linux 2) EC2 operating system
Scripting Bash User data scripts
VCS Git/GitHub Version control
DevOps Terraform, AWS, Docker-ready CI/CD foundation

🚀 Future Improvements

Phase 2: Enhanced Features

  • RDS database integration (MySQL/PostgreSQL)
  • Elasticache (Redis/Memcached) for caching
  • CloudFront CDN for edge content delivery
  • AWS Secrets Manager for sensitive data
  • VPN gateway for secure access
  • Database replication across AZs

Phase 3: Advanced DevOps

  • CI/CD pipeline with GitHub Actions / GitLab CI
  • Automated testing (unit, integration, smoke tests)
  • Container support (ECR + ECS)
  • Infrastructure scanning and compliance checks
  • Cost optimization and budget alerts
  • Automated infrastructure updates

Phase 4: Enterprise Features

  • Multi-region deployment
  • Disaster recovery (DR) strategy
  • Backup and restore automation
  • Terraform Cloud for state management
  • OIDC authentication integration
  • Service mesh (Istio/App Mesh)

Phase 5: Observability Enhancements

  • Prometheus + Grafana stack
  • Distributed tracing (X-Ray)
  • Application performance monitoring (APM)
  • Error tracking (Sentry)
  • Log aggregation (ELK stack)
  • Real-time dashboards

👤 Author

Chaitanya Bhosale

Connect With Me

About This Project

This production-grade Terraform automation platform demonstrates industry best practices for AWS infrastructure management. It's designed to be:

  • Educational: Learn enterprise DevOps patterns
  • Scalable: Grow from DEV to PROD with ease
  • Maintainable: Clean, modular code structure
  • Reliable: High availability built-in
  • Secure: Security best practices implemented

Getting Help

  • 📚 Documentation: Read DESTROY_FIXES.md for troubleshooting
  • 🐛 Issues: Report bugs and request features
  • 💬 Discussions: Share knowledge and experiences
  • 📖 References: Check AWS and Terraform documentation

License

This project is licensed under the MIT License - see LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to:

  1. Fork the repository
  2. Create a feature branch
  3. Commit your changes
  4. Push to the branch
  5. Create a Pull Request

Acknowledgments

  • AWS Documentation and best practices
  • Terraform community modules
  • DevOps industry standards
  • Cloud security frameworks

📞 Support & Resources

Official Documentation

Learning Resources

Helpful Tools

Built with ❤️ for DevOps Engineers

⭐ If this project helped you, please consider giving it a star! ⭐

About

Production-grade AWS multi-tier infrastructure automation platform using Terraform with Auto Scaling, ALB, Lambda, S3-triggered deployments, CloudWatch monitoring, SNS alerts, and DEV/PROD environments.

Topics

nginx aws devops lambda monitoring ec2 terraform s3 sns iac vpc cloud-computing infrastructure-as-code amazon-web-services autoscaling observability ssm application-load-balancer terraform-module cloud-watch

Resources

Readme

License

MIT license
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages