Automated Cloud Detection and Response System on AWS

This project demonstrates the implementation of an automated security operations (SecOps) system on Amazon Web Services. The system is designed to detect, respond to, and notify about specific security threats in real-time, significantly reducing the Mean Time to Detect (MTTD) and Mean Time to Respond (MTTR).

📑 Table of Contents

• Architecture Diagram
• Key Features
• Tech Stack
• Project Implementation Steps
  • Phase 1: Lab Environment Setup
  • Phase 2: Detection Services Configuration
  • Phase 3: Automated Response Flow Implementation
  • Phase 4: Attack Simulation & Results
    • Test 1: Data Exfiltration
    • Test 2: Lateral Movement
• Conclusion & Key Takeaways

---

Architecture Diagram

Key Features

• Automated Threat Detection: Utilizes native AWS services like GuardDuty and CloudWatch to detect malicious activities and policy violations.
• Real-time Automated Response: Employs AWS Lambda and EventBridge to execute predefined response playbooks, such as isolating compromised resources and gathering forensic evidence.
• Data-Centric Security Monitoring: Actively monitors sensitive data stored in S3 for signs of exfiltration.
• Behavioral Analysis: Detects post-compromise activities like internal network reconnaissance (Lateral Movement).
• Serverless & Scalable: Built on a serverless architecture, ensuring high availability and cost-efficiency.

Tech Stack

• Threat Detection: Amazon GuardDuty, AWS CloudTrail, Amazon CloudWatch Alarms
• Automated Response & Processing: AWS Lambda (Python), AWS EventBridge
• Infrastructure: Amazon EC2, Amazon S3, Amazon RDS, Amazon VPC
• Notifications: Amazon SNS (Simple Notification Service)
• Identity & Access Management: AWS IAM

Project Implementation Steps

This project was built from the ground up on AWS. The following sections outline the key phases of implementation, from building the foundational infrastructure to testing the final automated response system.

Phase 1: Lab Environment Setup

The first phase involved creating a realistic, isolated cloud environment in the ap-southeast-1 (Singapore) region to serve as the attack surface.

• VPC and Networking: A Virtual Private Cloud (VPC) was created with one public and one private subnet. This provides a secure and isolated network foundation for all other resources.

• EC2 Instance: A t3.micro EC2 instance was launched into the public subnet to act as the initial point of compromise for attack simulations.

• S3 Bucket: A private S3 bucket was created to store simulated sensitive files (customer-list.txt, financial-report-q3.txt, etc.), serving as the target for the data exfiltration scenario. Public access was completely blocked.

• RDS Database: A db.t3.micro MySQL RDS instance was deployed into the private subnet with public access disabled. This serves as a high-value internal target for the lateral movement scenario.

• IAM Role: An IAM Role (EC2-S3-Access-Role) was created and attached to the EC2 instance, granting it read-only permissions to the S3 bucket. This is a crucial step to enable the data exfiltration simulation.

Phase 2: Detection Services Configuration

With the environment in place, the next step was to deploy the security "sensors" to monitor for malicious activity.

• AWS CloudTrail: A trail (CDR-S3-Monitoring-Trail) was configured to specifically log S3 Data Events (Read actions like GetObject). Logs are delivered to a dedicated S3 bucket and CloudWatch Logs for analysis.

• Amazon CloudWatch: A Metric Filter was created to parse the CloudTrail logs and count GetObject events. An Alarm (S3DataExfiltrationDetected) was then configured to trigger if more than 2 GetObject events occurred within a 5-minute window.

• Amazon GuardDuty: GuardDuty was enabled for the region. This service acts as an intelligent threat detection system, automatically analyzing VPC Flow Logs and CloudTrail events to identify threats like port scanning.

Phase 3: Automated Response Flow Implementation

This phase involved connecting the detection services to an automated response engine.

• AWS Lambda: A Python-based Lambda function (CDR-Response-Function) was created to act as the central "brain" of the system. The code contains the logic to handle different threats and execute specific containment actions.

👉 Source code: Playbooks/lambda_function.py

• Lambda Permissions: The Lambda function's execution role was granted the necessary permissions (AmazonEC2FullAccess, AmazonSNSFullAccess) to manage EC2 instances and send notifications.

• Amazon SNS: Send nofitication through email

• AWS EventBridge: Two rules were created to act as the "nervous system":

- A rule (`Catch-S3-Exfiltration-Alarm`) to capture the `ALARM` state from the specific CloudWatch Alarm.

- A rule (`Catch-Lateral-Movement-Finding`) to capture all new findings from GuardDuty.

- Both rules were configured to trigger the same `CDR-Response-Function` Lambda function.

Phase 4: Attack Simulation & Results

The final phase was to test the system by simulating the two defined attack scenarios.

Test 1: Data Exfiltration

• Attack: Executed aws s3 cp commands from the compromised EC2 instance to download 3 files from the sensitive S3 bucket.

• Result: The system worked perfectly.

• The CloudWatch Alarm successfully transitioned to the "In alarm" state.

• An email notification was immediately sent via SNS.

Test 2: Lateral Movement

• Attack: Executed an nmap -Pn port scan from the compromised EC2 instance against the internal RDS database endpoint.

• Result: The system detected and responded successfully.

• GuardDuty generated a Recon:EC2/Portscan finding.

• The EC2 instance's security group was changed to a "Quarantine-SG", successfully isolating it from the network.

• An EBS snapshot was created for forensic purposes.

• A detailed notification was sent via SNS, including the error for rate-limiting on subsequent snapshot attempts, proving the error handling worked.

Conclusion & Key Takeaways

This project successfully demonstrates the power of native AWS services in building a robust, cost-effective, and highly automated Cloud Detection and Response system. By integrating services like GuardDuty, CloudWatch, Lambda, and EventBridge, it's possible to create a security posture that not only detects threats in real-time but also contains them automatically, drastically reducing the window of opportunity for attackers.

Key achievements and learnings from this project include:

• Hands-On Experience: Gained practical, hands-on experience in configuring and integrating core AWS security and serverless services.
• Security as Code: Developed a Python-based Lambda function to programmatically execute security playbooks, a fundamental concept in modern DevSecOps.
• Understanding Attacker Techniques: Successfully simulated realistic attack scenarios, providing valuable insight into how threats like data exfiltration and lateral movement manifest in a cloud environment.
• Demonstrated Value of Automation: Proved that automation is key to achieving rapid response times (low MTTR), which is critical for effective incident response and minimizing potential damage from a security breach.

This project serves as a strong foundation and a practical demonstration of the skills required for a role in cloud security, system engineering, or network engineering, showcasing the ability to not only build but also secure cloud infrastructure.