Visualize AWS Security Groups and Rules from Terraform State

In an ever-changing AWS environment, maintaining manually created architecture diagrams can be challenging. For this reason, I decided to find an automated solution for generating diagrams based on the Terraform State File. In this blog post, I will introduce a tool that simplifies the maintenance of architecture diagrams, ensuring their accuracy and providing a clear understanding of AWS security groups and their interactions.

Issue

The challenge is to demonstrate how multiple security groups interact with each other. Manually maintaining diagrams with tools like draw.io can be time-consuming and may lead to inconsistencies and errors, which can impact overall security.

Introducing the Python Solution for Visualizing AWS Security Groups and Rules

Inspired by Gernot’s blog post about “diagrams as code” on AWS with CDK and D2, I developed a Terraform-specific solution using Python to parse the Terraform State File and generate diagrams. The full code can be found on GitHub.

How It Works

  1. Parse the Terraform State File to extract security group information as well as prefix lists and hardcoded IP ranges.
  2. Store the data in a custom format for easier diagram creation.
  3. Generate the diagram using the extracted data and a diagram generation library.

Demonstration

The following example demonstrates the effectiveness of this solution, using sample Terraform code for a cross-account setup.

  1. First, clone the GitHub repository containing the Python tool:
git clone https://github.com/tecracer/aws-security-group-diagram-from-terraform.git
  1. Next, navigate to the repository folder and install the required dependencies:
cd aws-security-group-diagram-from-terraform
pip install -r requirements.txt
  1. Now, from the directory of your terraform project run:
terraform show -json | python3 /path/to/aws-security-group-diagram-from-terraform/main.py -i --output_filename fancy_diagram --output_format png

This will directly read from the state file and will generate a diagram with the specified file name and format.

Open the generated diagram file to review the visual representation of your AWS security groups and rules.

Created Diagram

In the current version, the diagram may not look perfect, but there is much room for adjustments as the diagrams library offers many ways in which to customize the output file. Note that the icons used are not the actual icons for security groups as they are not released yet for the library, but changing the icon is easy and can be done later.

Conclusion

This Python tool offers a practical solution for visualizing AWS security groups and rules directly from the Terraform State File, saving time and effort while improving the accuracy and maintainability of architecture diagrams. Give it a try and let me know how it works for you!

Similar Posts You Might Enjoy

Streamline Security Group Maintenance in Terraform with Markdown Tables

Managing security groups and their rules in AWS infrastructure can become cumbersome and error-prone as your environment grows. In this blog post, I introduce a Terraform solution that simplifies this process by using a Markdown table to define security group rules. - by Fabian Brakowski

Using AI to generate Terraform Code from actual AWS resources

The world is changing, with new AI tools emerging every day. One such tool that has been making waves recently is ChatGPT. It is only the first of many such tools hitting the market, and it urges us to think about the future of our work. I recently used it to help with a standard task that I often perform and was amazed by how well it helped me to automate it. - by Fabian Brakowski

Streamlined Kafka Schema Evolution in AWS using MSK and the Glue Schema Registry

In today’s data-driven world, effective data management is crucial for organizations aiming to make well-informed, data-driven decisions. As the importance of data continues to grow, so does the significance of robust data management practices. This includes the processes of ingesting, storing, organizing, and maintaining the data generated and collected by an organization. Within the realm of data management, schema evolution stands out as one of the most critical aspects. Businesses evolve over time, leading to changes in data and, consequently, changes in corresponding schemas. Even though a schema may be initially defined for your data, evolving business requirements inevitably demand schema modifications. Yet, modifying data structures is no straightforward task, especially when dealing with distributed systems and teams. It’s essential that downstream consumers of the data can seamlessly adapt to new schemas. Coordinating these changes becomes a critical challenge to minimize downtime and prevent production issues. Neglecting robust data management and schema evolution strategies can result in service disruptions, breaking data pipelines, and incurring significant future costs. In the context of Apache Kafka, schema evolution is managed through a schema registry. As producers share data with consumers via Kafka, the schema is stored in this registry. The Schema Registry enhances the reliability, flexibility, and scalability of systems and applications by providing a standardized approach to manage and validate schemas used by both producers and consumers. This blog post will walk you through the steps of utilizing Amazon MSK in combination with AWS Glue Schema Registry and Terraform to build a cross-account streaming pipeline for Kafka, complete with built-in schema evolution. This approach provides a comprehensive solution to address your dynamic and evolving data requirements. - by Hendrik Hagen