Articles in the category "aws"

Working with datasets in pandas will almost inevitably bring you to the point where your dataset doesn’t fit into memory. Especially parquet is notorious for that since it’s so well compressed and tends to explode in size when read into a dataframe. Today we’ll explore ways to limit and filter the data you read using push-down-predicates. Additionally, we’ll see how you can do that efficiently with data stored in S3 and why using pure pyarrow can be several orders of magnitude more I/O-efficient than the plain pandas version.

Yesterday, AWS announced support for Terraform Open Source in Service Catalog. For me, this sounds like a game changer! Which is why I had to test it out immediately.

A diagram should convey a clear message about the intention of the architecture. For this message, you only need a few primary resources. Most generated diagrams are overloaded. This new app generates a diagram as code from your annotations in the CDK code.

Proper version control is an essential part of a fast-paced, agile development approach and the foundation of CI/CD. Even though databases are an important aspect of nearly every application, database migrations, and schema evolutions are often not versioned and not integrated into the automation process. In this blog post, I would like to show you how you can leverage Flyway on AWS to version control your schema changes and automate your database migrations.

Keeping track of your passwords is already challenging in your personal life. It can be more difficult when you want to build and deploy secure applications in the cloud. Today we’ll show you a few ways of managing secrets in your Terraform deployment. We’ll teach you about common pitfalls like the random_password resource and more appropriate alternatives.

Have you ever heard about CloudFront Functions as an alternative to Lambda@Edge when doing redirects within CloudFront? In this post I will give you an example of the different steps for an implementation of both variants with the same result.

Using SQS as a queue to buffer tasks is probably the most common use case for the service. Things can get tricky if these tasks have a wide range of processing durations. Today, I will show you how to implement an SQS consumer that utilizes heartbeats to dynamically extend the visibility timeout to accommodate different processing durations.

Joining an AWS Training allows you to learn new things for your daily work. Attending a training commonly happens in groups of up to 13 people and has more of a frontal teaching character. An alternative event are workshops are more practical and done in a small group. And now, a third solution brings teams and people together and plays a competitive game: AWS GameDays.