Automating Athena Queries with Python

Automating Athena Queries with Python Introduction Over the last few weeks I’ve been using Amazon Athena quite heavily. For those of you who haven’t encountered it, Athena basically lets you query data stored in various formats on S3 using SQL (under the hood it’s a managed Presto/Hive Cluster). Pricing for Athena is pretty nice as well, you pay only for the amount of data you process and that’s relatively cheap at $5 per TB when you consider the effort to set up EMR Clusters for one-time or very infrequent queries and transformations.

Chef Interactive

As you probably are aware, Chef is a tool which is meant for automatic provisioning and configuring of systems. So if you have a particular problem falling outside of the regular use cases, both posts on the internet and support enquiries of any kind will probably result in one of two answers: “that is not possible” or “you are doing it wrong”. But - what if you really need this for a rather exotic task or even as an transitory solution?

tRick: simple network 2 - Geschwindigkeit

Vergleich Infrastructure as Code (IaC) Frameworks - tRick Alle Posts Abstraktion und Lines of Code Geschwindigkeit Diversity (polyglott), Tooling, Fazit Benchmark Ausführungsgeschindigkeit Ausführungsgeschwindigkeit Direkt aus dem tRick Repository wird mehrfach (n=10) der Zyklus Build -> Check -> Deploy -> Remove ausgeführt. Damit sollen Cache Effekte statistisch gemittelt werden. Dazu nehme ich das Tool hyperfine zur Hilfe. Es führt Kommandos automatisch mehrfach aus und mittelt die Ergebnisse. Meine Annahme ist es, dass Terraform vorne liegt, da das Programm selber statisch kompiliert in go geschrieben ist. Außerdem geht die Ausführung direkt auf die API.

Managing multiple stages with Terraform

Managing multiple environments in Terraform Introduction I recently started learning Terraform. For those who haven’t encountered it: Terraform is in essence a framework to describe Infrastructure as code by Hashicorp. When I began doing that, I was struggling with the staging-concept of Terraform. I did my research and came upon numerous 1 articles and blogs that described ways to manage (multiple) environments or stages in Terraform2. Since I wasn’t really happy with the other solutions and there didn’t seem to be a canonical way to handle multiple environments, I decided to try and figure out my own solution.

Building Lambda with terraform

Note: An updated version of this post is available here Building Lambda Functions with Terraform Introduction Many of us use Terraform to manage our infrastructure as code. As AWS users, Lambda functions tend to be an important part of our infrastructure and its automation. Deploying - and especially building - Lambda functions with Terraform unfortunately isn’t as straightforward as I’d like. (To be fair: it’s very much debatable whether you should use Terraform for this purpose, but I’d like to do that - and if I didn’t, you wouldn’t get to read this article, so let’s continue)

tRick: simple network 1 - Abstraktion und LoC

Vergleich Infrastructure as Code (IaC) Frameworks - tRick Ein Toolvergleich für Infrastructure as Code. Um effektiv AWS oder generell Cloud Ressourcen zu erzeugen, verwendet man zur Erhöhung des Automatisierungsgrades “Infrastracture as Code”, d.h. die Server, Datenbanken usw. werden in einer Sprache kodiert. Dieser Vorgang wird sinvollerweise über ein Framework, welches Tools dafür zur Verfügung stellt unterstützt. Aber für welches Tool/Framework entscheidet man sich? Hier werden wir mit Dimensionen für den tRick Benchmark Entscheidungshilfen geben.