Apache Airflow is a scalable distributed workflow scheduling system. Once deployed, Airflow cluster can be reused by multiple teams within an organization, enabling them to automate their workflows. Here I will share lessons learnt in deploying Airflow into an AWS Elastic Container Service (ECS) cluster.
It all started with a friend P., a professor at Leuphana Universität Lüneburg, asking me one day out of the blue: “wanna teach with us?”. Of course I do!
Linux Trace Toolkit Next Generation - LTTng - is a powerful tracing framework providing flexible kernel- and user-space tracing tools. LTTng-UST - a user space tracer module, allows to integrate C/C++ applications with its powerful infrastructure. This post shares the results of a synthetic performance benchmark for LTTng-UST, attempting to quantify its instrumentation and tracing overhead.
Few days ago, together with boxdot, VeaaC and heremaps we finally released one little secret to Open Source community. Library, created as a nice side effect of massive work to improve performance of one of our core services. Meet flatdata: zero copy memory-mapped data storage.
Linux uprobes allows to dynamically instrument user applications, injecting programmable breakpoints at arbitrary instructions. It is remarkably handy in ad-hoc profiling, debugging and tracing, especially when combined with eBPF. In this post we will use gdb to inspect binary image of a traced application.