Apache Airflow is a scalable distributed workflow scheduling system. Once deployed, Airflow cluster can be reused by multiple teams within an organization, enabling them to automate their workflows. Here I will share lessons learnt in deploying Airflow into an AWS Elastic Container Service (ECS) cluster.
Posts by Category
It all started with a friend P., a professor at Leuphana Universität Lüneburg, asking me one day out of the blue: “wanna teach with us?”. Of course I do!
Few days ago, together with boxdot, VeaaC and heremaps we finally released one little secret to Open Source community. Library, created as a nice side effect of massive work to improve performance of one of our core services. Meet flatdata: zero copy memory-mapped data storage.
It has been a while since I wrote my first blog post, a while which at the time I am writing this is about 232'389'749 seconds, evaluating to something around seven years. During this time I wrote a lot about my life experiences, traveling, posted 2146 pictures, redesigned the entire blog three times and switched from Russian to English as a main language. And that excluding real life events
23-27 October I had a chance to visit the Open Source Summit Europe 2017. As I found this conference quite amazing, here - my personal major highlights, the most interesting talks and a few notes from the event.
Linux Trace Toolkit Next Generation - LTTng - is a powerful tracing framework providing flexible kernel- and user-space tracing tools. LTTng-UST - a user space tracer module, allows to integrate C/C++ applications with its powerful infrastructure. This post shares the results of a synthetic performance benchmark for LTTng-UST, attempting to quantify its instrumentation and tracing overhead.
Linux uprobes allows to dynamically instrument user applications, injecting programmable breakpoints at arbitrary instructions. It is remarkably handy in ad-hoc profiling, debugging and tracing, especially when combined with eBPF. In this post we will use gdb to inspect binary image of a traced application.
Linux User- and Kernel- Probes remain not known widely, partly, due to specialized nature, partly - because they aren’t too easy to use. In this post I would try to explain what they are, how they work and how they can be used.