Faculty of Computer Science

Research Group Theoretical Computer Science

Oberseminar: Heterogene formale Methoden

Abstract:

In this talk we'll see how Apache Airflow is used in practice to structure and streamline the data processing pipeline which is getting developed as part of the eGo^n project. Being a follow up project to open_eGo, eGo^n re-uses parts of a data processing pipeline developed as part of the former. We'll see how Apache Airflow helps in overhauling and refactoring the old pipeline's structure. To this end, the talk consists of a short introduction to Apache Airflow and its inner workings, followed by a comparison of the old pipeline's implementation to the current, very early state of the new implementation used in eGo^n. This will be concluded by a short demonstration of how the new data processing pipeline can be used.