Vacatures voor Data Specialisten

Prefect — A modern, python-native data workflow engine

Bron: Michael Ludwig op Make It New - 16 september 2020

At one point working in a data project, you will always end up with multiple (dependent or independent) data processing steps that you need to somehow connect, schedule for periodic execution, and also monitor for errors in production. This is usually the time you either hack together some unreliable bash script orchestrating these steps or you are looking for a dedicated workflow engine. In this article, I want to give you some insights into one of the newer tools in this area called Prefect.

The space of workflow orchestration tooling is quite big and there are some well known and often used engines in there like Apache AirflowLuigi, or Oozie besides many commercial options. Nevertheless, this list is long and it could mean that many of these tools are not perfect and don’t fulfill all user requirements. When the need presented itself for such a tool in a previous project, I began looking for new promising options without defaulting to the hyped Apache Airflow.

Lees verder >>


Wil je dit nieuwsbericht verder delen?


Transistorstraat 7
1322 CJ Almere

Postbus 60184
1320 AE Almere

Tel: 036 - 7440 136

KvK 32090652
ING Bank NL91INGB065.42.67.456
BTW NL.8106.57.041.B01

Wie we zijn is onderdeel van het platform van CareerGuide, 25 vacaturebanken voor specialisten!
Onze vacaturebanken (geen bemiddeling) bieden professionals relevante vacatures binnen hun expertise.

Ook een vacature plaatsen? Neem contact met ons op:

Nienke Smit   Pieter Lammers
Nienke Smit
  Pieter Lammers