In this blog post, I will show you how to install apache airflow on ubuntu,
According to Apache’s official web site, Apache airflow is a platform for programmatically author schedule and monitor workflows. It is one of the best workflow management system.
Airflow was originally developed by Airbnb (Airbnb Engineering) to manage their data based operations. Now, Airbnb open sourced it under apache license and hence got more popular.
Apache airflow makes your work flow little bit simple and organized by allowing you to divide it into small independent (not always) task units, So that it’s easy to organize and easy to schedule ones.
Your entire workflow can be converted into a DAG (Directed acyclic graph) with Airflow. Defining workflow makes your code more maintainable
You need to have following thing installed already in order to install apache airflow, we will install this using
pip (python interpreter)
- python & pip
Let’s start by installing Prerequisites first, run below commands.
sudo apt-get install python-dev libsasl2-dev gcc sudo apt-get install libffi-dev sudo apt-get install libkrb5-dev sudo apt install virtualenv
We are using virtualenv for this installation because we have issues with direct install on pip so let’s create a a
virtualenv. So, let’s install virtualenv and configure it for the airflow
sudo virtualenv airflow_virtualenv cd airflow_virtualenv/bin source activate export SLUGIFY_USES_TEXT_UNIDECODE=yes export AIRFLOW_HOME=~/airflow pip install apache-airflow pip install apache-airflow[all] airflow initdb
Finally, this should install airflow and you can start using it, also remember that you can export
AIRFLOW_HOME in you
Let’s run the server and see if we can load the web page, execute the command below,
airflow webserver -p 8080
This will take about a few minutes for service to be up and running, Let’s test it on the browser. Finally, there you see the admin page!
Please check more tags on goacademy