Virtual Lab A
Virtual environment for elegant workflow management
Apache Airflow is an elegant solution for data engineers to create reliable and maintainanble processes. Virtual Lab A is a pre-setup optimised machine that includes Airflow, SQL Database and No SQL Database to test and deploy DAGs. You can deploy Virtual Lab A on Microsoft Azure in seconds and save on configuration cost and complexity.
Airflow is a platform that lets you build and run workflows. A workflow is represented as a DAG (a Directed Acyclic Graph), and contains individual pieces of work called Tasks, arranged with dependencies and data flows taken into account.
You find more at Airflow Architecture Overview.
The Airflow makes it easy to monitor and troubleshoot your data pipelines. You can see exactly how many tasks succeeded, failed, or are currently running at a glance.
Main Features:
- Easy to Use: Anyone with Python knowledge can deploy a workflow. They can use it to build ML models, transfer data, manage your infrastructure, and more.
- Pure Python: Use standard Python features to create your workflows, including date time formats for scheduling and loops to dynamically generate tasks.
- Robust Integrations: Airflow provides many plug-and-play operators that are ready to execute your tasks on Google Cloud Platform, AWS, Azure and other third-party services.
Previous
Next