I am looking for an open-source scheduler tool with cross-functional application dependencies. Some of the tasks I am looking to schedule are as follows:
- Trigger Matillion ETL loads
- Trigger Attunity Replication tasks that have downstream ETL loads
- Trigger Golden gate Replication Tasks
- Shell scripts, wrappers, file watchers
- Event-driven schedules
I have used Airflow in the past, and I know we need to create DAGs for each pipeline. I am not familiar with Jenkins, but I know it works with configuration without much underlying code. I want to evaluate both and appreciate any advise
Hi Teja, Jenkins is more a CI/CD tool for triggering build/test and other CD tasks, from what you're describing you may be able to get along with #Jenkins and add lot's of plugins and create pipelines. But, eventually you're going to need to know #Groovy language to orchestrate all those tasks which is going to be similar to what you do with #Airflow, So, IMHO , Airflow is more for production scheduled tasks and Jenkins is more for CI/CD non-production tasks.
