Junior/Regular/Senior Data Pipeline Development Engineer

Project Description:

We are seeking experienced Software Engineering Analysts / Data Workflow Engineers to design, implement, and operate scalable data orchestration workflows using Apache Airflow. You will work as part of an agile engineering team, contributing to the full lifecycle of workflow development—from concept and design to implementation, operation, and continuous improvement.

This role requires strong technical expertise in Python, workflow orchestration, and system integration, along with the ability to collaborate closely with cross-functional software teams.

Responsibilities:

• Analyze existing process flows and architecture diagrams to design scalable Airflow workflows

• Design end-to-end orchestration solutions, including integrations with databases, APIs, and external systems

• Implement and configure Apache Airflow DAGs using Python

• Migrate existing Jenkins pipelines and workflows to Apache Airflow

• Operate, monitor, and maintain the Airflow scheduler and workflows in production

• Troubleshoot workflow failures and performance issues

• Ensure solutions meet robustness, reliability, and performance expectations

• Actively participate in agile ceremonies, governance meetings, weekly status updates, and daily stand-ups during active delivery phases

• Collaborate closely with internal development teams and stakeholders

Mandatory Skills Description:

• Minimum 3 years of hands-on experience with Apache Airflow (ETL/ELT workflow development and orchestration)

• Strong proficiency in Python

• Experience integrating multiple data sources (SQL databases, REST APIs, cloud services)

• Experience working in agile development environments

• Familiarity with CI/CD pipelines and DevOps practices

• Strong ability to understand complex technical workflows and suggest improvements

• Excellent written and spoken English, especially for technical documentation

Nice-to-Have Skills Description:

Meritorious / Nice to Have: Experience with cloud platforms (Azure, AWS, GCP); experience operating Airflow in enterprise or large-scale environments; prior experience migrating workflows from Jenkins or similar orchestration tools.

Languages:

English: B2 Upper Intermediate