Senior Python Developer

We are looking for a Software Engineer to play a key role in migrating our workflow orchestration system from Azkaban to Apache Airflow This is a high-impact opportunity to contribute to modernizing our data processing infrastructure.

Key Responsibilities

• Design, develop and execute the migration strategy between a legacy system to a new system.  
• Develop Python scripts and automation tools to support the migration process.
• Implement best practices for testing and data validation to ensure the accuracy and reliability of flows.
• Work with dev team on automating several manual processes related to release, upgrades, deployments etc
• Collaborate with platform owners and data teams to ensure workflows continue to operate with the same performance post-migration.
• Optimize Airflow DAGs for efficiency, modularity, and maintainability.
• Have previous DevOps experience focusing on continuous deployment and monitoring.

Preferred candidate profile

• Bachelors degree in computer science, Software Engineering, or a related field.
• Programming languages
• Proficiency in python.
• Should be able to write python scripts to automate tasks such as calling APIs, perform database interaction etc.
• Kubernetes ecosystem
• Experience creating YAMLs, helm charts to create Kubernetes resources
• Have DevOps experience
• Orchestration System:
• Proficiency with Airflow or one orchestration system.
• Create DAGs to run custom workflows is a plus
• Familiarity with Jupyter notebooks is a plus