Senior Data Engineer - Apache Airflow (4-6 yrs)

Responsibilities :

- Collaborate with a cross-functional team of engineers to design and build scalable data processing pipelines using Airflow.

- Develop tools and services to support data discovery, lineage, resiliency, and data privacy across the data platform.

- Implement CI/CD pipelines for infrastructure components using tools like Terraform to ensure high availability and agility.

- Champion best practices in software engineering and architecture within the team, fostering a culture of quality, innovation, and experimentation.

- Advocate for the data platform and best practices, identify new use cases and features, and drive adoption among stakeholders.

Technical Skills (Must-Have) :

- Proven experience with Python programming language.

- In-depth knowledge of Airflow, including its architecture, operation, and best practices.

- Experience deploying and managing cloud-based applications (AWS preferred).

- Familiarity with infrastructure-as-code tools like Terraform.

- Solid understanding of data engineering principles and tools (e.g., Spark, Snowflake, ETL).

- Ability to troubleshoot complex technical issues and understand the big picture of the data ecosystem.

- Passion for writing clean, maintainable, and well-documented code.

- Excellent written and verbal communication skills.

Technical Skills (Good to Have) :

- Experience with AWS data tools (e.g. , Glue, EMR).

- Knowledge of the broader data ecosystem (e.g. , Data Management Systems, Stream Processing)