Lead Data Engineer - Python/Apache Airflow (6-8 yrs)

About the Role :

We are seeking a highly skilled and experienced Lead Data Engineer to join our team in Bengaluru. In this role, you will be responsible for leading and mentoring a team of 2-3 developers, while also actively contributing to the design, development, and delivery of critical data solutions. You will work within an agile environment following Scrum methodologies, ensuring timely and high-quality deliverables.

Key Responsibilities :

Team Leadership :

- Manage and mentor a team of 2-3 data engineers.

- Plan, assign, and track team deliverables, ensuring timely completion and adherence to project timelines.

- Foster a collaborative and high-performing team environment.

Data Engineering & Development :

- Design, develop, and maintain robust and scalable data pipelines using Python and Airflow DAGs.

- Leverage Google Cloud Platform (GCP) components extensively, including BigQuery, Cloud Storage (buckets), IAM, and others.

- Proficient in SQL and possess strong data analysis and problem-solving skills.

- Experience with various database technologies such as Cassandra, Cosmos DB, and SQL Server.

- Exposure to cloud platforms like Azure Databricks is a plus.

Agile Development :

- Work closely with cross-functional teams (e.g., product, data science) in an agile environment.

- Participate in daily stand-up meetings, sprint planning, and retrospectives.

- Ensure adherence to agile principles and best practices.

Communication & Collaboration :

- Excellent written and verbal communication skills.

- Effectively communicate technical concepts to both technical and non-technical audiences.

- Collaborate effectively with team members, external teams, and leadership.

Nice to Have :

- Experience within the Walmart ecosystem.

- Domain expertise in supply chain.

- Experience working in an onshore-offshore model.

Qualifications :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 6-8 years of experience as a Data Engineer or in a similar role.

- Strong proficiency in Python, SQL, and data warehousing concepts.

- Experience with cloud platforms, preferably GCP.

- Experience with data pipelines and ETL/ELT processes.

- Experience with Agile methodologies (Scrum).

- Excellent analytical and problem-solving skills.