Data Engineer Lead - PySpark/DataBricks (4-6 yrs)

Key Responsibilities :

Technical Leadership :

- Provide technical vision and leadership for the data engineering team.

- Design and implement high-impact, scalable data solutions using cloud-based big data technologies.

- Stay up-to-date on the latest advancements in big data and cloud technologies.

- Mentor and coach junior data engineers, helping them develop their skills and expertise.

Hands-on Execution :

- Develop and manage large-scale data pipelines using Databricks and PySpark.

- Utilize your Python proficiency for data processing and application development.

- Work closely with data scientists and analysts to understand their data needs.

Team Management :

- Build, lead, and motivate a high-performing data engineering team.

- Foster a collaborative and innovative team culture.

- Provide clear direction, delegate tasks effectively, and hold team members accountable.

Required Skills :

Technical Expertise :

- 4+ years of experience in data engineering or a related field.

- Proficiency in Python for data processing and application development.

- Strong experience with Databricks and PySpark for building and managing large-scale data pipelines.

- Hands-on expertise with Azure services : Azure Data Factory, Azure Synapse, Azure Databricks, Azure Data Lake, and other MS Stack tools.

Leadership and Management :

- Proven track record in people management, team building, and leadership.

- Excellent problem-solving, decision-making, and conflict-resolution skills.

Data Management :

- Strong understanding of big data ecosystems, distributed computing, and data governance principles.

Bonus Points :

- Experience with cloud platforms like AWS or GCP (alongside Azure).

- Experience with data warehousing and data modeling techniques.

- Excellent communication and presentation skills