Lead Data Engineer

Data Engineer

Job Location: Bangalore, Pune, Chennai, Gurgaon

Shift Time: 3 pm - 12 am

Role & responsibilities

• Work closely with analysts/data scientists to understand impact to the downstream data models.
• Write efficient and well-organized software to ship products in an iterative, continual release environment.
• Contribute to and promote good software engineering practices across the team
• Communicate clearly and effectively to technical and non-technical audiences.

Preferred candidate profile

• University or advanced degree in engineering, computer science, mathematics, or a related field
• Strong hands-on experience in Databricks using PySpark and Spark SQL (Unity Catalog, workflows, Optimization techniques)
• Experience with at least one cloud provider solution (GCP preferred)
• Strong experience working with relational SQL databases.
• Strong experience with object-oriented/object function scripting language : Python.
• Working knowledge in any transformation tools, DBT preferred.
• Ability to work with Linux platform.
• Strong knowledge of data pipeline and workflow management tools (Airflow)
• Working knowledge of Git hub /Git Toolkit
• Expertise in standard software engineering methodology, e.g. unit testing, code reviews, design documentation
• Experience creating Data pipelines that prepare data for ingestion & consumption appropriately. Experience in maintaining and optimizing databases/filesystems for production usage in reporting, analytics.
• Working in a collaborative environment and interacting effectively with technical and non-technical team members equally well.
• Good verbal and written communication skills.