Senior Data Engineer

Job Title: Senior Data Engineer

Location: Bangalore, Mohali, Gurugram

Job Type: Full-Time

Position Overview:

We are looking for a Senior Data Engineer with 8-10 years of experience in data engineering and 2-3 years of hands-on experience with Databricks. The ideal candidate will be a strategic thinker with a deep understanding of data architectures and a passion for leveraging Databricks to optimize data workflows and analytics.

Key Responsibilities:

Data Architecture & Design: Design, develop, and optimize scalable and efficient data pipelines, data lakes, and data warehouses to support business intelligence and analytics.

Databricks Implementation: Utilize Databricks to build and manage big data processing and analytics solutions. Implement data models, ETL processes, and real-time data pipelines using Databricks and Apache Spark.

Data Integration: Integrate diverse data sources into unified data models, ensuring high data quality and consistency across platforms.

Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical specifications.

Optimization & Performance Tuning: Monitor and tune performance of data pipelines and Databricks clusters to ensure high efficiency and low latency.

Best Practices: Develop and enforce data engineering best practices, including coding standards, data governance, and documentation.

Troubleshooting: Identify and resolve data pipeline issues and perform root cause analysis to prevent future occurrences.

Mentorship: Provide guidance and mentorship to junior data engineers, fostering a collaborative and innovative team environment.

Qualifications:

Experience: 8-10 years of experience in data engineering, with a strong background in designing and managing data pipelines, data warehouses, and big data solutions.

Databricks Expertise: 2-3 years of hands-on experience with Databricks and Apache Spark. Proven track record of building and optimizing data pipelines using Databricks.

Technical Skills: Proficiency in SQL, Python, Snowflake and other relevant programming languages. Experience with cloud platforms (e.g., AWS, Azure, GCP).

Data Tools: Familiarity with data modelling tools, ETL tools, and data integration technologies. Experience with data visualization tools (e.g., Tableau, Power BI) is a plus.

Education: Bachelor s degree in computer science, Engineering, or a related field. Advanced degree is a plus.

Problem-Solving: Strong analytical and problem-solving skills with the ability to troubleshoot complex data issues.

Communication: Excellent verbal and written communication skills, with the ability to convey technical concepts to non-technical stakeholders.