Infiniti Research - Senior Data Engineer (5-7 yrs)

Experience Level : 5+ years.

Job Type : Full-time

Job Title : Senior Data Engineer

Job Description :

We are seeking a highly skilled Senior Data Engineer to join our team and design, develop, and maintain scalable data pipelines and systems using Apache Spark, Databricks, and SQL.

In this role, you will collaborate with data scientists, analysts, and other stakeholders to deliver high-quality data solutions, optimize workflows for performance and scalability, and ensure data integrity across multiple sources.

Responsibilities :

- Design, develop, and maintain robust data pipelines and systems using Apache Spark and Databricks.

- Collaborate with cross-functional teams to gather data requirements and implement effective data solutions.

- Optimize ETL processes for data ingestion and transformation, ensuring performance, scalability, and data quality.

- Implement data quality checks to ensure integrity across various data sources.

- Monitor, troubleshoot, and resolve data pipeline issues promptly.

- Stay updated with the latest technologies and industry trends in data engineering.

Requirements :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 5+ years of experience in data engineering or related roles.

- Expertise in Apache Spark, Databricks, and SQL, including performance optimization.

- Strong proficiency with cloud platforms (AWS, Azure, or Google Cloud).

- Experience with Python or Scala for data engineering tasks.

- Familiarity with data warehousing technologies (e., Snowflake, Redshift).

- Strong problem-solving abilities and attention to detail.

- Excellent communication and collaboration skills.

Preferred Qualifications :

- Experience with real-time data processing (e., Kafka, Flink).

- Knowledge of data governance, security best practices, and CI/CD pipelines.