Data Engineer - Scala/PySpark (5-8 yrs)

Data Engineer (Infosys Consulting - Large US Client)

Location :

We help companies find the perfect talent to fuel their growth, and we empower tech professionals to land their dream jobs.

About the Role Infosys Consulting is seeking a highly skilled Data Engineer to join their team, supporting a large US client.

In this role, you will be responsible for developing, maintaining, and monitoring data pipelines using Scala, Spark, and Azure technologies.

You will play a critical role in ensuring the accuracy, consistency, and efficiency of data ingestion and transformation processes.

Responsibilities :

- Design, develop, and implement data pipelines using Scala and Spark on Azure Databricks. Monitor, troubleshoot, and resolve data pipeline errors and performance issues.

- Investigate and address data discrepancies to ensure data quality and consistency.

- Collaborate with data analysts and stakeholders to understand data requirements and provide technical support.

- Manage source code and configurations using Git for version control.

- Automate data pipeline deployments using appropriate tools and techniques.

- Stay up-to-date on the latest advancements in big data technologies and best practices.

Qualifications :

- 5+ years of experience as a Data Engineer or related role.

- Proven expertise in Scala, Spark, and Azure technologies (Synapse, ADF, SQL).

- Strong understanding of data warehousing and data lake concepts.

- Experience with data quality tools and techniques.

- Excellent problem-solving and analytical skills.

- Ability to work independently and as part of a team.

- Excellent communication and collaboration skills.

- Experience with Git and CI/CD pipelines is a plus.

- Opportunity to work on cutting-edge technologies

- Collaborative and supportive work environment Career growth and development opportunities

- Make a real impact on the success of innovative companies