DynPro - Data Engineer - SQL/Spark/Scala (3-6 yrs)

Hiring Alert!!

- Only immediate joiners preferred, notice period won't be considered- .

Minimum experience : 3 years!!.

We are seeking a skilled and motivated Data Engineer to join our team.

- The ideal candidate will play a critical role in designing, building, and maintaining scalable data pipelines and infrastructure to support data-driven decision-making across the organization.

- Proficiency in SQL, Spark, and Scala is essential for success in this role.

Key Responsibilities :

- Data Pipeline Development : Design, build, and optimize robust data pipelines and workflows to collect, process, and transform large datasets.

- Data Integration : Integrate data from various sources, ensuring consistency, accuracy, and reliability.

- Big Data Processing : Leverage Apache Spark to perform distributed data processing and manage large-scale datasets efficiently.

- Database Management : Use SQL to query and manage structured and semi-structured data, ensuring high performance and scalability.

- Code Development : Develop clean, reusable, and efficient code using Scala for data processing and transformations.

- Data Quality and Validation : Implement mechanisms to monitor and maintain data quality across pipelines.

- Collaboration : Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.

- Performance Optimization : Optimize data pipeline performance and troubleshoot bottlenecks or issues in processing workflows.

- Documentation : Create and maintain technical documentation for data pipelines, processes, and infrastructure.

Key Skills and Qualifications :

Technical Expertise :

- Proficiency in SQL for querying, analyzing, and managing large datasets.

- Hands-on experience with Apache Spark for distributed data processing.

- Strong programming skills in Scala for building and optimizing data pipelines.

- Big Data Frameworks : Experience with big data technologies and frameworks such as Hadoop, Spark Streaming, or Hive is a plus.

- Data Modeling : Understanding of data modeling, ETL processes, and data warehousing concepts.

- Cloud Platforms : Experience with cloud-based data solutions (e.g, AWS, Azure, or Google Cloud) is advantageous.

- Problem-Solving : Strong analytical and problem-solving skills for troubleshooting complex data workflows.

- Tools : Familiarity with version control systems (e.g, Git) and CI/CD pipelines for data engineering workflows.

- Communication : Excellent written and verbal communication skills to collaborate with technical and non-technical teams.