i
Sopra Steria
175 Sopra Steria Jobs
3-5 years
₹ 9.6 - 20L/yr (AmbitionBox estimate)
Noida
PySpark Module Lead
Sopra Steria
posted 13hr ago
Flexible timing
Key skills for the job
We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will collaborate closely with our Data Scientists to develop and deploy machine learning models. Proficiency in below listed skills will be crucial in building and maintaining pipelines for training and inference datasets.
Responsibilities:
• Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines.
• Utilize PySpark for data processing, transformation, and preparation for model training.
• Leverage AWS EMR and S3 for scalable and efficient data storage and processing.
• Implement and manage ETL workflows using Streamsets for data ingestion and transformation.
• Design and construct pipelines to deliver high-quality training and inference datasets.
• Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities.
• Optimize and fine-tune pipelines for performance, scalability, and reliability.
• Ensure IAM policies and permissions are appropriately configured for secure data access and management.
• Implement Spark architecture and optimize Spark jobs for scalable data processing.
Total Experience Expected: 04-06 years
Employment Type: Full Time, Permanent
Read full job descriptionPrepare for Sopra Steria Module Lead roles with real interview advice
One of the best Europian Company to work with. It provides best Work Culture, lots of new things to learn.
Any organization can not be 100% as per your expectation so here also. Mostly depends on project to project, sometimes Department to deparment.
8-10 Yrs
Noida, Chennai