107 Vy Systems Jobs

Job Title : AWS Bigdata Engineer (Spark, Scala)

Job Description :

Responsibilities :

- Design, develop, and maintain scalable data pipelines using Spark and Scala.

- Implement data processing workflows on AWS services like S3, Redshift, Glue, and EMR.

- Collaborate with data scientists and analysts to ensure data availability and quality.

- Optimize data storage and retrieval for performance and cost-efficiency.

- Monitor and troubleshoot data pipelines to ensure reliability and scalability.

- Develop and maintain CI/CD pipelines for data engineering workflows.

- Ensure data security and compliance with organizational policies.

Requirements :

- Strong experience with Spark and Scala for data processing.

- Proficiency in AWS services such as S3, Redshift, Glue, Lambda, and EMR.

- Solid understanding of distributed computing and big data technologies.

- Experience with SQL and NoSQL databases.

- Familiarity with DevOps practices and tools like Jenkins, Docker, and Kubernetes.

- Excellent problem-solving and communication skills.

- Bachelor's degree in Computer Science, Engineering, or a related field.