PySpark Developer (6-10 yrs)

Pyspark Developer

Key things : Python, Pyspark

Offshore Years exp 6+ yrs

No of positions : 5 roles

Hyderabad- 5 days onsite

Shift timings are 1pm-10pm

Responsibilities :

- Develop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation.

- Collaborate with product and technology teams to design and validate the capabilities of the data platform

- Identify, design, and implement process improvements: automating manual processes, optimizing for usability, re-designing for greater scalability

- Provide technical support and usage guidance to the users of our platform's services.

- Drive the creation and refinement of metrics, monitoring, and alerting mechanisms to give us the visibility we need into our production services.

Qualifications :

- Experience building and optimizing data pipelines in a distributed environment

- Experience supporting and working with cross-functional teams

- Proficiency working in Linux environment

- 4+ years of advanced working knowledge of SQL, Python, and PySpark

- Knowledge on Palantir

- Experience using tools such as: Git/Bitbucket, Jenkins/CodeBuild, CodePipeline

- Experience with platform monitoring and alerts tools