Senior Associate_Pyspark _D&A

• Handson experience in Pyspark, preferably more than 3 years familiarity with RDD level programming as well as coding using Spark DataFrames API.
• Ability to develop and maintain data pipelines and ETLs using Python and Pyspark.
• Should have good knowledge of Python and Spark concepts like Driver vs Worker, actions and transforms, data partitioning and bucketing etc
• Experience in data management activities and data lifecycle management for example activities related to integrating source databases to a data lake via full batch or incremental data updates.
• Ability to design, implement, and optimize Spark jobs for performance and scalability.
• Perform data analysis and troubleshooting to ensure data quality and reliability.
• Experience in cloud technologies esp. object storage for data lakes is a must.
• Experience in Spark Streaming, GraphX is a plus.

Degrees/Field of Study required Master of Business Administration, Bachelor of Technology, Bachelor of Engineering Degrees/Field of Study preferred PySpark Accepting Feedback, Accepting Feedback, Active Listening, Analytical Thinking, Business Case Development, Business Data Analytics, Business Intelligence and Reporting Tools (BIRT), Business Intelligence Development Studio, Communication, Competitive Advantage, Continuous Process Improvement, Creativity, Data Analysis and Interpretation, Data Architecture, Database Management System (DBMS), Data Collection, Data Pipeline, Data Quality, Data Science, Data Visualization, Embracing Change, Emotional Regulation, Empathy, Inclusion, Industry Trend Analysis