Python Pyspark Developer

Senior Python Developer/Lead:

Job description:

• Lead and mentor a team of Python developers.
• Design, develop, and maintain highly scalable data processing applications
• Write efficient, reusable and well documented code
• Deliver big data projects using Spark, Scala, Python, SQL, HQL, Hive
• Leverage data pipelining application to package work
• Maintain and tune existing Hadoop applications
• Work closely with QA, Operations and various teams to deliver error free software on time
• Perform code reviews and provide constructive feedback.
• Actively participate in daily agile scrum meetings

Job skill

• 7+ years of software development experience with Hadoop framework components(HDFS, Spark, PySpark, Sqoop, Hive, HQL, Spark, Scala)
• Experience in a leadership or supervisory role.
• 4+ years of experience using Python, SQL and shell scripting
• Experience in developing and tuning spark applications
• Excellent understanding of spark architecture, data frames and tuning spark
• Strong knowledge of database concepts, systems architecture, and data structures is a must
• Process oriented with strong analytical and problem solving skills
• Excellent written and verbal communication skills
• Bachelor's degree in Computer Science or related field
• Components(HDFS, Spark, PySpark, Sqoop, Hive, HQL, Spark, Scala)
• 4+ years of experience using Python, SQL and shell scripting
• Experience in developing and tuning spark applications
• Excellent understanding of spark architecture, data frames and tuning spark
• Strong knowledge of database concepts, systems architecture, and data structures is a must
• Process oriented with strong analytical and problem-solving skills
• Excellent written and verbal communication skills
• Bachelor's degree in Computer Science or related field