PySpark Developer

Designing and developing big data applications using the PySpark framework to meet the needs of the business.Writing and optimizing Spark SQL statements to extract and manipulate large datasets.Developing and deploying Spark algorithms to perform data processing and analytics tasks, such as machine learning and graph processing.Debugging and troubleshooting Spark code to resolve any issues and improve the performance of the applications.Collaborating with cross-functional teams, such as data engineers and data analysts, to ensure that the PySpark applications are integrated with other systems.Creating and maintaining documentation to ensure that the big data architecture, design, and functionality are well understood by others.Should be detail-oriented, have excellent problem-solving and communication skills.