Build and optimize data pipelines using Spark, Hadoop, and Kafka.
Collaborate with data scientists to ensure seamless integration with data analytics platforms.
Design and implement data quality checks using Sparks data quality features.
Develop and maintain data transformation scripts using Sparks data transformation features.
Troubleshoot and resolve complex technical issues related to big data and analytics.
Requirements:
A strong background in data engineering with a minimum of 8+years of experience. Excellent programming and big data skills. Good communication skills and the ability to work collaboratively with our clients and team members.