As a Senior Data Engineer at Bright Vision Technologies, you will be responsible for designing, developing, and maintaining Data-based solutions. You will collaborate with cross-functional teams to implement data processing and analytics pipelines, optimize performance, and ensure data reliability and security.
Responsibilities:
Build and optimize data pipelines using Spark, Hadoop, and Kafka.
Collaborate with data scientists to ensure seamless integration with data analytics platforms.
Design and implement data quality checks using Sparks data quality features.
Develop and maintain data transformation scripts using Sparks data transformation features.
Troubleshoot and resolve complex technical issues related to big data and analytics.
Requirements:
A strong background in data engineering with a minimum of 8+years of experience. Excellent programming and big data skills. Good communication skills and the ability to work collaboratively with our clients and team members.