Big Data Engineer

Responsibilities:

• Design and implement big data solutions , integrating them with existing data architecture.

• Develop, construct, test, and maintain architectures such as databases and large-scale processing systems.

• Implement optimization methods to improve the performance and scalability of data processing systems.

• Work on data modeling, mining, and warehousing .

• Create data tools for analytics and data scientist team members .

• Ensure architectures support business requirements .

• Maintain security and data privacy in data flow and storage.

• Collaborate with data scientists and architects on several projects.

• Resolve issues that arise in data management systems and improve data reliability, efficiency, and quality.

Qualifications:

• Bachelors or Masters degree in Computer Science, Engineering, or a related field .

• Proven experience as a Big Data Engineer .

• Strong experience with big data tools such as Hadoop, Spark, Kafka, etc.

• Proficiency in programming languages like Python, SQL, PySpark.

• Strong understanding of data structures, algorithms, and software design .

• Experience with data modeling, data warehousing, and building ETL pipelines .

• Excellent problem-solving skills and attention to detail .

• Strong communication and collaboration skills

Pyspark, Big Data, Haddop, Spark