Data Engineer - SME

Responsibilities:

Collaborate with data scientists to understand their data requirements and ensure that the data architecture meets their needs.

Develop, maintain, and optimize data pipelines to efficiently process large volumes of data.

Implement best practices for data governance, ensuring data quality, integrity, and security.

Optimize and tune data systems for improved performance and scalability, addressing bottlenecks and optimizing resource usage.

Provide technical leadership, guidance, and mentorship to junior data engineers, helping them grow their technical skills.

Troubleshoot and resolve data-related issues, ensuring timely and accurate data delivery.

Stay updated with the latest trends in big data technologies, cloud platforms, and data engineering methodologies.

Qualifications:

Bachelors degree in computer science, Data Engineering, or a related field.

At least 2 years of proven experience in data engineering role.

Strong expertise in data warehousing concepts and technologies.

Hands-on experience with big data technologies such as Hadoop, Spark, and related ecosystems.

Proficiency in ETL tools (e.g., Apache NiFi, Talend).

Experience with cloud platforms such as AWS or Azure, including data storage, processing, and analytics services.

Excellent problem-solving and analytical skills with the ability to work in complex environments.

Strong programming skills in Python or Java.

Skills:

Data Modeling: Ability to design and implement robust data models for various use cases.

ETL Tools: Experience with tools like Apache NiFi, Talend, or similar for building scalable ETL pipelines.

Databases: Proficiency in working with both SQL and NoSQL databases.

Big Data Technologies: Experience with Hadoop, Spark, or other big data frameworks.

Cloud Platforms: Hands-on experience with cloud environments, particularly AWS or Azure.

Programming: Strong programming skills in Python or Java to build and maintain data systems.