Data Engineer

Role & responsibilities

• Design and implement robust, scalable data pipelines and architectures.
• Develop and maintain ETL processes to extract, transform, and load data from various sources.
• Optimize and improve existing data systems for performance and reliability.
• Collaborate with data scientists and analysts to understand data needs and provide solutions.
• Monitor and troubleshoot data pipeline performance issues.
• Ensure data quality and integrity through testing and validation.
• Document data processes, architectures, and best practices.

Stay updated with the latest industry trends and technologies in data engineering

Preferred candidate profile

Programming Languages:

Proficiency in languages like Python, Java, or Scala.

Proficiency with SQL for database querying.

Database Management: Experience with relational databases (Netezza, Redshift, Snowflake, PostgreSQL, MySQL).

Data Warehousing Solutions: Familiarity with data warehousing technologies (Amazon Redshift, Google BigQuery, Snowflake).

ETL Processes: Knowledge of Extract, Transform, Load (ETL) tools (Apache Airflow, Talend, Informatica, Azure Databricks).

Big Data Technologies: Experience with Hadoop, Spark, and Kafka for processing large datasets.

Cloud Platforms: Familiarity with cloud services (AWS, Azure, Google Cloud Platform) for data storage and processing.

Data Modelling: Understanding of data modelling techniques and concepts (star schema, snowflake schema).

Data Governance and Security: Knowledge of data privacy regulations and best practices for data security.

• Bachelors degree in Computer Science, Engineering, or a related field (or equivalent experience).
• Proven experience as a Data Engineer or in a similar role.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with SQL and NoSQL databases (e.g., PostgreSQL, MongoDB, etc.).
• Familiarity with data warehousing solutions (e.g., Snowflake, Redshift).
• Knowledge of big data technologies (e.g., Hadoop, Spark) is a plus.
• Experience with cloud platforms (e.g., AWS, Google Cloud, Azure).
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration skills.