Data Engineer

Role & responsibilities

We are seeking a talented and enthusiastic Data Engineer with 2-4 years of experience to join our data engineering team. The ideal candidate will have a strong background in ETL processes, data warehousing, and big data technologies, and will play a key role in building and maintaining our data infrastructure to support analytics and business intelligence needs.

Key Responsibilities:

• ETL Development: o Design, develop, and maintain efficient ETL pipelines to process and transform large volumes of data from various sources. o Implement data transformation and cleansing processes to ensure high-quality data.

• Data Warehouse Development: o Assist in building and optimizing scalable data warehouse solutions using Snowflake, Databricks, Redshift, or similar technologies. o Ensure that data models are efficient, scalable, and support the needs of analytics teams.

• Big Data & Cloud Technologies: o Utilize AWS Glue and PySpark for processing and transforming large datasets in the cloud. o Implement data pipelines using Apache Airflow for scheduling and orchestration.

• Database Management: o Manage and optimize relational databases (RDBMS) to support both transactional and analytical workloads. o Write efficient SQL queries to support data extraction, reporting, and analytics.

• Collaboration: o Work closely with your lead and other stakeholders to understand data requirements and deliver reliable data solutions. o Contribute to the design and architecture of data systems, ensuring alignment with business needs and technical best practices.

• Data Quality & Governance: o Ensure data integrity, accuracy, and availability through rigorous testing, validation, and monitoring of ETL processes. o Support data governance initiatives by implementing best practices for data management and security.

Required Skills & Qualifications: Education:

Bachelors degree in Computer Science, Information Systems, Engineering, or a related field.

Experience:

• 3-4 years of experience in data engineering, with a strong focus on ETL processes and data warehousing.

• Hands-on experience with ETL tools and frameworks, as well as Python and PySpark for data processing.

• Experience with cloud-based data platforms such as AWS Glue, Snowflake, Databricks, or Redshift.

• Proficiency in SQL and experience with RDBMS platforms (e.g., MySQL, PostgreSQL, Oracle). • Familiarity with data orchestration tools like Apache Airflow. Technical Skills: • Strong understanding of data warehousing concepts and best practices.

• Experience in building and optimizing data pipelines and workflows.

• Knowledge of cloud infrastructure (AWS, Azure, GCP) and data processing technologies. Soft Skills: • Strong analytical and problem-solving skills.

• Excellent communication and teamwork abilities.

• Ability to work in a fast-paced environment and manage multiple tasks simultaneously. Preferred Qualifications:

• Experience with machine learning pipelines and big data processing.

• Familiarity with DevOps practices and continuous integration/continuous deployment (CI/CD) pipelines.

• Certification in AWS or other relevant cloud platforms