Data Engineer

We are looking for a skilled and innovative Data Engineer to join our dynamic team. In this role, you will design, build, and manage data pipelines, ensuring the effective collection, storage, and processing of data across various platforms. As a key player in our data engineering team, you will work closely with data scientists, analysts, and other stakeholders to deliver clean, scalable, and reliable data solutions that support data-driven decision-making.

Key Responsibilities:

• Data Pipeline Development:

  • Design, develop, and maintain efficient, scalable, and reliable data pipelines to collect, process, and store large volumes of structured and unstructured data.
  • Integrate data from multiple sources, ensuring consistency and quality of the data pipeline from end to end.

• Database Management:

  • Design and implement databases, data warehouses, and data lakes that support business requirements.
  • Optimize database performance for faster data retrieval, storage, and processing.

• Data Modeling:

  • Develop and implement data models to organize and structure data for analysis and reporting.
  • Create clear, concise data dictionaries and documentation to ensure clarity and consistency in data usage across teams.

• Data Transformation:

  • Clean, transform, and enrich raw data to make it usable for downstream analysis, reporting, and machine learning applications.
  • Implement ETL (Extract, Transform, Load) processes to ensure that data is accurately transformed and available in the right formats for analysis.

• Collaboration with Stakeholders:

  • Work closely with data scientists, business analysts, and other stakeholders to understand data requirements and ensure that data solutions meet business needs.
  • Provide data support and guidance to internal teams to help them extract actionable insights from data.

• Data Quality & Governance:

  • Implement data validation and quality assurance procedures to ensure the integrity, accuracy, and consistency of data.
  • Help enforce data governance best practices and ensure compliance with privacy and security regulations.

• Performance Optimization:

  • Continuously monitor and optimize data pipelines, databases, and processes for improved performance, scalability, and reliability.
  • Troubleshoot and resolve issues related to data infrastructure and processing.

• Automation & Tools:

  • Automate manual data processes and repetitive tasks to improve efficiency and reduce human error.
  • Stay updated with the latest trends in data engineering tools, technologies, and best practices, and apply them to enhance the data infrastructure.

• Documentation & Reporting:

  • Create detailed documentation for all data engineering processes, models, and workflows to ensure transparency and future maintainability.
  • Provide reports and regular updates to management on the status of data engineering initiatives, issues, and improvements.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 3+ years of experience as a Data Engineer or in a similar role, with hands-on experience designing and implementing data pipelines and data management systems.
• Strong knowledge of SQL and experience with relational and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB).
• Proficiency in programming languages like Python , Java , or Scala for data manipulation, processing, and pipeline development.
• Experience with data warehousing solutions such as Amazon Redshift , Google BigQuery , or Snowflake .
• Hands-on experience with ETL tools and frameworks (e.g., Apache Spark , Apache Airflow , Talend , Fivetran ).
• Familiarity with cloud platforms such as AWS , Azure , or Google Cloud and services like AWS S3 , Google Cloud Storage , AWS Lambda , etc.
• Experience with data modeling and designing data architecture for complex systems.
• Understanding of big data technologies like Hadoop , Kafka , or Apache Flink is a plus.
• Experience with version control systems (e.g., Git ) and Agile development practices.

Preferred Skills:

• Knowledge of machine learning algorithms and how they interact with data infrastructure.
• Familiarity with data visualization tools (e.g., Tableau, Power BI) and the ability to work with teams to integrate data pipelines into analytical workflows.
• Understanding of data privacy regulations and security standards (e.g., GDPR, HIPAA).
• DevOps skills and familiarity with automation and CI/CD pipelines for data engineering workflows.

Soft Skills:

• Problem-solving: Strong analytical and troubleshooting skills with the ability to resolve complex data infrastructure issues.
• Attention to detail: Ability to ensure data accuracy and consistency, even when handling large and diverse datasets.
• Collaboration: Excellent teamwork and communication skills, with the ability to work effectively with cross-functional teams.
• Adaptability: Ability to work in a fast-paced environment and quickly adapt to changing business and technical requirements.
• Time management: Ability to manage multiple tasks, prioritize effectively, and meet deadlines.