Data Engineer

ql-editor ">

Job Title: Data Engineer

We are seeking a skilled Data Engineer to join our team. In this role, you will design, implement, and maintain robust data pipelines and architectures to support our data-driven initiatives. You will work closely with data scientists, analysts, and business teams to ensure the efficient flow and integration of data across various systems, allowing for insightful analysis and decision-making.

• Data Pipeline Development: Design, build, and maintain scalable and efficient data pipelines to collect, clean, process, and store data from multiple sources.
• Data Integration: Collaborate with cross-functional teams to integrate data from various databases, APIs, and third-party services into centralized data platforms.
• Data Modeling Architecture: Develop and optimize data models, schemas, and databases to support data analysis and reporting.
• ETL Process Management: Implement and manage Extract, Transform, Load (ETL) processes to ensure data is properly prepared and available for analytics.
• Performance Optimization: Monitor and optimize the performance of data processing and storage solutions to handle large volumes of data efficiently.
• Collaboration: Work closely with data scientists, analysts, and business teams to understand data needs and provide data solutions that meet organizational goals.
• Data Quality Assurance: Ensure data integrity, quality, and security across all data systems and pipelines.
• Documentation: Maintain clear documentation of data engineering processes, architectures, and workflows.

• Education: Bachelor s degree in Computer Science, Information Technology, Engineering, or a related field. A Master s degree is a plus.
• Experience:
• Minimum of 4+ years of experience in a data engineering role or similar.
• Experience with building and maintaining data pipelines and ETL processes.
• Familiarity with cloud platforms (AWS, Azure, Google Cloud) and big data tools (e.g., Hadoop, Spark).
• Technical Skills:
• Proficiency in programming languages like Python, Java, or Scala.
• Strong knowledge of SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB).
• Experience with data warehousing solutions (e.g., Redshift, BigQuery, Snowflake).
• Familiarity with data orchestration tools (e.g., Apache Airflow, Luigi).
• Knowledge of data modeling, data governance, and data security best practices.
• Soft Skills:
• Strong analytical and problem-solving abilities.
• Excellent communication skills, both written and verbal.
• Ability to work collaboratively in a fast-paced, cross-functional team environment.

• Familiarity with data visualization tools (e.g., Tableau, Power BI).
• Experience with machine learning frameworks and workflows.
• Knowledge of containerization tools (e.g., Docker, Kubernetes).
• Understanding of DevOps principles for data infrastructure management.