Data Engineer

Job Overview:

We are seeking a talented and motivated Data Engineer with a minimum 3 years of experience to join our dynamic team. The ideal candidate will possess a strong background in designing, building, and maintaining scalable data pipelines and databases. You will work closely with engineering teams and product owner to translate requirements into resilient, secure, and high-performance software. Key responsibilities include designing, coding, testing, and implementing new or existing software products based on specifications. A crucial aspect of this role is a commitment to continuous learning and adaptability to evolving technologies and market trends.

Key Responsibilities:

• Data Pipeline Development:
  • Design, build, and maintain robust and scalable data pipelines to process and transform large volumes of data from various sources.
  • Implement ETL (Extract, Transform, Load) processes to ensure data is accurately and efficiently integrated into data warehouses or data lakes.

Database Management:

• Develop, build and optimize database queries and schema designs to enhance performance and support analytical needs.

Data Integration:

• Collaborate with cross-functional teams to gather data requirements and integrate data from diverse sources including APIs, flat files, and databases.
• Ensure data consistency, accuracy, and integrity across various systems.Data Quality and Monitoring:
• Implement data quality checks and monitoring processes to identify and resolve data issues proactively.
• Develop and maintain data documentation and metadata for transparency and reproducibility.

Performance Tuning:

• Monitor and tune the performance of data pipelines and databases to ensure optimal operation and scalability.
• Troubleshoot and resolve data-related issues in a timely manner.

Collaboration and Communication:

• Work closely with various stakeholders to understand their data needs and provide appropriate solutions.
• Communicate technical concepts and solutions effectively to both technical and non-technical team members.

Continuous Learning and Adaptation:

• Stay current with emerging technologies, trends, and best practices in data engineering.
• Demonstrate a proactive attitude towards learning new tools and technologies, and adapt quickly to changes in the market and industry requirements.

Qualifications:

• Education:
  • Bachelor s degree in Computer Science, IT, Engineering or a related field

Experience:

• 3+ hands-on experience in data engineering
• Proficiency with data warehousing solutions and ETL tools.

Technical Skills :

• Proficient in SQL: Extensive experience in writing complex queries, stored procedures, and functions in SQL and Azure SQL Managed Instance (SQL-MI).