HiLabs - Data Engineer I/II - Python/Apache Airflow (3-5 yrs)

Job Summary :

We are a leading Software as a Service (SaaS) company revolutionizing the U healthcare industry through transformative Artificial Intelligence (AI) solutions.

Our mission is to power the next generation of digital products and services through innovation, collaboration, and transparency.

We are seeking talented Data Engineering who are passionate about building scalable and efficient data systems.

The ideal candidate will have a strong technical background, a passion for data-driven solutions, and the ability to work in a fast-paced, dynamic environment.

Key Responsibilities :

Data Pipeline Development :

- Design, develop, and maintain robust and scalable ETL/ELT pipelines for ingesting and transforming large datasets from various sources.

- Optimize data workflows to ensure reliability, scalability, and performance.

Database Management :

- Manage and optimize SQL and NoSQL databases for efficient data storage, retrieval, and manipulation.

- Handle both structured and unstructured data to support diverse business needs.

Collaboration & Integration :

- Collaborate with data scientists, analysts, and other engineers to integrate and process data from disparate sources.

- Work closely with cross-functional teams to understand their data requirements and deliver tailored solutions.

Data Validation & Monitoring :

- Implement and maintain data validation processes to ensure data accuracy, consistency, and availability.

- Set up automated monitoring systems to track data health and resolve anomalies promptly.

Process Automation & Documentation :

- Automate repetitive data engineering tasks to streamline workflows and reduce manual effort.

- Document all processes, workflows, and infrastructure to enable efficient maintenance and scalability.

Desired Profile :

Educational Qualifications : Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Technical Skills :

- Expertise with tools like Apache Airflow, Talend, Informatica, or similar.

- Proficiency in Python, Scala, or Java for data manipulation and pipeline development.

- Advanced knowledge of SQL databases (e., MySQL, PostgreSQL) and NoSQL databases (e., MongoDB, Cassandra).

- Big Data Frameworks : Familiarity with frameworks such as Hadoop, Spark, or Flink.

- Experience with AWS, Google Cloud, or Azure and their data services (e., S3, Redshift, BigQuery).

- Understanding of data warehousing concepts and experience with platforms like Snowflake or Redshift.

- Proficiency with Git for version control and collaboration.

Domain Knowledge :

- Understanding of data governance, data security best practices, and compliance regulations (e., GDPR, HIPAA).

- Experience in data modeling and building scalable data architectures.

Soft Skills :

- Strong analytical and problem-solving skills.

- Ability to work collaboratively in a team environment.

- Excellent communication skills to convey technical concepts to non-technical stakeholders.

- Proactive approach to identifying and resolving challenges.

What We Offer :

- An opportunity to work with a leading SaaS company driving innovation in healthcare AI.

- Collaborative and dynamic work environment.

- Opportunities for professional growth and skill enhancement.

- Competitive compensation and benefits package