1 Osfin.ai Senior Data Engineer Job

Responsibilities :

- Design, develop, and maintain data pipelines and ETL processes using a variety of tools and technologies, including but not limited to :

1. Cloud platforms : AWS, Azure, or GCP

2. Data warehousing : Snowflake, Redshift, BigQuery

3. Data streaming : Kafka, Kinesis, Pub/Sub

4. Data processing : Spark, Hadoop, Hive

3. Programming languages : Python, Java, Scala

- Develop and implement data quality checks and monitoring systems to ensure data accuracy and integrity.

- Collaborate with data scientists and analysts to understand business requirements and translate them into technical solutions.

- Troubleshoot and resolve data-related issues.

- Participate in the design and implementation of data governance policies and procedures.

- Stay up-to-date with the latest advancements in data engineering technologies and best practices.

- Mentor junior data engineers and provide guidance on technical and professional development.

Qualifications :

- 5-6 years of experience as a Data Engineer or in a similar role.

- Strong proficiency in SQL and at least one scripting language (Python, Java, Scala).

- Experience with cloud platforms (AWS, Azure, or GCP) and their data services.

- Experience with data warehousing and data lake technologies.

- Experience with data streaming and real-time data processing.

- Experience with data quality and data governance best practices.

- Excellent communication and collaboration skills.

- Ability to work independently and as part of a team.

- Strong problem-solving and analytical skills.

- Passion for learning and staying up-to-date with the latest technologies.

Bonus Points :

- Experience with containerization technologies (Docker, Kubernetes).

- Experience with machine learning and deep learning frameworks.

- Experience with data visualization tools (Tableau, Power BI).

- Experience with Agile development methodologies.

- Contributions to open-source projects