Data Engineer - Dataiku Platform (2-4 yrs)

Job Description :

We are seeking a highly skilled Data Engineer to join our team.

The ideal candidate will have expertise in managing model operations on the Dataiku platform, alongside a strong background in data engineering practices.

In this role, you will be responsible for building and maintaining robust data pipelines, ensuring the smooth operation of data models, and optimizing data workflows to drive actionable insights and support business objectives.

Key Responsibilities :

Model Operations Management :

- Oversee and manage model lifecycle operations on the Dataiku platform, including deployment, monitoring, and optimization.

- Collaborate with data scientists to integrate and operationalize machine learning models.

- Implement and manage model versioning and rollback procedures.

Data Pipeline Development :

- Design, build, and maintain scalable and efficient data pipelines to support analytics and reporting needs.

- Extract, transform, and load (ETL) data from various sources into Dataiku and other data repositories.

- Optimize data ingestion and processing workflows for performance and reliability.

Platform Optimization :

- Ensure the Dataiku platform is configured for optimal performance, scalability, and security.

- Troubleshoot and resolve platform-related issues, including data integration and processing challenges.

Collaboration & Documentation :

- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.

- Document data workflows, model operations, and data pipeline architectures.

- Provide training and support to team members on best practices for using the Dataiku platform.

Quality Assurance :

- Monitor data quality and integrity throughout the pipeline and model lifecycle.

- Implement automated testing and validation procedures to ensure data accuracy and consistency.

Continuous Improvement :

- Stay up-to-date with the latest advancements in data engineering, machine learning, and Dataiku platform features.

- Identify opportunities for process improvements and implement innovative solutions.

Qualifications :

Education : Bachelor's degree in Computer Science, Engineering, Data Science, or a related field. Master's degree or relevant certifications is a plus.

Experience :

- Proven experience working with the Dataiku platform, including model deployment and management.

- Strong background in data engineering, including experience with ETL processes, data warehousing, and data integration.

- Hands-on experience with SQL, Python, or other data processing languages.

Skills :

- In-depth knowledge of the Dataiku platform and its features.

- Proficiency in building and managing data pipelines and workflows.

- Strong analytical and problem-solving skills with a focus on data quality and performance.

- Excellent communication skills with the ability to collaborate effectively with cross-functional teams.

Preferred :

- Experience with cloud data platforms such as AWS, Azure, or Google Cloud.

- Familiarity with data modeling, machine learning algorithms, and their application in a business context