Gen AI Engineer

Gen AI Data Engineer Job Description

Grade : Sr. Executive

Years of Work Experience : 5+

Work Location : Malaysia (Work from office)

Position: Contract for 1 year

Objective

We are seeking a highly motivated and experienced Data Engineer to join our Strategic Data Democratization product development team. The ideal candidate will have strong skills in data modelling, data integration, and data warehouse design, as well as experience with ETL processes and data pipeline management. The Data Engineer will work closely with cross-functional teams to develop and maintain a scalable and high-performance data architecture that supports the needs of our business.

Key Responsibilities

1. Data Modelling:

• Develop and maintain logical and physical data models for the data warehouse.
• Design and implement data structures that support reporting and analytics requirements.
• Collaborate with data analysts and business stakeholders to understand data requirements.

2. Data Integration:

• Develop and maintain ETL processes to integrate data from various sources into the data warehouse.
• Optimize data integration processes for performance and reliability.
• Troubleshoot and resolve data integration issues.

3. Data Pipeline Management:

• Design and maintain data pipelines that support real-time data ingestion and processing.
• Ensure data quality and consistency throughout the pipeline.
• Monitor and optimize data pipeline performance.

4. Data Warehouse Design:

• Design and maintain the overall data warehouse architecture.
• Implement and maintain data governance policies and procedures.
• Ensure data security and privacy compliance.

5. Cross-Functional Collaboration:

• Work closely with data analysts, business stakeholders, and other technical teams to understand data needs and requirements.
• Collaborate with software engineers to ensure that data integration and reporting requirements are incorporated into software development projects.
• Participate in project planning and estimation.

Requirements & Skills

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in data engineering, with a focus on data integration, data modeling, and data warehouse design.
• Experience with Cloud & On Premise data platforms, such as Oracle, AWS, Azure, or Google Cloud Platform.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration skills.
• Familiarity with Agile SDLC.

Mandatory skill

• Proficiency in SQL and experience with relational databases, such as Teradata , Hadoop , MSSQL , MySQL, PostgreSQL, or Oracle.
• Strong experience with ETL tools, such as Apache NiFi, Talend, , Informatica , ODI

Nice to have skills

• Informatica, ODI
• Teradata, Hadoop (Hive)
• Dataiku

Gen AI Engineer Job Description

Grade : Sr. Executive

Years of Work Experience : 5-8

Work Location : Malaysia (Work from office)

Position: Contract for 1 year

Objective

Develop and optimize AI models and infrastructure to deliver cutting-edge AI applications and chatbots.

Key Responsibilities

• Develop AI models using the latest advancements in LLMs and ensure their seamless integration into application frameworks.
• Optimize existing AI systems and models for improved efficiency and accuracy.
• Collaborate closely with data engineers and architects to implement effective data strategies for model training and evaluation.
• Stay current with AI research by attending conferences and workshops, bringing innovative ideas back to the team.
• Conduct rigorous testing and validation of AI models to ensure they meet quality standards.
• Write clean, manageable code and maintain proper documentation for all implemented AI systems.
• Participate in peer code reviews and contribute to team knowledge by providing expertise on coding standards and best practices.

Requirements & Skills

• Deep technical expertise in AI and machine learning, particularly in building and deploying large language models and transformers.
• Experience with RAG pipelines, vector embedding, and tokenization techniques, crucial for developing sophisticated AI systems.
• Strong knowledge of in-context learning, fine-tuning, zero-shot learning, few-shot learning, and prompting techniques.
• Proficiency in programming languages like Python, and experience with ML frameworks such as TensorFlow, PyTorch, and libraries from Hugging Face.
• Experience with cloud services (AWS, Azure, Google Cloud) and on-premises deployments, focusing on scalability and security of AI deployments.
• Ability to work collaboratively in a fast-paced, team-oriented environment

Educational Qualifications

• Bachelors degree in Computer Science, Artificial Intelligence, or related technical fields.
• Master’s degree or advanced studies in fields like Machine Learning, Data Science, or Computational Linguistics is preferred.
• Certifications or demonstrable experience in specific AI technologies, such as TensorFlow or PyTorch, and familiarity with cloud services used for AI deployments.