Add office photos
All Filters
44 Eli Lilly and Company Jobs
AI Test Engineer- BU
Eli Lilly and Company
posted 10d ago
Job Role Insights
Flexible timing
Key skills for the job
Job Description
AI Testing Strategy and Planning
- Collaborate with cross-functional teams to develop comprehensive AI testing strategies and plans for AI-powered applications.
- Work closely with product managers, data scientists, and developers to understand AI model requirements, use cases, and project goals.
- Define the scope and objectives of AI testing efforts, including performance, accuracy, bias detection, and robustness of AI models.
Test Execution for AI Models and Algorithms
- Design, develop, and execute test cases for AI systems and models (including machine learning and deep learning algorithms).
- Test and validate AI solutions across various stages of the development lifecycle, including model training, testing, and deployment.
- Ensure that AI models meet business requirements and perform accurately under various real-world conditions.
- Evaluate the performance of AI models by assessing speed, efficiency, scalability, and resource utilization.
- Perform manual and automated testing on AI-based applications, platforms, and solutions.
AI Model Accuracy and Validation
- Test AI models for accuracy, precision, recall, F1 score, and other performance metrics.
- Ensure AI models fairness by conducting tests for potential bias in decision-making processes, especially in clinical or medical applications.
- Validate AI model predictions against real-world data, ensuring that results are consistent, reliable, and actionable.
Collaboration and Knowledge Sharing
- Work with data scientists, AI engineers and Test Manager to improve testing methodologies and continuously optimize AI model testing processes.
- Provide feedback on AI models, pointing out any potential improvements in testing coverage or areas for model retraining.
- Communicate findings, bugs, and issues related to AI models to technical teams, ensuring prompt resolution.
Test Automation for AI Projects
- Develop and implement automated testing scripts and frameworks specifically designed for AI applications.
- Utilize AI testing tools and frameworks (RAGAS etc.) to automate the validation of AI models and algorithms.
- Integrate automated AI testing within continuous integration and continuous deployment (CI/CD) pipelines.
Compliance and Regulatory Testing
- Ensure that AI applications comply with industry-specific regulations, especially in the pharma and healthcare sectors (e.g., FDA regulations, HIPAA compliance).
- Verify that all AI-driven processes adhere to ethical standards and data privacy laws.
Continuous Improvement and Research
- Stay up-to-date with the latest trends, tools, and techniques in AI testing and apply these advancements to optimize the testing process.
- Participate in AI testing forums and workshops, contributing insights to improve best practices within the team.
Reporting and Documentation
- Document test results, methodologies, and issues clearly, providing insights into test coverage, risk analysis, and performance benchmarks.
- Prepare detailed reports for both technical and non-technical stakeholders, summarizing testing outcomes and potential risks associated with AI implementations.
- Assist in the creation and maintenance of knowledge-sharing platforms related to AI testing best practices.
Key Skills and Qualifications:
Technical Expertise
- Strong knowledge of AI/ML testing methodologies and best practices.
- Experience with AI development frameworks and libraries such as TensorFlow, Keras, PyTorch, scikit-learn, RAGAS and MLlib.
- Proficient in testing tools and environments for AI-based systems (e.g., Jupyter Notebooks, Apache Spark, and DataRobot ).
- Experience with performance testing tools like Grafana and JMeter for AI solutions.
- Knowledge of Python, R, JavaScript or other programming languages frequently used in AI/ML.
- Understanding of test automation frameworks and experience in tools like Cypress , and JUnit for automating AI tests.
AI Model Evaluation
- Solid understanding of machine learning and deep learning models, including supervised and unsupervised learning techniques.
- Familiarity with evaluating AI models on metrics such as accuracy, precision, recall, F1 score, confusion matrices, and AUC.
- Ability to identify and test for model biases, fairness, and ethical implications, especially in sensitive applications like healthcare and pharma.
Analytical and Problem-Solving Skills
- Strong problem-solving abilities and keen attention to detail, with a systematic approach to diagnosing and resolving AI-related issues.
- Ability to perform root cause analysis of issues in AI algorithms and suggest actionable fixes.
Collaboration and Communication
- Excellent teamwork and communication skills, with the ability to collaborate with cross-functional teams, including data scientists, engineers, and product managers.
- Strong verbal and written communication skills to convey technical information clearly and concisely to both technical and non-technical stakeholders.
Experience
- Minimum of 8 years experience in software testing, with at least 2 years focused on testing AI/ML models or AI-based applications.
- Proven experience in testing AI/ML algorithms in production or staging environments.
- Experience working in a regulated industry (such as pharmaceuticals or healthcare) is a plus.
Preferred Qualifications:
- Experience with cloud platforms (e.g., AWS, Azure) for deploying AI applications and models.
- Familiarity with DevOps practices and integrating AI testing into CI/CD pipelines.
- Certification in AI/ML or related testing frameworks (e.g., Certified Software Test Professional (CSTP) or Certified AI Engineer ).
- Knowledge of big data technologies (e.g., Hadoop, Spark) and how they integrate with AI models.
Employment Type: Full Time, Permanent
Read full job descriptionPrepare for Test Engineer roles with real interview advice
What people at Eli Lilly and Company are saying
What Eli Lilly and Company employees are saying about work life
based on 406 employees
Flexible timing
Monday to Friday
No travel
Day Shift
Eli Lilly and Company Benefits
View more benefits
Compare Eli Lilly and Company with
Pfizer
Compare
GlaxoSmithKline Pharmaceuticals
Compare
Merck
Compare
Johnson & Johnson
Compare
Sanofi
Compare
AstraZeneca
Compare
Abbott
Compare
Bayer
Compare
Roche Holding
Compare
Hetero
Compare
Viatris
Compare
Akums Drugs & Pharmaceuticals Limited
Compare
DIVI'S Laboratories
Compare
Fresenius Kabi
Compare
Piramal Pharma
Compare
Novo Nordisk
Compare
Teva Pharmaceuticals
Compare
Sandoz
Compare
Par Pharmaceutical Companies
Compare
Novartis Healthcare
Compare
US Medical Omnichannel Lead-Associate Director
12-14 Yrs
Bangalore / Bengaluru
Manager/Sr. Manager-Pharmacovigilance
2-7 Yrs
Panipat, Yamunanagar, Faridabad +3 more
Associate Director - Corporate Communications, LCCI Bangalore
10-11 Yrs
Bangalore / Bengaluru
Share an Interview