At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY- GDS Consulting – Enterprise Risk (ER) – Regulatory Compliance –Senior – GxP AI/Model Evaluation Consultant As part of our EY-ER- Regulatory Compliance team, you will help the clients by understanding their business needs and delivering solutions in accordance with the EY guidelines & methodologies. As a Regulatory Compliance senior actively establish, maintain and strengthen internal and external relationships. In line with our commitment to quality, you’ll consistently drive projects to completion with high quality deliverables, achieve operational efficiency, proactively identify and raise risks with the client as well as EY senior management team and lead internal initiatives. The opportunity We are seeking a highly experienced and detail‑oriented AI Model Evaluation Consultant with strong expertise in validating AI/ML models, Generative AI (GenAI) systems, and multi‑agent workflows within GxP‑regulated and quality‑critical environments. The ideal candidate will ensure the reliability, fairness, robustness, and regulatory compliance of AI solutions while supporting cross‑functional teams in designing, testing, and maintaining production‑ready, inspection‑ready AI systems. This role is responsible for leading AI model evaluation and validation activities across the full model lifecycle, including model development, training, testing, deployment, and post‑deployment monitoring, in alignment with GxP principles, risk‑based validation, and data integrity expectations. The consultant will play a key role in embedding Responsible AI, quality‑by‑design, and governance‑by‑design practices, enabling the safe, trustworthy, and explainable use of AI and agent‑based solutions in regulated life sciences and pharmaceutical contexts. Your key responsibilities • Support development of AI/ML and Agentic AI strategies by providing analysis, validation insights, and readiness assessments. • Perform AI/ML model evaluation and validation across development, training, testing, and deployment phases. • Evaluate model performance using precision, recall, accuracy, F1‑score, and confusion matrices. • Assess model behavior across datasets, edge cases, and failure scenarios to identify bias, drift, and instability. • Demonstrate strong understanding of supervised and unsupervised learning models, including classification, regression, clustering, and anomaly detection. • Conduct hands‑on evaluation of GenAI and LLM‑based systems, including prompt behavior, response quality, and consistency. • Design and execute testing strategies for AI agents and multi‑agent workflows, covering orchestration logic, tool invocation, and output validation. • Use LangChain, LangGraph, and Langfuse for agent workflows, observability, traceability, and evaluation logging. • Collaborate with data scientists and engineers to identify model improvement opportunities and risk mitigations. • Document evaluation results, limitations, risks, and acceptance criteria in a clear, audit‑ready manner. Qualifications: