We are looking for an experienced Senior Data Scientist with strong expertise in Generative AI to design and build advanced AI systems including agentic applications, LLM evaluation frameworks, and observability solutions. The role involves developing, testing, and optimizing LLM-powered applications while ensuring performance, scalability, and reliability in production environments. The ideal candidate will work closely with cross-functional teams to implement GenAI solutions, evaluate model performance using modern frameworks, and continuously improve AI pipelines, prompts, and workflows to deliver high-quality intelligent systems. Key Skills and Responsibilities Design, develop, and deploy Generative AI and Agentic AI applications Build, test, and optimize LLM-based applications ensuring performance, reliability, and scalability Implement LLM evaluation frameworks, including LLM-as-a-Judge using open-source tools Develop and maintain testing frameworks for agentic workflows and AI systems Use LangFuse or similar observability tools to monitor and analyze LLM performance Work with cross-functional teams to integrate AI solutions into production systems Optimize prompts, pipelines, and AI workflows to improve model accuracy and efficiency Implement robust testing, evaluation, and monitoring mechanisms for GenAI applications Work with LLM frameworks, vector databases, and RAG-based architectures Stay updated with advancements in LLM architectures, agent frameworks, and GenAI technologies Key Qualifications 8–10 years of experience in Data Science, Artificial Intelligence, or Machine Learning Strong hands-on experience in Generative AI (GenAI) Experience building Agentic AI applications Experience in testing and evaluating LLM-based systems Hands-on experience implementing LLM-as-a-Judge evaluation frameworks Experience with LangFuse or similar LLM observability tools Strong programming expertise in Python and data science workflows Experience working with LLM frameworks and GenAI ecosystems Experience with LangChain or LlamaIndex (preferred) Knowledge of prompt engineering, evaluation pipelines, vector databases, and RAG architectures Experience building production-grade GenAI solutions Ability to work in Agile environments and collaborate with cross-functional teams Location : Pune,Bangalore,Hyderabad Apply by sending your CV to [email protected]