Job Title: Gen AI Architect Location: Fremont, CA (Onsite) Job Type: C2C Remote: No Job Summary We are seeking an experienced Senior Technical Lead / Gen AI Architect with strong expertise in Langfuse v3, Azure AI services, and GenAI lifecycle management. The role will involve architecting, implementing, and optimizing advanced GenAI evaluation frameworks, ensuring enterprise-grade reliability, scalability, and observability for AI-driven solutions. Key Responsibilities • Provide technical leadership to development teams, ensuring best practices, coding standards, and performance optimization in complex GenAI projects. • Architect and deploy Langfuse v3 in production environments, including upgrades from Langfuse v2 with backward compatibility. • Design and build modular components for prompt management, tracing, metrics, evaluation, and playground features using Langfuse v3. • Leverage Langfuse capabilities for: • Prompt Management versioning, templating, optimization • Tracing end-to-end visibility of GenAI workflows • Metrics performance, latency, and usage analytics • Evaluation automated/manual scoring of outputs • Playground interactive prompt testing/debugging • Integrate Azure AI Evaluation SDK into enterprise pipelines for scalable evaluation of GenAI models. • Build reusable components and templates for evaluation across diverse RAG, multi-agent, and GenAI lifecycle workflows. • Ensure scalability, observability, and traceability of evaluation frameworks in both offline and online environments. • Collaborate with cross-functional teams (product, data science, and engineering) to align evaluation metrics with business KPIs. • Conduct feasibility studies, recommend technical alternatives, and ensure compliance with architecture best practices. • Stay current with emerging GenAI evaluation tools and methodologies (e.g., TruLens, W&B, Helicone). Required Skills & Qualifications • Hands-on expertise in Langfuse (including v3 features) and integrations. • Proficiency with Azure AI Services and Azure AI Evaluation SDK. • Strong understanding of LLMOps, prompt engineering, and GenAI lifecycle management. • Experience with Retrieval-Augmented Generation (RAG), fine-tuning, and multi-agent orchestration. • Proficiency in Python (TypeScript or other GenAI-related languages is a plus). • Experience with cloud-native architectures (Azure preferred). • Familiarity with observability tools, tracing frameworks, and evaluation metrics for GenAI systems. • Excellent communication, documentation, and stakeholder collaboration skills. Nice to Have • Experience with additional observability tools such as TruLens, Weights & Biases (W&B), Helicone. • Exposure to evaluation standards for enterprise AI adoption. • Knowledge of multi-cloud and hybrid deployment strategies.