Principal Technical Program Manager
Microsoft · microsoft.com
Mountain ViewLead / Manager6+ yrs
Posted 11 days ago. 9 more Program / Project Manager jobs in Mountain View, California went up on company career pages this week.
See the newer ones
Straight from the company’s own careers site.
The job in short
Lead offline quality measurement strategy and roadmap for Bing search and AI experiences.
Generative AILLMsRecommendation systemsData scienceAgenticLLM
Details
- Found
- 9 d ago · posted 2026-10-27
- Location
- Mountain View, CA, US
Full description
Define the future of quality measurement Own the long-term vision and roadmap for Bing's offline quality metrics ecosystem. Define measurement strategies for AI-powered search, answer experiences, ranking systems, retrieval, multimodal content, and agentic experiences. Identify gaps in existing quality measurements and drive invention of next-generation metrics. Ensure offline metrics accurately predict user value and align with online outcomes. Build diagnostic frameworks that surface which scenarios, segments, and page configurations drive competitive gaps, and translate those into concrete investment opportunities. Ensure teams can confidently use measurement systems for product development, experiment evaluation, launch readiness, executive scorecards, competitive tracking, and quality monitoring. Bachelor's Degree AND 6+ years experience in engineering, product/technical program management, data analysis, or product development 3+ years of experience managing cross-functional and/or cross-team projects. Bachelor's Degree AND 12+ years experience engineering, product/technical program management, data analysis, or product development OR equivalent experience. Experience defining or operating metrics used for large-scale product decisions. Demonstrated ability to lead cross-functional initiatives across multiple teams. Solid analytical and quantitative problem-solving skills. Excellent communication and stakeholder management abilities. Experience in search, recommendation systems, ranking, AI products, or large-scale consumer experiences. Experience building evaluation frameworks for LLMs, retrieval systems, or generative AI applications. Familiarity with offline evaluation methodologies and human rating systems Experience partnering with Data Science, Applied Science, and Research organizations. Proven success driving strategy in ambiguous and rapidly evolving technical domains.
Program / Project Manager jobs in Mountain View, California · Program / Project Manager jobs in the United States · All Microsoft jobs · Senior Program / Project Manager jobs in Mountain View, California · Senior Program / Project Manager jobs in the United States · Senior and manager-level jobs in Mountain View, California