As a Member of Technical Staff on Evals, you'll own how we measure AI Employee performance on the job. Areas you may work in: • Evaluation systems: datasets, offline replay, scorers, and regression alerts • Feedback loops for models and agents • Autonomous failure analysis • Defining what good, degraded, and failed runs look like, and alerting on them You may be a good fit if you: • Have built evaluation or measurement systems, such as AI evals, experimentation, ranking, or search quality • Can turn ambiguous quality questions into concrete metrics and decisions • Have strong software engineering fundamentals and ship production systems • Are comfortable where there are few established patterns • Want to work in person in San Francisco Even better: • You've shipped LLM systems to production and learned where they break • You have strong data instincts and work well with researchers • You contribute to open source projects