OpeningsFirst
How it worksPricingLog in
OpeningsFirst › Jobs in the United States › Jobs in New York, New York

Remote | Member of Technical Staff, Legal Research — $400,000–$800,000/year

24 Mag · 24-mag.com
New YorkRemoteLead / Manager
Apply on 24 Mag’s site

Share this job on LinkedIn · X

Straight from the company’s own careers site.

The job in short

A lead / manager full-time remote Legal AI Evaluation Specialist role at 24 Mag. Skills mentioned: LLM, Agentic, Compliance, Legal.

LLMAgenticComplianceLegal

Details

Found
44 d ago · posted 2026-10-03
Location
New York
Pay (as listed)
$400,000–$800,000 a year

Full description

We are sharing a full-time opportunity for experienced legal researchers and legal professionals with deep expertise in legal analysis, research methodology, benchmarking, and complex legal workflows to contribute to advanced AI research and evaluation initiatives. Selected professionals will design rigorous evaluation frameworks for legal AI systems, conduct applied research on legal reasoning and interpretation, develop realistic benchmark environments, and collaborate with researchers and engineers to assess and improve AI performance across sophisticated legal tasks. Key Responsibilities Legal AI Evaluation Frameworks • Design and own evaluation frameworks for AI systems operating in legal domains • Develop benchmark suites, scoring methodologies, and legal-quality rubrics • Establish research-grade protocols for evaluating complex legal tasks • Define consistent standards for assessing legal reasoning and analytical quality • Evolve evaluation methodologies as legal AI capabilities advance Legal Research & Reasoning • Conduct applied research on legal reasoning and statutory interpretation • Analyse legal arguments, authorities, and structured decision-making processes • Research complex questions across relevant areas of law • Develop methodologies for assessing legal outcomes and research quality • Translate sophisticated legal concepts into structured evaluation criteria Benchmark & Dataset Development • Develop and curate datasets reflecting realistic legal challenges • Create case studies and benchmark environments for legal AI evaluation • Design tasks involving legal research, analysis, and workflow execution • Support evaluation of agentic legal workflows and automation • Maintain benchmark quality, consistency, and practical relevance Model Evaluation & Failure Analysis • Evaluate legal-focused AI models and agentic systems • Analyse reasoning quality, output reliability, and recurring failure modes • Identify weak interpretations, unsupported conclusions, and methodological errors • Assess AI performance across different legal use cases • Provide structured findings to support model and evaluation improvements Research Collaboration & Standards • Collaborate with legal researchers, engineers, and interdisciplinary teams • Produce research findings, evaluation methodologies, and best-practice guidance • Monitor developments in legal technology, jurisprudence, regulation, and AI research • Contribute to legal benchmarking and knowledge-management standards • Help establish rigorous practices for evaluating AI-supported legal workflows Ideal Profile • Advanced degree in Law, such as JD, LLM, SJD, PhD in Law, or equivalent • Deep expertise in one or more substantive legal domains • Relevant areas may include corporate law, contracts, litigation, regulatory compliance, intellectual property, employment law, or public policy • Demonstrated experience conducting legal research, legal analysis, or policy work • Strong understanding of legal reasoning and research methodology • Experience developing structured approaches for assessing legal arguments or outcomes • Strong analytical reasoning and written communication skills • Ability to translate legal expertise into rigorous evaluation criteria • Experience working with interdisciplinary research, technology, or legal-operations teams • Familiarity with legal technology or AI-enabled legal workflows is advantageous • Background in legal benchmarking or legal knowledge management is beneficial • Published legal scholarship or policy research is valuable • Contributions to legal standards, frameworks, or research methodologies are advantageous Engagement Details • Full-time engagement • Fully remote • Compensation: $400,000–$800,000/year • The source materials contain a substantial compensation inconsistency, so final compensation terms should be confirmed during the hiring process • Work will involve legal research, AI evaluation, benchmark development, methodology design, and failure analysis • Assignments may span corporate law, contracts, litigation, regulatory compliance, intellectual property, employment law, and public policy • Strong legal-research methodology and structured analytical judgement are central to this role • Collaboration will involve researchers, engineers, legal professionals, and other multidisciplinary contributors • Equity compensation and performance-based incentives may be available depending on applicable employment terms and policies • Benefits may include health-insurance support, paid time off, retirement-plan contributions, and other employment benefits • Work must be completed without using confidential, privileged, proprietary, or otherwise restricted information belonging to any employer, client, law firm, institution, or other third party About the Platform This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy

All 24 Mag jobs · Senior and manager-level jobs in New York, New York · Senior roles at 24 Mag

Get jobs like this every morning

Jobs in India · Jobs in the US · Who’s hiring in India · Who’s hiring in the US · Browse all jobs · Companies · How OpeningsFirst works · Free jobs widget · Leadership jobs in India · Leadership jobs in the US · Senior jobs in India · Senior jobs in the US · Fresher jobs in India · Entry-level jobs in the US · Remote jobs in the US · Jobs paying $150k+ · Leadership hiring this week · Guides
OpeningsFirst · Use with AI · Terms · Privacy · Refunds · Contact · Follow us on LinkedIn · X