Applied Scientist, Artificial General Intelligence, AGI Data Services
Role overview
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems.
Responsibilities
- check_circle Collaborate closely with core scientist team developing Amazon Nova models.
- check_circle Lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows.
- check_circle Design auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks.
- check_circle Perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities.
- check_circle Develop and maintain LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment.
- check_circle Set up the configuration of data collection workflows and communicate quality feedback to stakeholders.
- check_circle Enhance customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services.
- check_circle Support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards.
- check_circle Work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
Basic qualifications
- Master's degree in computer science, mathematics, statistics, machine learning or equivalent quantitative field
- Experience programming in Java, C++, Python or related language
- Experience with SQL and an RDBMS (e.g., Oracle) or Data Warehouse
Preferred qualifications
- Experience implementing algorithms using both toolkits and self-developed code
- Have publications at top-tier peer-reviewed conferences or journals
Benefits
- check_circle Comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
- check_circle 401(k) matching
- check_circle Paid time off
- check_circle Parental leave
About the company
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
Tags & Focus Areas
About Amazon.com
Amazon is looking for an AI Content Expert II to help with annotations, content generation, and data analysis. As part of the Data Team, you will be responsible for delivering high-quality training data to improve and expand AGI's Large Language Models' (LLMs) capabilities. Key job responsibilities As an AI Content Expert, you will be responsible for creating training data that are complex in nature and will require you to make informed and high judgement decisions in each case. You wi...
Ready to Join the Team?
Apply once with DevFound — we route your profile to Amazon.com and keep you posted on matching AI roles.