Engineering Analyst, AI Answers, Google Search

Responsibilities

• check_circle Help to establish metrics to benchmark Generative-AI issues and quantify improvements. Develop datasets and run evals for engineers to evaluate and improve Search, Labs & Assistant products.
• check_circle Perform complex data analyses to derive insights that help identify and fight abuse across our products. Analyze and debug escalations and identify trends that can indicate bigger product risks.
• check_circle Use technical judgement to drive project delivery, challenge proposals and identify risks. Manage multiple interdependent projects of moderate to high complexity and scope against crashed timelines.
• check_circle Improve tools through data analysis, technical expertise, and presentations to key stakeholders. Enhance our operational workflows via process improvements and identification of automation opportunities.
• check_circle Review graphic, controversial, and sometimes offensive/sensitive content. Collaborate with other teams & functions: RAI, Policy, Product, etc.

Basic qualifications

• Bachelor's degree or equivalent practical experience.
• 2 years of experience in data analysis, including identifying trends, generating summary statistics, and drawing insights from quantitative and qualitative data.
• 2 years of experience utilizing Python and SQL to solve engineering problems.
• Experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).

Preferred qualifications

• Experience working with abuse, spam, fraud or malware.
• Experience working on product policy analysis and identifying policy risks.
• Experience working with CBRN and cyber threat risks in GenAI.
• Experience using statistical analysis and hypothesis testing.
• Experience analyzing ML models performance or working on LLMs.
• Strong influencing skills and ability to work in a dynamic environment.

About the company