Role overview
As a Staff AI Engineer, you will be a technical multiplier and a strategic partner in our journey to leveraging AI to make the world financially strong. You will lead the design and implementation of production AI systems while ensuring that AI adoption is seamless and safe for the rest of the engineering organization.
We are looking for an AI Engineer with broad technical mastery who operates with high integrity, exhibits empathy for their peers and customers, and has a growth and teaching mindset. You know that your success is measured by quality and velocity metrics as well as how effectively your colleagues can leverage the tools and platforms you build. We’re looking for world-class builders and collaborators, not research scientists. We understand that an AI generalist won’t be an expert in every one of the bullet points below. If you are missing experience in some of these areas, we still want to speak with you.
Responsibilities
- Architect Production RAG Systems: Design and scale Retrieval-Augmented Generation pipelines, optimizing for precision and recall while managing the complexities of financial data structures.
- LLM Orchestration & Governance: Implement LLM gateways to handle provider failover, load balancing, and prompt caching. You will be part of the team controlling our cost, latency, and availability metrics. Advise when a use case is best served by AI, and when to avoid it. Governance should include processes to validate use cases and operational monitoring for policy compliance.
- Model Optimization & SFT: Identify opportunities to leverage Supervised Fine-Tuning (SFT) of existing models. You know when to prune a model for efficiency and how to curate high-quality synthetic data for training.
- Context Engineering: Master dynamic context window management to ensure models are "well-informed" without exceeding token budgets or inducing "lost in the middle" phenomena.
- Agentic Workflows & Tool Use: Design frameworks that allow LLMs to safely interact with internal financial APIs, moving beyond simple chat to autonomous task execution within strict guardrails.
- Technical training and leadership: Provide guidance and training for product engineering teams.
- Security: Design and implement systems that enforce strict access and authorization to data, validation of outputs, and enable integrity and non-repudiation, even with LLMs in the stack.
- The "Eval" Moat: Build automated evaluation frameworks (LLM-as-a-judge) to quantify model performance, ensuring that "hallucinations" are caught long before they reach a customer.
- Mindset: You possess the judgment to bypass the hype cycle and cut through analysis paralysis. You take calculated risks with a clear plan for mitigation, knowing exactly when to innovate and when to rely on battle-tested, proven patterns. You communicate and collaborate through every step of your process.
- Empathy: You treat your fellow technologists and product managers as your primary customers. You build abstractions that make it easy for others to do the right thing and hard to do the wrong thing.
- Integrity: You possess the integrity to view security and privacy controls not as hurdles, but as the infrastructure that enables us to move quickly and safely. You have the courage to flag risks, own your mistakes, and tell the truth even when it’s uncomfortable or inconvenient.
Basic qualifications
- Demonstrated experience designing systems that capture production feedback to create “ground truth” datasets, turning our operational exhaust into a competitive advantage.
- Expertise with BigQuery and VertexAI
- Production experience with Python, langchain, and langgraph
- Experience tuning and monitoring models in production environments
- Experience with PCI or other highly regulated environments
- AWS Bedrock and SageMaker
- Data engineering experience