Staff AI Scientist
Mentions using AI coding tools in workflow and focuses on building AI observability and guardrails—so you'll use AI-assisted coding and tooling to ship models and pipelines quickly.
About the Role
Staff AI Scientist to lead applied research and productionization of classifiers, datasets, and generative insights for Fiddler's AI Observability platform, ensuring safety, security, and quality of LLM and agentic applications in production. The role partners with engineering, product, and customers to design, train, and scale models and dataset pipelines under production constraints.
Job Description
Role
Lead applied research and engineering for models and datasets that power Fiddler’s Trust Service and guardrail classifiers, focusing on safety, security, and quality detection for LLMs and agentic applications. Translate research into robust, scalable production services and partner closely with Product, Platform, Backend, and Customer Success.
Key Responsibilities
- Lead applied research and development for models and datasets used in safety, security, and quality detection (e.g., prompt injection, jailbreaks, PII, hallucination, faithfulness).
- Design, train, and ship production classifiers under latency and cost constraints.
- Build synthetic and adversarial data pipelines, including methods for generation, filtering, and validation to expose relevant failure modes.
- Drive technical direction for generative insights (LLM/agent-powered analysis) to help diagnose AI application failures.
- Contribute to evaluation and experimentation infrastructure to measure model quality, regression, and drift.
- Explore reinforcement learning and preference-based methods where beneficial.
- Collaborate with backend and platform engineers to productionize research prototypes into hardened, observable services.
- Partner with Product, Solutions Engineering, and Customer Success to map enterprise needs to research problems and product features.
- Mentor AI Scientists, raise technical quality via code/design review, and represent the company externally when appropriate.
Requirements
- 7+ years of applied AI experience with a proven record of taking models from research to production.
- Experience with LLMs and agentic systems: prompting, fine-tuning, and evaluation.
- Deep expertise training and fine-tuning classifier models, including encoder architectures (BERT-family, ModernBERT) and LLM-as-classifier approaches.
- Hands-on dataset engineering: sourcing, labeling, synthetic generation, adversarial augmentation, and quality control.
- Proficiency in Python and the modern ML stack (PyTorch, Hugging Face, common training/serving frameworks).
- Comfortable working in production environments and collaborating on real-time inference, monitoring, and rollouts.
- Strong written and verbal communication; customer-facing and cross-functional collaboration skills.
- Ability to work in the Palo Alto office 2–3 days per week.
Preferred Qualifications (Even Better)
- M.S. or Ph.D. in CS, ML, Statistics, Physics, or related quantitative field.
- Published research at top ML/NLP venues (NeurIPS, ICML, ICLR, ACL, EMNLP).
- Experience with reinforcement learning, RLHF/RLAIF, or preference-based fine-tuning.
- Experience building synthetic data pipelines at scale, AI safety, red-teaming, or adversarial ML.
- Experience working with enterprise customers in regulated industries (finance, healthcare, government).
Compensation
- Bay Area, Seattle & New York City: $220,000 - $260,000 + equity & benefits
- Other cities: $175,000 - $230,000 + equity & benefits
Benefits
Competitive pay + equity; premium health, dental & vision (100% premium coverage for employees); 401(k) plan; open PTO; monthly fitness reimbursement; paid parental leave; team/company events and offsites; on-site perks at Palo Alto HQ (Caltrain pass, in-office massages, lunch Mon–Thu, snacks/drinks, social events).
Location & Work Arrangement
Hybrid role with expectation to be in the Palo Alto office 2–3 days per week.