HubMub
-
Perplexity

Member of Technical Staff (Data Scientist, Evals)

Perplexity
Germany · Full-time · Not Applicable
Perplexity serves tens of millions of users daily with reliable, high-quality answers grounded in an LLM-first search engine and our specialized data sources. We aim to use the latest models as they are released, but the intelligence frontier is a jagged one, and popular benchmarks do not effectively cover our use cases. In this role, you will build specialized evals to improve answer quality across Perplexity, covering search-based LLM answers and other scenarios popular with our users.

Responsibilities

  • Architect and maintain automated evaluation pipelines to assess answer quality across Perplexity's products, ensuring high standards for accuracy and helpfulness
  • Design evaluation sets and methods specifically to measure the impact of tool calls (particularly web search retrieval) on the final answer's quality
  • Develop VLM-based solutions to programmatically evaluate how final answers render visually across different platforms and devices
  • Continuously review public benchmarks and academic evaluations for their applicability to the Perplexity product, adapting and incorporating them into our regular performance measurements
  • Operate within a small, high-impact team where your evaluation metrics directly shape product changes, collaborating closely with technical leadership to measure and improve Answer Quality

Qualifications

  • PhD or MS in a technical field or equivalent experience
  • 4+ years of experience in data science or machine learning
  • Strong proficiency in Python and SQL (expected to write production-grade code)
  • Experience building within a modern cloud data stack, specifically AWS and Databricks
  • Comfortable with agentic coding workflows and using AI-assisted development tools to iterate faster

Preferred Qualifications

  • 1+ years of experience working with LLMs at scale, specifically with LLM-as-a-judge setups
  • Prior experience working on customer-facing web products or consumer apps, with real user traffic at scale
  • A strong research background, with experience applying research methods to real-world ML problems
  • Experience defining evaluation metrics (e.g., factual consistency, hallucination rate, retrieval precision) and building ground truth datasets

Key Skills

Ranked by relevance

python cloud sql aws ai
Login to Apply
🇩🇪

Country Guide

Germany

Strong tech sector & EU Blue Card visa

Posted
May 11, 2026
Type
Full-time
Level
Not Applicable
Location
San Francisco
Company
Perplexity

Industries

Software Development

Categories

Engineering Information Technology

Related Jobs

3 roles aligned with this opportunity

View all jobs
View Job Details
beqom
Related

DevOps Engineer

2026-06-18

Full-time
Not Applicable
Poland
Software Development
Engineering
View Job Details
Accenture
Related

Senior Python Engineer

2026-06-19

Full-time
Mid-Senior
Turkey
Software Development
Information Technology
View Job Details
TrustLab
Related

Full Stack Engineer (Python/React) - Ankara, TR

2026-06-19

Full-time
Not Applicable
Turkey
Software Development
Engineering