Senior Data Search Engineer (Data Scientist)

About The Role & Team

IDC is building the next generation of AI-powered intelligence platforms that transform how technology decisions get made. Our platform re-imagines the way decision-makers discover and interact with trusted research and data, and is foundational to IDC's future. We are looking for a Data/Search Engineer to join the Search & Data Services team. This role sits at the intersection of data engineering and information retrieval. You will automate and extend the pipelines that feed IDC's content into the platform, build embedding pipelines for semantic search, and develop the document processing infrastructure that powers customer uploads. Your work ensures that when users ask questions, the system finds the right content and returns it accurately. This is a foundational role. The quality of every AI-generated response depends on the data infrastructure you build and maintain.

What You’ll Do

• Automate and extend ETL pipelines that ingest, transform, and deliver IDC's research content to the platform
• Build embedding pipelines that power semantic search across the platform's content library
• Develop document processing and extraction capabilities for customer uploads, including text extraction, chunking, and citation anchor generation
• Improve search relevance through indexing strategies, ranking tuning, and retrieval optimization
• Build data infrastructure that supports AI agent queries with accurate, well-structured results
• Monitor pipeline performance and data quality, identifying and resolving issues before they affect users
  
  

What You Bring

• 3+ years of experience in data engineering, search engineering, or a related field
• Experience building and maintaining ETL pipelines and working with data warehouses
• Familiarity with semantic search concepts and embedding-based retrieval
• Proficiency in Python and SQL
• Experience with document processing, text extraction, or content transformation
• A strong data quality mindset. You care about correctness, completeness, and consistency
  
  

Preferred Qualifications

• Experience with Snowflake or Snowflake Cortex
• Hands-on experience with vector databases
• Background in information retrieval or natural language processing
• Experience implementing access control at the data layer
• Familiarity with data quality monitoring and pipeline observability tools
  
  

Why This Role Stands Out

At IDC, your work helps shape how the world understands technology and where it goes next. You collaborate with curious, high-caliber colleagues who value rigor, integrity, and shared success. As the premier global provider of trusted technology intelligence, IDC equips business and technology leaders with the evidence they need to make confident decisions. Our insights inform strategy, investment, and innovation across industries and regions.

Recognized by IIAR as Analyst Firm of the Year for five consecutive years, IDC sets the standard for credibility and impact. With more than 1,000 analysts worldwide and a truly global perspective, we combine deep expertise with practical relevance. Here, your ideas matter, your voice is heard, and your contributions provide the insights leaders rely on every day. It is meaningful work, backed by a culture that supports growth, collaboration, and long-term career development with a globally respected brand.

What We Offer

• Hybrid/remote work model (about 1-2 days in the office per month).
• A position in a highly professional and globally respected market research and advisory firm, where initiative leading to results is rewarded.
• Individualized Culture: An environment where you can explore new areas outside your specialty and stay engaged with work you enjoy.
  
  

Equal Opportunity Employer

IDC is committed to providing equal employment opportunities for all qualified persons. Employment eligibility verification required. We participate in E-Verify.