WeLocalize Logo

WeLocalize

Search Relevance SME - Digital Content

Job Posted 5 Days Ago Posted 5 Days Ago
Be an Early Applicant
Remote
8 Locations
Junior
Remote
8 Locations
Junior
As a Search Relevance SME, you will evaluate data submissions, monitor quality, provide feedback to raters, and recommend training sessions based on error trends.
The summary above was generated by AI

Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant, diverse, and scalable to supercharge their AI models. As a Welocalize brand, WeloData leverages over 25 years of experience in partnering with the world’s most innovative companies and brings together a curated global community of over 500,000 AI training and domain experts to offer services that span:


ANNOTATION & LABELLING: Transcription, summarization, image and video classification and labeling.

ENHANCING LLMs: Prompt engineering, SFT, RLHF, red teaming and adversarial model training, model output ranking.

DATA COLLECTION & GENERATION: From institutional languages to remote field audio collection.

RELEVANCE & INTENT: Culturally nuanced and aware, ranking, relevance, and evaluation to train models for search, ads, and LLM output.


Want to join our Welo Data team? We bring practical, applied AI expertise to projects. We have both strong academic experience and a deep working knowledge of state-of-the-art AI tools, frameworks, and best practices. Help us elevate our clients' Data at Welo Data.


Role Overview


We are seeking a Search Relevance SME to join our team. In this role, you will act as a Subject Matter Expert for a high-profile streaming platform. Working closely with our raters, you will evaluate submissions in Label Studio, monitor data quality, and recommend targeted training interventions. By identifying recurring errors and offering actionable feedback, you will ensure our team consistently meets and exceeds client expectations.

Key Responsibilities

  • Weekly Data Analysis & Error Trend Identification
  • - Review rating submissions to spot recurring mistakes, patterns, and quality concerns.

    - Compile and present weekly error trend reports to stakeholders.


  • Quality Assurance & Guideline Adherence
  • - Evaluate/correct data annotations for accuracy and compliance with the client’s rating guidelines.

    - Collaborate with team members to clarify and update guidelines as needed.


  • Provide Expert Guidance & Feedback
  • - Serve as the primary point of contact for rater questions about best practices and rating methodologies.

    - Offer actionable feedback and mentor raters to improve performance.


  • Training Recommendations
  • - Design and/or recommend targeted training sessions based on error trends and gaps in team knowledge.

    - Work with our internal Quality Team to develop strategies that address recurring mistakes or misconceptions.


  • Collaboration & Reporting
  • - Interface regularly with cross-functional teams (Project Managers, Quality Managers, Program Leads, etc.) to report on performance, progress, and key insights.

    - Flag low performers and specific challenging cases, ensuring timely resolution and support.

Qualifications

  • Previous QA experience: Prior exposure to any kind of QA is highly desirable.
  • Strong Analytical & Communication Skills: Ability to synthesize error trends and present clear, actionable insights. Comfortable mentoring and guiding larger teams, both verbally and in writing.
  • Localization Experience: Understanding linguistic and cultural nuances in data annotation is a plus.
  • Availability for 10–20 hours per week: Flexible schedule with some overlap in working hours for communication with global teams.
  • Self-Motivated & Detail-Oriented: Capable of working independently, meeting deadlines, and maintaining strict quality standards in a remote setting.

Top Skills

Label Studio

Similar Jobs

11 Hours Ago
Remote
Bengaluru, Karnataka, IND
Senior level
Senior level
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
As a Senior Data Engineer, you'll manage a large data lake, build reliable data services, and transform vague requirements into solid solutions, emphasizing data quality and modern software practices.
Top Skills: AirflowAWSEmrHiveJavaKinesisPythonRdsS3ScalaSparkSQLSqs
Yesterday
Remote
Hybrid
Pune, Maharashtra, IND
Senior level
Senior level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Senior Data Engineer is responsible for developing data pipelines, improving data models, and integrating new data management technologies while consulting on complex projects.
Top Skills: AirflowAnsibleChefHadoopHiveKafkaProtobuf RpcPythonScalaSparkSQLTerraform
3 Days Ago
Remote
Hybrid
2 Locations
Mid level
Mid level
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
As a Senior Associate in Data Science, you will utilize data science and AI technologies to optimize manufacturing processes and drive innovation through collaboration and implementation of advanced solutions.
Top Skills: AWSMatplotlibNumpyPandasPlotlyPythonPyTorchScikit-LearnSeabornShapSQLTensorFlow

What you need to know about the Toronto Tech Scene

Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account