(Internal Recruitment) - AI Quality Analyst, Helpshift Care AI

This opportunity is available across all Player Engagement locations and is intended for internal hiring only.

French translation of the job description will be available soon.

Role Overview

As Keywords continues to expand Helpshift Care AI across Player Engagement, quality remains a critical factor in successful AI adoption.

As an AI Quality Analyst, you will help ensure Care AI performs accurately, safely, and in alignment with each client’s tone, standards, and player experience expectations. Working closely with AI Ops Managers, Platform Leads, Product, and Development teams, you will review AI performance, identify quality risks, and contribute to continuous improvement initiatives that help scale AI responsibly.

This role is ideal for someone who is detail-oriented, analytical, and passionate about quality, player experience, and the future of AI-powered support.

What are we looking for?

Communication

·        Review AI agent interactions to ensure responses are accurate, helpful, and aligned with client tone and communication standards.

·        Document quality findings clearly and provide actionable feedback to AI Ops, Product, and Development teams.

·        Support calibration sessions and quality discussions to help maintain consistency across reviews.

·        Help communicate quality risks, trends, and improvement opportunities to internal stakeholders.

Analytics

·        Conduct quality audits of AI agent conversations across live client accounts.

·        Identify recurring issues such as incorrect responses, content gaps, escalation failures, tone mismatches, and hallucination risks.

·        Analyze quality trends and contribute to root cause investigations.

·        Support reporting efforts by tracking audit results and improvement progress.

Business

·        Assist in validating AI performance against support workflows, player experience expectations, and operational requirements.

·        Contribute feedback that supports content improvements, workflow enhancements, and product optimization.

·        Support quality reviews of new AI features and capabilities before wider adoption.

·        Help maintain quality documentation, audit standards, and review processes.

Leadership

·        Demonstrate ownership in identifying and escalating quality concerns.

·        Participate in continuous improvement initiatives that support AI quality and client success.

·        Contribute to quality best practices and knowledge-sharing activities across teams.

·        Help protect player trust by ensuring quality standards are consistently applied.

Requirements

You'd be a great fit for this role if you have:

·        Experience in Quality Assurance, Customer Support, Player Support, AI Operations, Knowledge Management, or similar operational environments.

·        Strong attention to detail and the ability to assess interactions fairly and consistently.

·        Understanding of support quality principles, player experience, client tone, and escalation behavior.

·        Experience identifying quality issues, process gaps, and improvement opportunities.

·        Familiarity with root cause analysis, audit processes, QA scorecards, or continuous improvement methodologies.

·        Strong written and verbal communication skills.

·        Analytical mindset with the ability to identify patterns and interpret quality data.

·        Ability to work collaboratively with Operations, Product, Development, and client-facing teams.

·        Interest in AI, automation, gaming support, and customer experience.

Nice-to-haves

·        Experience reviewing chatbot, AI assistant, or automation performance.

·        Familiarity with Helpshift, Zendesk, or other customer support platforms.

·        Experience in gaming, player support, or digital customer service environments.

·        Exposure to AI-powered support solutions or conversational AI technologies.

·        Passion for gaming and emerging AI technologies.

What Success Looks Like

·        AI interactions are reviewed consistently and quality risks are identified early.

·        Client tone and player experience standards are maintained across AI deployments.

·        Actionable quality insights help improve AI performance over time.

·        Product and Development teams receive meaningful feedback from live environments.

·        Quality findings contribute to stronger client outcomes and greater confidence in Care AI.

·        Keywords continues to scale AI adoption responsibly while maintaining trust, quality, and a positive player experience.

Benefits

Based on location-specific benefits.