Graduate Conversational AI QA

Description

IG Group is a FTSE 100 fintech operating across five continents, serving over 1.4m customers and handling billions of dollars in transactions – built on scale, trust, and

Desirable

• An internship, placement, or part-time role in customer service, compliance, data, or a regulated environment such as financial services.

• Hands-on experience with AI or large language model (LLM) tools, for example through a dissertation, university project, hackathon, or personal project.

• A second language. While not a requirement, the ability to evaluate bot conversations in languages other than English would be a real asset given our global client base.

• Some exposure to data analysis or basic coding (Excel, SQL, or Python) - useful for spotting patterns across large volumes of conversations, but not essential.

• An understanding of research methods, experiment design, or how to measure quality or accuracy in a structured way.

Essential

• A degree in any discipline. We are as interested in graduates from psychology, linguistics, law, or philosophy as from computer science, maths, or economics.

• A genuine interest in AI and how it behaves in the real world - you are the kind of person who has poked at a chatbot to see where it breaks and wondered why.

• Excellent written English with a strong grasp of spelling, punctuation and grammar (SPAG). You will be judging the clarity, accuracy, and tone of the bot's written responses, so your own writing needs to be sound.

• A genuinely customer-centric mindset: you can tell the difference between a conversation that technically answered the question and one that actually served the client well.

• Strong analytical and problem-solving skills, with the curiosity to dig into why the bot responded the way it did rather than simply recording that it got it wrong.

• Attention to detail and consistency. The credibility of QA rests on evaluations being fair, accurate, and defensible.The confidence to share honest, constructive views - or the drive to build that confidence quickly with support.

• An interest in financial services and why regulation matters. You do not need prior knowledge of Consumer Duty or conduct rules - we will teach you - but you should want to understand why they exist.

The perks:

• Competitive salary

• Flexible Benefits Package on top of your salary (12%)

• Private medical cover for you and your family

• Life insurance

• Contribution to gym memberships

• 25 Days holiday, with 1 additional day off to celebrate your Birthday & 2 additional days off a year for voluntary work (28 in total)

• The option to buy or sell holiday days

• A comprehensive global and local onboarding process

• Enhanced primary (maternity), secondary (paternity), and shared parental pay and leave, as well as a range of support and benefits for parents

What's in It for You:

• A front-row seat in conversational AI at a FTSE 100 FinTech, with real responsibility from the start.Structured training in QA, financial services regulation, and AI evaluation, plus a dedicated mentor.

• Exposure to wider AI projects and cross-functional teams, and a foundation to grow into QA, AI quality, conversation design, or product roles as your interests develop.

Your First Few Months:

• You cannot fairly judge work you have never done yourself, so you will start by going through our CX agent training. You will learn our products, systems, and processes from the agent's seat and get a real feel for what a good client interaction looks like in practice. You will then spend time across the Conversational AI team to understand how the bot is built, what it is and is not allowed to do, and how changes to it are made. Throughout, you will have a dedicated mentor in the team and structured training on financial services regulation and Consumer Duty. From there, you will ramp into the role itself:

• First, getting to know our bot evaluation framework and tooling inside out, and completing evaluations under guidance until you are scoring fairly and consistently.

• Then, carrying out root cause analysis on failed conversations independently, contributing to test scenarios, and presenting your findings to the wider team and our CX colleagues.

• In time, taking on a role in wider AI projects, spotting the patterns and conduct risks others miss, and helping shape how the bot is improved and what gets tested next.

Feedback that drives improvement

• Share clear, evidence-based findings that the rest of the Conversational AI team and our product and knowledge colleagues can actually use - with example transcripts, the standard that was missed, and what good would have looked like.Learn to stand behind your assessments.

• Not everyone will agree that a response was a failure, and you will build the confidence to make your case with evidence - and the judgement to concede when the evidence points the other way.

• Connect what you see in evaluations to the bigger picture, feeding insight into content updates, conversation design, guardrails, and release decisions so that the bot's quality improves over time, not just gets measured.undefined

Getting involved in wider AI projects

• Get hands-on with AI improvement work beyond day-to-day QA - for example testing new bot features, knowledge content, or model updates before they reach clients.

• Contribute to how we measure AI quality: helping build evaluation datasets, refine scoring criteria, and explore how automated evaluation can work alongside human review.

• Work closely with colleagues across the Conversational AI team and with product, data, and compliance, seeing first-hand how a regulated business builds and governs AI - experience that opens doors across the business.

Testing, investigation and root cause

• Carry out root cause analysis on failed conversations, DSATs, and other negative outcomes, working out whether the problem sits in intent recognition, the knowledge base, conversation design, the model's behaviour, or the handoff process.

• Look for patterns across conversation types, intents, and topics rather than treating every failure as a one-off, and surface them clearly.

• Help design and run test scenarios - including edge cases and adversarial prompts - to probe how the bot behaves before and after changes, and check that fixes work without breaking something else.undefined

Regulation and Consumer Duty at the centre

• Treat bot QA as what it is: a core line of defence for regulatory compliance, not a box-ticking exercise. The same standards apply whether a client is served by a person or a machine.

• Identify conduct risk in bot responses - anything that strays into investment advice or a personal recommendation, unbalanced representation of risk and reward, inaccurate or invented information about products, fees, or margin, market abuse indicators, and mishandling of personal data or DSAR requests - and flag it correctly.

• Pay particular attention to Consumer Duty obligations, especially whether the bot recognises expressions of dissatisfaction as complaints, picks up signs of client vulnerability, and routes those clients to a human with appropriate care.

Quality evaluations of AI assistant conversations

• Review client conversations with our AI assistant across chat and other digital channels, scoring them fairly and consistently against a bot-specific evaluation framework.

• Assess the things that matter most: whether the bot understood what the client was actually asking, whether its answer was accurate and grounded in approved content, whether it resolved the query or routed it appropriately, the clarity and tone of its responses, and - critically - whether the conversation met our conduct and regulatory standards.

• Scrutinise handoffs to human agents: whether the bot escalated when it should have, avoided escalating when it did not need to, and passed the client over with enough context that they did not have to start again.

• Use sampling, AI-assisted scoring, and conversation analytics to focus your review where it adds the most value, applying our quality standards consistently and in line with how human interactions are assessed.