In regulated debt collections, traditional chatbot KPIs like containment and deflection miss the real risk. This article introduces Vulnerable Time-to-Human (V-TTH): the KPI that measures how quickly high-risk customers reach a human agent.
If a chatbot keeps a vulnerable, disputing, or distressed customer inside automation for too long, that is not efficiency. It is a product risk. In regulated collections, the KPI that matters most is not deflection. It is how fast the customer reaches the right human agent.Most chatbot dashboards still begin with the same familiar measures: containment, completion, average handling time, self-service success, and deflection. Those metrics are useful in low-risk journeys. In regulated collections, they are incomplete. Sometimes they are actively misleading. A bot can look efficient on paper while doing exactly the wrong thing in practice: keeping a customer inside automation when the conversation has already become contested, vulnerable, or high-risk.
The gap is clear: the industry has spent too much time asking how to make the bot smarter, and not enough time asking how to make failure safe. In collections, the highest-risk moments are rarely the routine ones. They are the moments where a customer disputes a balance, says they are already working with an adviser, discloses hardship, struggles to understand a regulated statement, or reveals circumstances that make them more vulnerable. The safest product stance is to assume that these moments will appear, sometimes unexpectedly, mid-journey.[1][3]
The UK gives us some of the clearest operational examples. The FCA defines a vulnerable customer as someone who, because of personal circumstances, is especially susceptible to harm when a firm is not acting with appropriate levels of care.[1] It expects firms to understand those needs, design for them, and monitor the outcomes customers actually receive.[1][2] In practice, that means vulnerability is not an edge case. It is a design condition.
That same design logic extends well beyond the UK. The EU AI Act emphasises effective human oversight for higher-risk systems.[6] The ASEAN Guide on AI Governance and Ethics asks organisations to determine the right level of human involvement in AI-augmented decision-making and to assign clear internal responsibilities.[7] Saudi rules on handling objections require creditors not to remind consumers of defaults while an objection is being resolved.[8] In the Philippines, the BSP requires fair treatment and effective recourse and notes that accountability for AI-driven decision-making still rests with humans.[10][11] Different markets use different language. The product lesson is remarkably consistent: higher-risk conversations need faster human intervention, not deeper automation.
Collections is not just another customer service use case. It sits close to a customer’s finances, credit standing, stress level, and legal rights. Because of that, the cost of getting the wrong answer is higher. A confused answer can become a complaint. A missed dispute can become a recoveries error. A poorly handled hardship disclosure can become a conduct issue. A chatbot that continues pushing for payment after a formal protection or objection may create avoidable harm in a matter of minutes.
That is why the best product question is not simply, ‘Can the bot answer this?’ It is, ‘Should the bot continue at all?’ When the answer is no, the experience must move quickly into a human lane. The product problem is not to eliminate every failure. It is to make failure safe, measurable, and recoverable. In collections, recovery often means reaching a trained human who can apply judgement, explain the next step clearly, and stop the automation from doing further damage.
Traditional chatbot KPIs optimise for automation. Regulated collections needs a different lens. The table below shows why.
|
Common KPI |
Why teams track it |
What it misses in collections |
Safer companion measure |
|
Containment rate |
How often the bot kept self-service |
Can hide risky journeys that were not escalated |
Stop-moment capture and V-TTH p95 |
|
Deflection rate |
Reduction in calls or agent workload |
Treats human help as failure even when it is the right outcome |
High-risk handoff SLA |
|
Completion rate |
Whether a task reached an end state |
A ticket or callback is not the same as real help |
Actual human connection rate |
|
Average bot handling time |
Speed and efficiency |
Can encourage rushed journeys and shallow diagnosis |
Time from trigger to safe resolution |
|
Promise-to-pay conversion |
Commercial effectiveness |
May reward pressure in journeys that should have exited automation |
Conversion after high-risk journeys are excluded |
Matrix 1. Traditional chatbot KPIs still have value, but they do not show whether the bot failed safely when a higher-risk customer needed human support.
The clearest reason to prioritise speed-to-human is that collections already contains several ‘stop moments’ in law, regulation, and supervisory expectation. In the FCA handbook, disputed debt is one of the most obvious examples. When a customer disputes a debt on valid grounds, or on grounds that may be valid, firms must suspend recovery steps and investigate.[3] The same rules require firms to provide information on the outcome of that investigation, and they make clear that establishing identity and the amount owed is a firm responsibility, not something pushed back on to the customer.[3]
The same sourcebook also points to other moments where the automated path should not simply carry on. If a customer is seeking help from a debt counsellor, firms are expected to suspend active pursuit for a reasonable period.[3] If mental capacity limitations are raised or ought reasonably to be apparent, pursuit of recovery should be suspended.[3] UK Breathing Space guidance is even more direct: if a creditor is told that a debt is in a breathing space, it must stop all action related to that debt and apply the protections.[4]
This is not only a UK story. In Saudi Arabia, the Saudi Central Bank’s rules on handling objections require creditors to document the complaint electronically, give the consumer the expected time frame for resolution, support the outcome with documents, and crucially, not communicate with the consumer or guarantor to remind them of defaults until the complaint is resolved.[8] Saudi rules on customer care go further by requiring multiple complaint channels, complaint tracking, visible status updates, defined time frames, and formal service-level indicators for handling quality.[9]
In Southeast Asia, the governance signal is similar. The ASEAN Guide on AI Governance and Ethics says organisations should determine the level of human involvement in AI-augmented decision-making according to the risk of the use case.[7] The BSP’s financial consumer protection framework requires fair treatment and accessible, timely, and efficient complaint resolution mechanisms, including the use of technological innovations where appropriate.[10] Its 2024 thematic review on AI and machine learning in financial services states that gaps such as accuracy, hallucination, data quality, and ethical issues still need to be addressed, and that accountability for decision-making lies not on AI systems but on humans.[11]
|
Stop moment |
What the customer may say |
Minimum safe bot behaviour |
Supporting frameworks |
|
Dispute or objection |
This is not my debt. The balance is wrong. I want to challenge this. |
Stop the collections script, acknowledge the issue, record the objection, and route to a human reviewer. |
FCA CONC[3], SAMA Article 6[8] |
|
Hardship or debt advice |
I am speaking to an adviser. I cannot afford essentials. I need help, not pressure. |
Pause automated pressure and move the customer into a specialist support path or confirmed callback. |
FCA CONC[3] |
|
Vulnerability or capacity |
I am unwell. My carer handles this. I do not understand what you are asking. |
Slow the journey down, avoid pressure language, and transfer to a trained human. |
FCA FG21/1[1], BSP fair treatment[10] |
|
Formal protection |
I am in Breathing Space. This account is protected. |
Stop action related to the debt and route to a specialist handler. |
GOV.UK Breathing Space[4] |
|
Complaint or review request |
I want this reviewed. I want to complain. I need a written explanation. |
Provide a clear route, expected timing, and status visibility. Do not bury the customer in a loop. |
SAMA customer care rules[9], BSP effective recourse[10] |
|
Confusion about a regulated statement |
I do not understand this notice, fee, or legal statement. |
Do not improvise. Use controlled language or escalate to a human who can explain the point correctly. |
EU AI Act human oversight[6], ASEAN transparency and explainability[7] |
Matrix 2. Different markets describe the trigger differently, but the safe product response is consistent: stop pressure, preserve context, and move into human handling.
From an AI product management perspective, this changes the objective. The goal is not to create a bot that can improvise through every sensitive path. The goal is to create a bot that knows when it should stop, and that can prove it stopped safely. That is a more mature way to think about AI in collections. It does not reject automation. It places automation inside a controlled operating model.
The EU AI Act’s language on human oversight is useful here, even for teams operating beyond the EU. It frames oversight as a practical design issue: humans must be able to oversee operation and intervene when needed.[6] The ASEAN Guide makes the same point in a business-friendly way by urging organisations to decide how much human involvement a given AI use case requires and to assign responsibilities clearly.[7] The message for AI PMs is simple: human handoff should not be treated as a fallback of last resort. In higher-risk collections journeys, it is part of the product design.
Figure 1. Vulnerable Time-to-Human (V-TTH): A bot should be measured from the moment risk appears, not from the moment a ticket is created.
The headline KPI proposed in this article is Vulnerable Time-to-Human, or V-TTH. It measures the elapsed time between the first high-risk trigger in a conversation and the moment a qualified human becomes available to the customer. That human might join the chat, answer the phone, or confirm a scheduled callback with a real service commitment. The crucial point is that this is not a ‘handoff requested’ metric. It is a ‘human actually available’ metric.
The trigger can be user-declared, system-detected, or both. In practice it usually includes disputes, objections, hardship disclosures, vulnerability indicators, formal complaint language, formal protections, and confusion around regulated statements. The KPI should then be reported with percentiles, not only averages. In this domain, the tail matters. A good average can still hide a very poor experience for the customers who most needed help.
|
Measure |
What it tells you |
|
V-TTH median and p95 |
How quickly high-risk customers reach a human, including tail risk. |
|
High-risk handoff SLA attainment |
Whether the team met its promised connection time for sensitive journeys. |
|
Stop-moment capture rate |
How often the system correctly recognises when automation should end. |
|
High-risk abandonment rate |
How many customers drop before human contact after a sensitive disclosure or objection. |
|
Repeat-contact rate after high-risk disclosure |
Whether customers had to restate the same dispute, hardship, or vulnerability more than once. |
KPI stack. The primary KPI is V-TTH, but it should sit alongside measures that show whether the handoff was real, timely, and repeat-free.
A failure-safe collections bot follows a clear pattern. First, it surfaces a visible human option early enough that the customer does not feel trapped. Second, it uses conservative policy triggers so that disputes, objections, hardship, vulnerability, and complaint language exit the automated pressure path quickly. Third, it preserves context, so the customer does not need to repeat sensitive information. Fourth, it gives the customer a credible next step rather than a dead end.
That final point matters more than many teams realise. A created case is not a safe outcome if the customer has no clarity on what happens next. SAMA’s customer care rules are helpful here because they turn complaint handling into something operational: multiple channels, visible status, timed handling, and measurable service levels.[9] BSP language on effective recourse points in the same direction, requiring complaint resolution mechanisms to be accessible, timely, fair, and efficient.[10] AI PMs should treat those as product design requirements, not only compliance notes.
Figure 2. Failure-safe escalation flow: In a mature product, the system does not debate whether a sensitive journey deserves human help. It stops, preserves context, and routes the customer safely.
This is also where the EXUS perspective matters. EXUS already speaks publicly about responsible AI in collections in terms of transparency, compliance, and human oversight.[12] The next practical step is to connect those principles to product measurement. For a solution such as EXUS AI Negotiator, the question should not be only how effectively the system negotiates routine journeys. It should also be how reliably it exits the automation lane when a journey becomes contested, vulnerable, or regulated.
That framing is commercially useful as well as ethically sound. It gives clients a governance model they can take into deployment reviews, risk committees, and operational scorecards. It also makes the value story clearer: AI Negotiator is not only about efficiency. It is about controlled efficiency, where performance is measured alongside customer protection and conduct discipline.
In practical terms, that means the best-practice KPI framework around AI Negotiator should include visible human options, policy-led stop triggers, context-preserving handoffs, market-specific service-level targets, and management information that surfaces exceptions quickly. That is the difference between a product that looks impressive in a demo and a product that can be trusted in a live regulated environment.
Good AI governance becomes real when it appears on a dashboard that product, operations, risk, and compliance can all use. The table below is a simple example of how that dashboard can be structured.
|
Dashboard measure |
Question it answers |
Primary owner |
Review cadence |
|
V-TTH p95 |
How long do higher-risk customers wait in the worst common cases? |
Product and Operations |
Weekly |
|
High-risk handoff SLA attainment |
Did we connect people within the time we promised? |
Operations |
Daily / weekly |
|
Stop-moment capture rate |
Did the system recognise that automation should stop? |
Product and Risk |
Monthly |
|
High-risk abandonment rate |
Are customers dropping out after they reveal risk or ask for help? |
Operations and QA |
Weekly |
|
Repeat-contact rate |
Did customers have to restate the same problem more than once? |
Complaints / Customer Office |
Weekly |
Governance view. The aim is not to produce more dashboard noise. It is to make safe failure visible and reviewable.
The easiest way to misunderstand AI in collections is to think the challenge is purely one of model intelligence. It is not. The more important challenge is product design under risk. A collections chatbot must know when to stop, when to explain carefully, and when to hand a customer to a human. The product team must then prove that this happens quickly enough, consistently enough, and visibly enough to be trusted.
That is why the real collections chatbot KPI is not ‘How many conversations did we keep away from agents?’ It is ‘How fast did a vulnerable or contesting customer reach the right human?’ If AI PMs start there, they will build better products, stronger governance, and safer automation. And if we want AI in collections to scale with credibility, that is the number we should be prepared to show. For EXUS, it is also the right benchmark to embed into AI Negotiator as the product moves from promise to trusted practice.
Talk to an EXUS expert to design AI collections journeys that balance efficiency with customer protection and prove it with the right KPIs.
This article shares product and governance viewpoints, not legal advice. Market-specific implementation should always be reviewed with legal and compliance teams.
The author would like to thank Dimitris Papadopoulos, Kashyap Raiyani, Davide Mastricci, and Panagiotis Tassias from the AI team for their valuable technical contributions to this article.
[1] Financial Conduct Authority, FG21/1: Guidance for firms on the fair treatment of vulnerable customers.
https://www.fca.org.uk/publication/finalised-guidance/fg21-1.pdf
[2] Financial Conduct Authority, Firms' treatment of customers in vulnerable circumstances – review.
https://www.fca.org.uk/publications/multi-firm-reviews/firms-treatment-vulnerable-customers
[3] Financial Conduct Authority, Consumer Credit sourcebook (CONC 7: arrears, default and recovery).
https://handbook.fca.org.uk/handbook/conc7
[4] GOV.UK, Debt Respite Scheme (Breathing Space) guidance for creditors and creditors’ responsibilities.
https://www.gov.uk/government/publications/debt-respite-scheme-breathing-space-guidance/debt-respite-scheme-breathing-space-guidance-for-creditors
[5] Financial Conduct Authority, Debt purchasers, debt collectors and debt administrators portfolio letter.
https://www.fca.org.uk/publication/portfolio-letters/debt-purchasers-collectors-administrators-portfolio-letter.pdf
[6] Regulation (EU) 2024/1689, Article 14: Human oversight.
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689
[7] ASEAN, ASEAN Guide on AI Governance and Ethics.
https://asean.org/wp-content/uploads/2024/02/ASEAN-Guide-on-AI-Governance-and-Ethics_beautified_201223_v2.pdf
[8] Saudi Central Bank Rulebook, Article 6: Handling Objections.
https://www.rulebook.sama.gov.sa/en/article-6-handling-objections
[9] Saudi Central Bank Rulebook, Regulations for Establishing Customer Care Departments in Banks.
https://www.rulebook.sama.gov.sa/en/entiresection/9610
[10] Bangko Sentral ng Pilipinas, Financial Consumer Protection Framework.
https://www.bsp.gov.ph/Pages/InclusiveFinance/FinancialConsumerProtectionNetwork.aspx
[11] Bangko Sentral ng Pilipinas, Thematic Review on the Use of AI and ML in Financial Services.
https://www.bsp.gov.ph/Media_And_Research/Special%20Publications/BSP_Thematic_Review_on_the_Use_of_AI_and_ML_in_Financial_Services.pdf
[12] EXUS, AI-powered debt recovery (trends, challenges & opportunities).
https://www.exus.co.uk/blog/ai-debt-collections-trends-challenges-applications