How to Measure the Success of Your Language Operations


Share on LinkedIn

In the last decade alone, we’ve seen a rise of operations-for-everything — from DevOps to marketing ops and revenue ops — powered by all types of technology that makes it easier to collaborate across teams. Above all else, technology (and the operations professionals who manage it) makes formerly difficult-to-measure processes consistent, measurable, and repeatable. Language is the next frontier.

For customer service leaders, applying AI and machine translation to language can make a major impact on key performance indicators (KPIs) like customer satisfaction (CSAT), net promoter score (NPS) and more. In addition, it can provide key insights on how to help agents become more efficient and effective at multilingual communications, and even unlock new revenue opportunities. Here are some of the top ways companies measure the success of their Language Operations, based on our 2021 Unbabel LangOps Survey conducted with global enterprise decision makers.

Translation quality remains important to customer success

Nearly half (46%) of our survey respondents said that translation quality would be the most important factor to consider for measuring Language Operations. But what does good or acceptable translation quality look like to organizations today? The top five indicators are:

  • Translated content correctly conveys the meaning of the original text (38%)
  • Proper communication of the company’s brand standards (37%)
  • Words and expressions resonate with the target audience (35%)
  • Correct usage of industry terminology (32%)
  • Follows guidelines for dates, addresses, and measurements (30%)

For companies that provide multilingual customer support, a range of factors such as accuracy, fluency, and style affect translation quality and the ability to convey ideas on the first try. Even human translators may be less proficient at communicating specific cultural nuances of one language versus another. When introducing a new support team, translation tool, or BPO provider, be sure to monitor that each has a positive or, at the very least, neutral impact on your First Contact Resolution rate (FCR).

Many organizations’ customer service efforts rely extensively on native-language speakers, which may not make financial sense as customer demands fluctuate. Others try to make do with basic or free online translation tools, which have major accuracy problems and lack the contextual training of machine translation solutions.

To contrast, a human-in-the-loop AI approach can help organizations achieve high quality at a fraction of the cost of native speakers. This enables near real-time translation and quality estimation, supported by human editors that ensure accuracy and help train their language translation engine over time.

Operational and customer-centric KPIs matter

When asked what ROI metrics or KPI improvements would be most valuable in determining whether to operationalize language, a third of our survey audience would like to see an increase in customer retention and/or an increase in customer satisfaction.

These tie into operational metrics such as:

  • Average response time
  • FCR
  • Cost per contact

In addition, they impact customer-centric metrics including:

  • Customer satisfaction score (CSAT)
  • Net promoter score (NPS)
  • Customer effort score (CES)

Let’s say a company with fluctuating seasonal demand notices that customers in Mexico are happy with their Spanish language support, yet customers in Spain are not. Examining these metrics could determine the reason for the disparity (i.e. using a more casual tone that one cohort views as friendly but another thinks is rude). From there, the customer service team can make the necessary adjustments.

Can language actually drive revenue?

Operationalizing language can help organizations quickly grow into new markets regardless of the native language speaking abilities of the team. Entering a new market and increasing demand in non-English speaking countries are just a few of the top priorities organizations selected when budgeting for language technology.

Many organizations measure operational KPIs like language agility (think metrics like first response times, ticket responses per hour per language, and average agent onboarding time) to make technology-based Language Operations a value-driver, rather than a cost center. Once customer service leaders can see how efficient (or inefficient) their agents are, they can make improvements and provide further training or tools to help streamline their workloads.

In the future, operationalizing language will empower universal understanding and business growth – without borders.

Vasco Pedro
Vasco Pedro is a co-founder and chief executive officer of Unbabel, a company that removes language barriers by blending artificial intelligence with real time, human translations. A serial entrepreneur, Vasco has led Unbabel since 2013, taking it through Y Combinator and raising a total of $31 million in funding.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here