Detection and enrichment steps

Overview

Detection and enrichment steps read a message and add information to it. They never change the text of the message and they never stop it, so they are safe to add to any pipeline. They run together, after the guard steps and before the transformation steps.

What they find is attached to the message that is delivered, shown in the Explorer, and, where a step has a metadata field, saved on the conversation.

Language identification

What it does. Identifies the language of each message, so later steps and your destinations know which language the customer is writing in.

How it works. Every message is analysed to find the most likely language, out of the 89 listed below. Short messages such as "ok" or a single name are hard to place, so the step does not decide on one message alone. It works with momentum: a conversation starts in the default language, and each message adds evidence to the conversation's score. The detected language only switches once the evidence is confident. As a result a one-word reply in another language does not flip a Dutch conversation to English.

Settings.

Setting	Description
Default language	The language a conversation starts in, until detection has enough momentum to switch. Choose from English, Dutch, German, French, Spanish and Italian. Defaults to English.

What you will see. The Explorer shows the language detected for each message, for example Language: Dutch. The delivered message carries the detected language and how confident the detection was.

Good to know.

• Translation in auto-detect mode needs this step. Translation uses the detected language as the language to translate from.
• Language identification is free when the pipeline also contains Translation.
• The default language is chosen from six languages. Detection itself covers every language in the list below.

Recognised languages

The step recognises these 89 languages:

Afrikaans, Albanian, Amharic, Arabic, Armenian, Assamese, Azerbaijani, Basque, Bengali, Bosnian, Bulgarian, Burmese, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Filipino, Finnish, French, Galician, Ganda, Georgian, German, Greek, Gujarati, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Inuktitut, Irish, Italian, Japanese, Javanese, Kannada, Kazakh, Khmer, Korean, Kurdish, Lao, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Marathi, Mongolian, Nepali, Norwegian, Odia, Pashto, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Sesotho, Sinhala, Slovak, Slovenian, Somali, Southern Ndebele, Spanish, Sundanese, Swahili, Swati, Swedish, Tamil, Telugu, Thai, Tsonga, Tswana, Turkish, Ukrainian, Urdu, Uzbek, Venda, Vietnamese, Welsh, Xhosa and Zulu.

Where a language is written in more than one script or variety, it is reported as one language: Serbian in Latin and Cyrillic, Norwegian Bokmål and Nynorsk, the Kurdish varieties, and the Arabic dialects. A language outside this list is not detected, so a message in one is matched to the closest language on the list.

Named entity extraction

What it does. Pulls specific details out of a message, such as an order number, a date, or a product, so you can use them without reading the message.

How it works. You define the entities you want, by name. The step then finds matching values in the message text. There is no list of patterns to maintain: you describe what you are looking for in words, and the step finds it however the customer words it.

Settings. Add one row for each entity:

Setting	Description
Entity name	What to look for, for example order number. Select a suggestion or type your own. Suggestions include order number, tracking number, product, brand, size, color, price, payment method, delivery method, store location, date, discount code and return reason.
Metadata field	Optional. The name of a metadata field to store the value in, for example order_number. It is filled in automatically from the entity name, and you can change it.

What you will see. The Explorer shows how many entities were found, for example Entities found: 2. Each entity is attached to the delivered message with its name, the text that was found and how confident the step was. When an entity has a metadata field, the value is saved there. If several values are found they are joined with a comma.

Good to know.

• Choose specific names. order number finds more reliable matches than number.
• The step extracts what is written in the message. It does not look anything up.
• Extraction runs before PII redaction, so it works on the message as the customer wrote it. A detail such as an order number or a name can be captured first, and the message that people read can still be redacted afterwards. For example, a support employee sees [FULL_NAME_0] in the message, while the extracted entity or metadata field holds the value your application needs.

Sentiment analysis

What it does. Classifies the tone of each message as positive, neutral or negative.

How it works. The whole message is read and given one label. Sentiment is a judgement about the message as a whole, not about individual products or topics in it.

Settings.

Setting	Description
Score context	Optional. The name of a metadata field to store the sentiment label in, for example score_context.

What you will see. The Explorer shows the sentiment of each message as Positive, Neutral or Negative. The sentiment is also attached to the delivered message and, when you set a score context, saved in that metadata field.

Good to know.

• Sentiment is scored per message. To follow the mood of a conversation, look at the messages in sequence.

CXP enrichment

What it does. Sends the information that this pipeline extracts, the entities and the sentiment, to the CXP.

Why use it. A message on its own tells an agent little. With this step the CXP receives structured data for every message, next to the conversation itself, so agents and anyone analysing conversations have the context without reading back through the chat:

• Which product a customer is talking about. An entity such as product or brand is available as a field, not only as words in a message.
• The right context for an order. An extracted order number is on hand when the conversation reaches an agent, so they can open the order instead of asking for it again.
• How the customer feels. The sentiment of each message makes frustrated customers easy to spot and to follow over time.

How it works. The step does not look at the message itself. It switches on the transfer of what Named entity extraction and Sentiment analysis found in the same pipeline. The values are sent for every message as it is processed.

Settings. None.

Good to know.

• It only has something to send when the pipeline also contains Named entity extraction or Sentiment analysis.
• It does not change the message, and it is not billed.