Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
By Alyssa Levitz. Updated: Dec. 22, 2020
One common channel for customer support is some form of synchronous online "chat." The National Association of State CIOs published a document about the broader use of chat by states during the pandemic; this document dives specifically into chats set up to support UI agencies: what types of chat do they use, and what are some of the best practices we see?
The target audience of this report is primarily state UI agencies that are looking to add or revamp their online chat offerings. Much of what is discussed here, though, is applicable to many other government use cases.
Prompted chat in which users are guided through a series of pre-defined questions and answers is likely to be most effective, but whatever chat type you use: make sure to set users' expectations up front.
Accessibility is a key consideration for chat experiences and should be incorporated from the beginning of the process.
Use data to inform the initial set up of your chat experience, and make sure to have data collection in place to iterate and improve over time.
Your system should handle 15-20 questions or scenarios very well and have clear paths for people to find more information on other topics.
"Prompted" chat is when people are presented with a series of options for what they can get help with. The options are frequently structured as a decision tree, with sub-questions for the primary scenarios. This chat seems to always come with f-text chat, so you can get all the pros (and cons) of that, though the intended primary interaction is for the user to click buttons within the chat window rather than typing anything into the chat box.
Pros:
You can configure the flow to answer the most common questions
People have clear expectations of what they can get help with
Content is created specifically with the chat format in mind
Cons:
Without careful set up, it is very easy to create "dead ends" for people seeking help.
Vendors:
AWS/Accenture
IBM
Microsoft
Salesforce
Twilio
We looked at each state's UI agency government website, and in some cases non-governmental but official websites that people would be directed to in order to learn about or apply for or manage unemployment claims. In many cases, we took screenshots for easy reference for myself; this also has the effect of documenting some changes over time.
Once a chat functionality was found, we used the Chrome browser's Developer Tools to find evidence of which vendor was used to provide it. With this method, we were unable to determine the vendors used in Kansas, Massachusetts, and Maryland. Vendors may have more chat-related offerings than are mentioned here; this document is not intended to be a comprehensive vendor analysis.
We have only been able to find results about chat’s effectiveness at reducing call volume for states that have implemented “prompted” chat systems, and from only two of the vendors.
Colorado (Google Chat Bot): “the virtual agent solves user questions about 90% of the time”
Illinois (Google Chat Bot): “the web chatbot interacts with upwards of 100,000 constituents a day.” Google Chat Bot
New Jersey Google Chat Bot): “In its first 3 days of operation, the chat feature engaged with approximately 50,000 user interactions, freeing up time for agents to focus on claims that need intervention rather than answering frequently-asked questions.” Google Chat Bot.
Texas (AWS & Accenture): Over several months, answered Qs from 2.3 million people and save their staff from answering hundreds of thousands of calls
While all this data is promising, it leaves out how many people were confused by the chat system or just otherwise didn’t get their questions answered. That said, if your chat system is set up to help people get answers to the most common questions you see coming in through higher-cost support channels, chances are that you are well set up to deflect those inbounds.
If you are interested in working with USDR on updating your customer support mechanisms, don't hesitate to reach out by filling out this form.
"Support" chat is when a chat-like interface is set up to help someone navigate available help articles that are likely seen elsewhere on the site, e.g., linked to from an FAQ page. When someone types into the message box, options will show up of articles that the user can click on; those articles are then either shown within the "chat" window or in a new tab. There may also be buttons within the "chat" that can also be clicked.
Pros:
People have clear expectations of what they can get help with
Repurpose existing content on website
Means that there are persistent links that anyone can use to provide someone with the most up-to-date answer on a given question
Cons:
All answers have enough content to warrant their own page, and so questions with shorter answers may not be handled as well
Vendors:
MedChat
ServiceNow
Zendesk
As it relates to UI agencies, online chat implementations fall into one of four categories, which I'm calling: Live, free-text, prompted, and support. Some states have multiple implementations on different parts of the site or at different times of day; others have multiple and let the customer choose which one suits their needs. For each type of chat, we will discuss its pros/cons as well as vendors seen to be offering that functionality to UI agencies as of December 14, 2020.
Links to how you can access each state's chat system, along with any notes like what vendor they used, can be found on the page where we're tracking solutions across many states:
Visual accessibility: There are many best practices when it comes to accessibility for the visually impaired. The chat tool should have high-contrast and work with screen readers at the very least. You should not choose vendors that don't meet these requirements -- or you should push your vendor to update their tool for ALL customers to begin meeting these requirements.
Translations: Based on the demographics of your state, there are likely one or two other languages that your site -- and all corresponding tooling like chatbots -- should be available and fully translated into. When it comes to chatbots, Rhode Island with Twilio gets top marks for availability in 5 languages (English, Spanish, Portuguese, Arabic, French).
This is much easier to do for Prompted and Support chat systems rather than free-text or live, because there is a known set of inputs from the customer (clicking on links or options) and outputs (pre-written responses). That doesn't mean multiple languages shouldn't be supported in Live and Free-text chat systems.
When choosing a vendor and building the contract, UI agencies should ask how the vendor will support them in providing help in multiple languages.
Mobile-friendly: For many people, their only way of accessing the internet is through their mobile phones, potentially with limited data plans. Chat integrations should be designed with this smaller screen in mind and be evaluated for how much data they use in the course of operating. If your integration is anything other than directly "out of the box," you should request support be made available to you for ensuring a great mobile experience.
Availability: You should make sure that the chat is available on any/all pages of your site (or sites!) that get meaningful traffic by people who might be looking for help with unemployment insurance. This would include any separate sites you might have for, e.g.: UI Agency administration, Standard UI/PEUC management, PUA management, re-employment services and support.
Specificity: In addition to wanting to put your chat content on multiple parts of your site or sites, you want to make sure that chat content from non-UI sites doesn't show up here. This shows up frequently in Support Chat implementations that draw on FAQ content from across the DOL. You should ask the vendor how to set up multiple instances of the chat
Data usage and speed: If the chat implementation is slow, it increases the odds of someone abandoning chat and turning to a different support channel. People are used to the internet being blazing fast and might find that more annoying and cumbersome than putting their phone on the table and doing other things while on hold. Part of chatbot speed is also related to how it is using data; this is also important as it relates to burdening people with limited data plans as little as possible.
Quality assurance:
It should be easy to test the chat before it becomes available on the website. This is especially true for live chats, to make sure that agents are comfortable with their side of the interface.
If the chat is available on multiple pages or sites, it should be easy to test it before it goes live on those different pages. Testing on multiple pages is important to ensure that the chat is always "on top" of the content of the rest of the page; you don't want page content hiding the chat
Availability: For your chat system to have the most impact, you should make sure that the chat is available on any/all pages of your site (or sites!) that get meaningful traffic by people who might be looking for help with unemployment insurance. This would include any separate sites you might have for:
UI Agency administration
Standard/PEUC UI management
PUA UI management
Re-employment services and support
Some design basics:
Don't call it "live" if it's a bot or virtual agent, either in the chat itself or in content that references it. This sets the wrong expectation for people and might lead them to be more likely to provide private information even if it is not appropriate for them to do so.
The first message from the system should be short and to the point. Make sure that it is fully visible within the interface without the customer having to scroll. This means that the chat's "header" should be relatively narrow, and that there aren't large images or large blocks of text.
If content is split into multiple messages, then leave some time in between message sends for the person to at least register that there are multiple messages, if not fully read one message before the next one gets sent
For prompted chats, make it easy for users to get back to the "main menu" so that people can get multiple questions answered or change their mind about an answer response.
When implementing a chat or chat-like experience for your UI agency, you need to prepare a list of likely questions and acceptable answers. This Q&A list is used to build the decision tree and responses for Prompted Chat, to prepare agents for Live chat, to prepare Free-text chat to understand a variety of inputs, and to provide answers for all those chat types as well as "support chat.
You should prepare your system to handle 15-20 questions or scenarios very well, and have clear paths for people to find more information on other topics. There are many kinds of data you can leverage to inform this initial Q&A set, for example:
From your IVR system: what branches are most commonly used?
From your call center: what are common topics of calls? What questions are they answering over and over again?
From your claims agents: what mistakes are they seeing people make, or misunderstandings people have?
All of this data is useful in other ways, too, as you look to make more fundamental improvements to your UI system. E.g., if people frequently ask about their claim status, you could consider prioritizing work to make that possible online (or if it's already possible, to move it to a more expected location or change the language around it to make it clearer).
Regardless of the type of chat, you should make sure to have metrics in place to track the effectiveness of the tool and make changes moving forward. As economic conditions, policies, and your benefits website change, the chat experience needs to be kept up-to-date.
Effectiveness measures:
CSAT for tracking people's satisfaction with the response they got
After the chat thinks it has done its job, you can have it ask, "Did that answer your question?" For making changes, you should keep an eye in particular on the sequence of back-and-forths that lead up to someone's question not being answered - though don't forget to celebrate all the times that it did answer someone's question!
"Engagement" metrics: e.g., button clicks, messages sent, clicks on outbound links
Time spent: for chats other than live chat, how much time are people spending interacting with the tool?
Language analysis: for chats that let users input any text they want - what are they asking for that the system doesn't understand?
"Free-text" chat is when a user types their own message. The system then uses Natural Language Processing to try to interpret the message in order to provide the most relevant pre-written response. This is what people most commonly think of when they hear about "chatbots."
Pros:
People can submit any question
Cons:
People may get more frustrated at the ability to ask any question but not be able to get all questions answered.
The system may not be able to interpret what someone is saying
Vendors:
Astute Chatbot
Microsoft
Zendesk
Live chat is akin to instant messaging; there is a human on both sides of the interaction. It can be set up in a multitude of different ways; the most important distinction among them is what the chat agent is able to do: Can they answer only generic questions and point people toward existing documentation? Can they help with password resets? Can they answer specific questions about someone's claim?
Pros:
Humans can interpret free-text responses better than any machine learning natural language processing model, so people are less likely to be left without a way to at least begin to get their question answered
More easily than the other methods, live chat can be set up to answer questions about individual claimants' concerns
Cons:
Still constrained by number of employees or contractors, the learning curve associated with training them, and how much system access they are given to answer people’s questions → i.e., people get put in queues, there isn’t 24/7 availability
Vendors:
Genesys
MedChat
SalesForce