Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
By Alyssa Levitz
Phone lines, or other high-touch customer support channels like in-person appointments when safe, are an important part of the UI system; there are some kinds of issues that can only be solved via a synchronous, human-to-human conversation. They are even more important for those with lower digital fluency or less internet access.
Black and Latinx adults are 2-3 times less likely than their White counterparts to be digitally literate. Having a better user experience, including the use of plain language (both in English and when translated), not only increases the likelihood that everyone can effectively navigate the UI system on their own, but it also reduces the need for high-touch customer support channels for those with higher digital literacy. Chatbots and other digital self-services to answer questions or resolve issues would be similarly helpful.
The improved UX needs to include clear designs for mobile devices. Black and Latinx adults are 25% less likely than White adults to own a laptop or desktop computer, an inequity which is compounded by the closing of libraries and other public spaces where such devices are typically available.
For the 14 things you can do that will most improve your UI system's user experience, see Section 5, Stage 2, Recommendations 2.3 of: "Centering Workers—How to Modernize Unemployment Insurance Technology," by The Century Foundation, NELP, and Philadelphia Legal Aid; October 5, 2020.
For a deeper understanding of how digitizing processes is important yet can deepen racial inequities -- and considerations for preventing that, see: Section 6: The Digital Divide & UI Modernization: States' Moves to Online Applications Worsen Accessibility from New America's New Practice Guide's Report on Unpacking Inequities in Unemployment Insurance.
By Alyssa Levitz. Updated: Dec. 22, 2020
One common channel for customer support is some form of synchronous online "chat." The National Association of State CIOs published a document about the broader use of chat by states during the pandemic; this document dives specifically into chats set up to support UI agencies: what types of chat do they use, and what are some of the best practices we see?
The target audience of this report is primarily state UI agencies that are looking to add or revamp their online chat offerings. Much of what is discussed here, though, is applicable to many other government use cases.
Prompted chat in which users are guided through a series of pre-defined questions and answers is likely to be most effective, but whatever chat type you use: make sure to set users' expectations up front.
Accessibility is a key consideration for chat experiences and should be incorporated from the beginning of the process.
Use data to inform the initial set up of your chat experience, and make sure to have data collection in place to iterate and improve over time.
Your system should handle 15-20 questions or scenarios very well and have clear paths for people to find more information on other topics.
"Support" chat is when a chat-like interface is set up to help someone navigate available help articles that are likely seen elsewhere on the site, e.g., linked to from an FAQ page. When someone types into the message box, options will show up of articles that the user can click on; those articles are then either shown within the "chat" window or in a new tab. There may also be buttons within the "chat" that can also be clicked.
Pros:
People have clear expectations of what they can get help with
Repurpose existing content on website
Means that there are persistent links that anyone can use to provide someone with the most up-to-date answer on a given question
Cons:
All answers have enough content to warrant their own page, and so questions with shorter answers may not be handled as well
Vendors:
MedChat
ServiceNow
Zendesk
We have only been able to find results about chat’s effectiveness at reducing call volume for states that have implemented “prompted” chat systems, and from only two of the vendors.
Colorado (Google Chat Bot): “the virtual agent solves user questions about 90% of the time”
Illinois (Google Chat Bot): “the web chatbot interacts with upwards of 100,000 constituents a day.” Google Chat Bot
New Jersey Google Chat Bot): “In its first 3 days of operation, the chat feature engaged with approximately 50,000 user interactions, freeing up time for agents to focus on claims that need intervention rather than answering frequently-asked questions.” Google Chat Bot.
Texas (AWS & Accenture): Over several months, answered Qs from 2.3 million people and save their staff from answering hundreds of thousands of calls
While all this data is promising, it leaves out how many people were confused by the chat system or just otherwise didn’t get their questions answered. That said, if your chat system is set up to help people get answers to the most common questions you see coming in through higher-cost support channels, chances are that you are well set up to deflect those inbounds.
If you are interested in working with USDR on updating your customer support mechanisms, don't hesitate to reach out by filling out this form.
By Alyssa Levitz
Unemployment Insurance agencies have been experiencing an unprecedented and overwhelming volume of telephone calls related to unemployment insurance during the COVID-19 crisis. There are simply not enough phone operators to handle the volume of incoming calls, and callers are having poor experiences (e.g., dropped calls, long wait times).
Having a complete "suite" of customer support tools not only helps agencies control costs associated with some of the higher-touch support channels, but it also helps their constituents by meeting them where they are. Phones (and when safely possible given COVID restrictions, in-person appointments) are incredibly important for a speedier resolution to complex cases -- and for people with limited sight or reading capabilities. They also provide a mechanism for those with limited or no access to the internet to get their questions answered and cases resolved.
Keeping phone lines and in-person appointments in place are important for reducing the impact of the digital divide. But without a suite of customer support tools, everyone relies on the phone and ends up crowding out the people who need the phone to do anything at all with their claims or questions.
We have done deep dives into some of the solutions that can be complementary to existing customer support systems:
They have implemented an in-house call management system, focused on call scheduling. Here’s how they describe the current system:
Customers call the main claims center line. A triage agent handles basic triaging and addresses the needs if possible. If the need is more complex, then a callback is offered. The callback is scheduled by the agent via webpage using a tool from a company that specializes in event scheduling. Available time slots are verbally presented to the customer
In addition, they have the option for a self-service page where claimants can schedule their own appointment using the same method offered to the triage agent. This self-scheduling is currently turned off, as there was too much demand compared to allocated staff, and there were too many duplicate appointments being made.
Each day, agents look on a separate webpage that has all the callbacks listed. Agents have basic information collected by the triage agent to pre-scan before making the call.
In a survey of other states’ Departments of Labor / Unemployment Insurance Departments, we found two with easily accessible online appointment/callback scheduling.
Colorado: Leverages YouCanBook.me (https://youcanbook.me/): see it embedded at the bottom of their contact page even when appointments don’t exist. They do not mention when new appointment slots are made available, but none were available every time we’ve checked.
Michigan: They use Acuity (https://acuityscheduling.com/): They also have a separate page for the scheduler; unlike Alabama and Georgia, it still lets you start the process of finding an appointment slot even if none exist. They say, “New appointments become available throughout the day exactly 7 calendar days in advance (for example, on a Tuesday morning appointments for the following Tuesday morning become available).” None have been available any time we’ve checked.
Two other states have systems in place that seem to target the same solution area: the phone lines are long and unpredictable, so having known times makes the experience better for some.
Alabama: Alabama’s tool takes basic constituent information and assigns them a time and location (theirs is not for callbacks but actual in-person appointments). You can see it on this page, but it’s unlikely to show you much unless it’s a Friday afternoon at 5PM CT when they’ve opened up new time slots.
Georgia: It is not clear what Georgia’s functionality is or how they’ve implemented it, as no appointment slots have been available on the separate page they built. It’s clear that going through that flow will result in some kind of appointment made, but not apparent how that appointment time is chosen/set. New appointments are released Monday mornings at 8 AM ET.
Since this survey was initially published in early Dec. 2020, Connecticut and Kentucky have added online appointment scheduling, but those have not been looked into.
Potentially interesting to you is how state Departments of Motor Vehicles or Departments of Licensing have implemented their online scheduling systems, as their use case is similar to UI agencies (e.g., the need to collect enough information to know who a constituent is, there’s a known number of appointments available per time slot, and it doesn’t matter within a group whom a constituent speaks with).
Vendors in this space with known government contracts are:
Acuity (https://acuityscheduling.com/): Michigan’s Secretary of State
Nemo-q (https://nemo-q.com/): Georgia, North Dakota, and Utah state licensing departments; Washington, D.C.’s DMV; and Florida’s Broward, Miami-Dade, and Volusia counties’ departments of transportation
Qmatic (https://www.qmatic.com/): New York’s Monroe County
Vcita (https://www.vcita.com/): Hartsfield-Jackson: Atlanta International Airport; Florida’s Osceola County’s Circuit Court Clerk; Hawai’i’s Kaua’i’s and North Carolina’s DMVs; Kansas state department of Revenue;
Additional vendors with DMV-targeted offerings:
Setmore (https://www.setmore.com/dmv)
Q-less (https://www.qless.com/)
States with unique-seeming solutions for their DMVs:
Unknown: California, Indiana, Oregon,
Tool referred to as “Fortress”: Washington
The evaluated vendors fall into three main categories, only one of which feels like a good match for the state UI agency's scenario: “General Appointment Scheduling,” which is also referred to as “Candidate” to reflect that match.
Healthcare appointment scheduling: These ones tend to have rich feature sets and good UX for the actual scheduling mechanics -- but are not as applicable in other ways that the agency would need. E.g., they may have only medical-specific intake forms (like uploading insurance information), or they don’t have integrations with CRMs (instead integrating with Electronic Health Record management systems). Several of them were also more focused on queue management than the actual scheduling of appointments.
Small service-sector business appointments: These vendors are targeting small businesses with a handful of employees for 1-1 or group appointments usually with a SPECIFIC individual. They also tend to offer a number of things that the agency wouldn’t use (like social media integration, payments integration, and marketing features), contributing to a higher price point than some of the other offerings.
General appointment scheduling: These vendors are targeting organizations or even individuals across sectors. Their systems are generic enough to be useful in a lot of scenarios, including the UI agency's, and so in this paper are referred to as “Candidates.” Some of these vendors have solutions targeting state Departments of Motor Vehicles or the equivalent, which is a useful comparison scenario for these purposes.
Here’s how the state agency describes their high-level needs, and USDR’s notes about that requirement after looking at the first dozen vendors or so. To see the evaluation matrix along these and other dimensions, download this spreadsheet:
SaaS-based scheduling tool.
Ability to define # of appointments available based on # of agents.
As we look at the different tools, it doesn’t appear on the surface that any offer this particular functionality. However, 1) calls with sales reps could verify this, or 2) we can think about this a different way. E.g., you may have to figure out how many appointments you can manage per slot, and the question becomes: how easy is it for you to set and update that “schedule”?
Ability for customers to self-schedule their own appointments.
Almost all the evaluated providers have the ability for you to “embed” the tool within your own webpage; some additionally have the option for you to create your own experience and use their API, or for you to use a landing page that they provide for you.
Ability to prevent duplicate appointments being set by same individuals.
No tool mentioned this as a feature, but we can ask about it in sales calls.
Note: if the system does a good enough job sending a confirmation email/SMS right after the person books the appointment, we might not see as many duplicate bookings.
Alternative: We might also be able to think through a solution to put “on top” of the vendor’s implementation. The following is an example of a solution that could work depending on the UI system’s technical limitations:
Scheduling appointments would be available only after logging in. If the tool has webhook/API support, whenever someone schedules an appointment, it would let the state's UI system know that it had happened, and then remove the option for that person to schedule another appointment, showing instead “Appointment already scheduled” or something to that effect.
You could take this a step further and have the tool send you back information about when the person’s appointment actually is to display that and give them a clear way to cancel or reschedule.
Confirmation emails/reminders to customers:
Almost all tools have both confirmations and reminders. The biggest question is email vs. SMS. SMS costs more from many of the vendors but may be more effective at reducing no-shows.
Ability to customize forms for collecting information
The tools labeled as “candidates” in the spreadsheet all have this ability; however, some only have this ability in different pricing tiers. We’d want to clarify what kind of intake data you want and run that past the sales folks to understand the feasibility of your specific need.
Ability to see all callbacks scheduled and make agent notes and set indicators for those we were unable to reach:
All the vendors marked as “candidates” have some level of reporting, though it can vary by pricing tier.
It was not apparent with most of the vendors how you might annotate individuals as un-reached; that might be something that you integrate with your CRM or have an agent manually look up and note in another tool at their disposal. Vcita has this capability in its higher-cost tiers, as does Nemo-Q (both vendors used by state DMVs).
Form localization support, including callbacks in the chosen language:
I need to go back and look for localization support of the intake form and calendar interface.
Re: callbacks in the chosen language: this could be figured out by collecting that information as data in the intake form.
Ability set workgroups so callbacks can be schedule by certain teams:
This gets to the ideas of “users” and “access”, which are ill-defined in many descriptions of the tools’ features. This will likely become clearer when we’re able to talk to salespeople and describe our scenario to get feedback.
Various reporting capabilities for metrics/quality control:
Again, we’d want to know more about what specifically the agency is looking for to evaluate this furhter.
Additional desirable features:
Some form of integration capabilities with CRM:
Many of the candidates do have this via Zapier, which is another app that helps provide the connection between different systems.
Ability to standup multiple instances to take advantage of different divisions’ needs:
Most of the tools offer this in some manner, though some might be sleeker than others.
Dashboard view of callback status for staff performance/monitoring:
This seems beyond the scope of many of the tools, but it is something that we could ask about in a sales call. Likely the first sticking point would be the fact that people wanting a call back aren’t scheduling an appointment with a specific individual.
Evaluated:
Acuity Scheduling
Appointy
Calendly
DocPace
FlexBooker
Juvonno
LumaHealth
Microsoft Bookings
MS Dynamics
QLess
Schedulicity
Setmore
Solv
Square Appointments
Vagaro
vcita
You Can Book Me
Not evaluated:
Appointlet
AppointmentPlus
EZnet Scheduler
Genbook
Marketing360
Qmatic
SimplyBook.me
TimeTap
By Alyssa Levitz. Updated: Jan. 25, 2021 to note CT and KY have added this functionality
There are three ways that the Candidate vendors price their tools: (1) per month, and (2) per month but different depending on the number of users, and (3) per user per month. Depending on the particulars of how the appointments made via this tool get surfaced to frontline agents, the vendors that charge per user per month could become astronomically expensive.
(1) Per month:
YouCanBook.Me: Their rate is actually per calendar per month, but given that you’d create some (likely small) number of calendars up-front, it’d be a known cost per month unrelated to the number of users you have. (Note: they might have discounts for government.)
Nemo-q: Their rate depends on the number of appointments you want to schedule a month.
(2) Per month, with a limit on the number of users at each pricing tier (i.e., the tier you choose is influenced by both functionality and the number of users needed -- but the cost doesn’t scale linearly with number of users):
Acuity
Appointy
FlexBooker
Vcita
(3) Per user per month (i.e., the cost scales linearly with number of users): This pricing set-up appears to be because the tool assumes that every agent would have a calendar within the product that gets synced to the “meta” calendar.
Calendly (Note: they might have discounts for government.)
Microsoft Booking (not actually known for certain that they charge per user per month, but it is the most likely case)
Microsoft Dynamics
Vcita (Note: they have discounts for government)
We looked at each state's UI agency government website, and in some cases non-governmental but official websites that people would be directed to in order to learn about or apply for or manage unemployment claims. In many cases, we took screenshots for easy reference for myself; this also has the effect of documenting some changes over time.
Once a chat functionality was found, we used the Chrome browser's Developer Tools to find evidence of which vendor was used to provide it. With this method, we were unable to determine the vendors used in Kansas, Massachusetts, and Maryland. Vendors may have more chat-related offerings than are mentioned here; this document is not intended to be a comprehensive vendor analysis.
Live chat is akin to instant messaging; there is a human on both sides of the interaction. It can be set up in a multitude of different ways; the most important distinction among them is what the chat agent is able to do: Can they answer only generic questions and point people toward existing documentation? Can they help with password resets? Can they answer specific questions about someone's claim?
Pros:
Humans can interpret free-text responses better than any machine learning natural language processing model, so people are less likely to be left without a way to at least begin to get their question answered
More easily than the other methods, live chat can be set up to answer questions about individual claimants' concerns
Cons:
Still constrained by number of employees or contractors, the learning curve associated with training them, and how much system access they are given to answer people’s questions → i.e., people get put in queues, there isn’t 24/7 availability
Vendors:
Genesys
MedChat
SalesForce
"Prompted" chat is when people are presented with a series of options for what they can get help with. The options are frequently structured as a decision tree, with sub-questions for the primary scenarios. This chat seems to always come with f-text chat, so you can get all the pros (and cons) of that, though the intended primary interaction is for the user to click buttons within the chat window rather than typing anything into the chat box.
Pros:
You can configure the flow to answer the most common questions
People have clear expectations of what they can get help with
Content is created specifically with the chat format in mind
Cons:
Without careful set up, it is very easy to create "dead ends" for people seeking help.
Vendors:
AWS/Accenture
IBM
Microsoft
Salesforce
Twilio
"Free-text" chat is when a user types their own message. The system then uses Natural Language Processing to try to interpret the message in order to provide the most relevant pre-written response. This is what people most commonly think of when they hear about "chatbots."
Pros:
People can submit any question
Cons:
People may get more frustrated at the ability to ask any question but not be able to get all questions answered.
The system may not be able to interpret what someone is saying
Vendors:
Astute Chatbot
Microsoft
Zendesk
As it relates to UI agencies, online chat implementations fall into one of four categories, which I'm calling: Live, free-text, prompted, and support. Some states have multiple implementations on different parts of the site or at different times of day; others have multiple and let the customer choose which one suits their needs. For each type of chat, we will discuss its pros/cons as well as vendors seen to be offering that functionality to UI agencies as of December 14, 2020.
Links to how you can access each state's chat system, along with any notes like what vendor they used, can be found on the page where we're tracking solutions across many states:
Microsoft Bookings: While the actual UX may not be that customizable, it has so many options, particularly around defining “service offerings” (what we would call “appointment types”) that would help it meet the needs of the organization.
Runners up: Both Vcita and Nemo-Q, with their targeted DMV offerings, appear to have all the configurability needed.
YouCanBook.me: This vendor does quite well with the various requirements against which we evaluated all the vendors, and its pricing model (per calendar per month) makes it the cheapest option. With the caveat that we're sure prices are negotiable, we're going to use an example to show why YouCanBook.me appears the best on price.
Say you wanted 3 calendars; it’d be $30/month (we think your use case could actually be served by appointment types within the same calendar, but we are using multiple calendars to make this comparison a little more starkly. To beat that monthly price, each vendor would only let you have the following number of users:
Acuity: 6 users, their “Growing” tier at $23/month
Appointy: 1 user, their “Growth” tier at $19.99/mo
Calendly: 3 users, their “Premium” tier at $8/user/mo; or 2 users, their “Pro” tier at $12/user/mo
FlexBooker: None
Microsoft Bookings: Unknown
Microsoft Dynamics: None
Nemo-Q: None
Vcita: 1 user, and it’s the “Essentials” tier that doesn’t have the functionality you’d need
Calendly: This was a bit of a hard one to determine without a sales call and demo, but at least from how their scheduling system was described, it seems like their “pooled availability” and/or “round robin scheduling” systems would be really great fits for what the agency is looking for -- and none of the other vendors at least so explicitly outlined a part of their service that would meet your needs.
Runners up: Any of the vendors with per-user calendars. It really depends on how you currently manage staff calendars and how you’d want that to integrate with the appointment scheduling solution.
Availability: For your chat system to have the most impact, you should make sure that the chat is available on any/all pages of your site (or sites!) that get meaningful traffic by people who might be looking for help with unemployment insurance. This would include any separate sites you might have for:
UI Agency administration
Standard/PEUC UI management
PUA UI management
Re-employment services and support
Some design basics:
Don't call it "live" if it's a bot or virtual agent, either in the chat itself or in content that references it. This sets the wrong expectation for people and might lead them to be more likely to provide private information even if it is not appropriate for them to do so.
The first message from the system should be short and to the point. Make sure that it is fully visible within the interface without the customer having to scroll. This means that the chat's "header" should be relatively narrow, and that there aren't large images or large blocks of text.
If content is split into multiple messages, then leave some time in between message sends for the person to at least register that there are multiple messages, if not fully read one message before the next one gets sent
For prompted chats, make it easy for users to get back to the "main menu" so that people can get multiple questions answered or change their mind about an answer response.
When implementing a chat or chat-like experience for your UI agency, you need to prepare a list of likely questions and acceptable answers. This Q&A list is used to build the decision tree and responses for Prompted Chat, to prepare agents for Live chat, to prepare Free-text chat to understand a variety of inputs, and to provide answers for all those chat types as well as "support chat.
You should prepare your system to handle 15-20 questions or scenarios very well, and have clear paths for people to find more information on other topics. There are many kinds of data you can leverage to inform this initial Q&A set, for example:
From your IVR system: what branches are most commonly used?
From your call center: what are common topics of calls? What questions are they answering over and over again?
From your claims agents: what mistakes are they seeing people make, or misunderstandings people have?
All of this data is useful in other ways, too, as you look to make more fundamental improvements to your UI system. E.g., if people frequently ask about their claim status, you could consider prioritizing work to make that possible online (or if it's already possible, to move it to a more expected location or change the language around it to make it clearer).
Regardless of the type of chat, you should make sure to have metrics in place to track the effectiveness of the tool and make changes moving forward. As economic conditions, policies, and your benefits website change, the chat experience needs to be kept up-to-date.
Effectiveness measures:
CSAT for tracking people's satisfaction with the response they got
After the chat thinks it has done its job, you can have it ask, "Did that answer your question?" For making changes, you should keep an eye in particular on the sequence of back-and-forths that lead up to someone's question not being answered - though don't forget to celebrate all the times that it did answer someone's question!
"Engagement" metrics: e.g., button clicks, messages sent, clicks on outbound links
Time spent: for chats other than live chat, how much time are people spending interacting with the tool?
Language analysis: for chats that let users input any text they want - what are they asking for that the system doesn't understand?
Visual accessibility: There are many best practices when it comes to accessibility for the visually impaired. The chat tool should have high-contrast and work with screen readers at the very least. You should not choose vendors that don't meet these requirements -- or you should push your vendor to update their tool for ALL customers to begin meeting these requirements.
Translations: Based on the demographics of your state, there are likely one or two other languages that your site -- and all corresponding tooling like chatbots -- should be available and fully translated into. When it comes to chatbots, Rhode Island with Twilio gets top marks for availability in 5 languages (English, Spanish, Portuguese, Arabic, French).
This is much easier to do for Prompted and Support chat systems rather than free-text or live, because there is a known set of inputs from the customer (clicking on links or options) and outputs (pre-written responses). That doesn't mean multiple languages shouldn't be supported in Live and Free-text chat systems.
When choosing a vendor and building the contract, UI agencies should ask how the vendor will support them in providing help in multiple languages.
Mobile-friendly: For many people, their only way of accessing the internet is through their mobile phones, potentially with limited data plans. Chat integrations should be designed with this smaller screen in mind and be evaluated for how much data they use in the course of operating. If your integration is anything other than directly "out of the box," you should request support be made available to you for ensuring a great mobile experience.
Availability: You should make sure that the chat is available on any/all pages of your site (or sites!) that get meaningful traffic by people who might be looking for help with unemployment insurance. This would include any separate sites you might have for, e.g.: UI Agency administration, Standard UI/PEUC management, PUA management, re-employment services and support.
Specificity: In addition to wanting to put your chat content on multiple parts of your site or sites, you want to make sure that chat content from non-UI sites doesn't show up here. This shows up frequently in Support Chat implementations that draw on FAQ content from across the DOL. You should ask the vendor how to set up multiple instances of the chat
Data usage and speed: If the chat implementation is slow, it increases the odds of someone abandoning chat and turning to a different support channel. People are used to the internet being blazing fast and might find that more annoying and cumbersome than putting their phone on the table and doing other things while on hold. Part of chatbot speed is also related to how it is using data; this is also important as it relates to burdening people with limited data plans as little as possible.
Quality assurance:
It should be easy to test the chat before it becomes available on the website. This is especially true for live chats, to make sure that agents are comfortable with their side of the interface.
If the chat is available on multiple pages or sites, it should be easy to test it before it goes live on those different pages. Testing on multiple pages is important to ensure that the chat is always "on top" of the content of the rest of the page; you don't want page content hiding the chat