Answer key questions

Given the rapidly evolving vendor landscape and the unique challenges with COVID vaccinations, here are some questions to keep in mind as you do vendor evaluations.

Key questions (Keep in mind throughout evaluation)	Specific questions to ask the vendors
Would I be the guinea pig? Understand the vendor's past government experience. If a vendor hasn't supported situations similar to yours, it'll likely be a lot of work to get what you want out of them, and there are likely some gotchas, of which neither of you are aware. It'll make your life easier to use a tool that has proven functionality solving similar problems in the past. Having experience with wide-scale COVID testing is the gold standard, because many of the requirements for COVID testing and COVID vaccinations are similar.	What’s your experience working with: Governments? The medical sector? COVID? How many customers do you already have signed on to use your tool for vaccinations? What are their biggest needs?
What capabilities are they offering? Be clear on timeline of capabilities: “available right now” vs. “near future” vs. “maybe later.” If you absolutely need a capability, make sure it is in the “available right now” bucket. Understand the experience for each type of person using the tool. E.g. sometimes a tool will be great for residents to use but terrible for administrators’ data reporting needs - ask to see both in the demo.	What do you currently offer? For features that are not currently available, exactly when will they be available for customers to use? What’s the experience like for: Residents Vaccine providers on-site IT Administrators setting up the tool Data Administrators analyzing the results IT or Data Administrators sharing data to other systems
Will I be supported? It’s unlikely any product will fulfill your needs perfectly out of the box, so make sure the vendor will support you in implementation, rollout, and in the early days of multi-user usage.	What kind of day-to-day support do you provide? How can I get help (email, phone call, chat)? What are your contractual SLAs (Service Level Agreements, e.g., resolution rates and speed they commit to clients)? What are the consequences if you don’t meet SLAs?
How are their operations scaling? Many vendors have been receiving an influx of business due to COVID, which can affect how well they provide support to you (for example, customer support might be slow to respond to emails because they are overwhelmed by requests).	How much has your company grown this year in employees and customers? Any growing pain points that I should be aware of? How well have you been adhering to your SLA over the last few months?
Is the tool reliable? Many vendors are seeing much higher day-to-day product usage than prior to COVID. Unexpectedly heavy usage can cause the tool to crash or be really slow.	Who are your biggest customers? How much do your biggest customers use your tool? Across all customers for this tool, how many transactions are you dealing with each day? What’s your maximum capacity? What kind of load-testing have you done? (“Load-testing” simulates how the tool will perform under heavy usage. If they don’t have heavy customer usage currently, insist they do this before committing to them.) What’s been your application’s up-time over the last few months? (up-time is how often the site is running and unavailable, e.g., Google’s site is up 99.978% of the time)
How does this fit in with my current workflows and tools?	Do you integrate with my EMS? Do you integrate with my IIS? Do you integrate with my billing system? Do you support data export? In what file format? (e.g., CSV, JSON)
What are the costs? Understand the fixed setup fee, ongoing fees, and costs for custom work.	What’s the setup fee? What’s the fee for setting up additional sites? What are the ongoing fees? And is the fee per site, per resident, per provider, etc? What is the per-hour cost for custom work? What circumstances would require custom work? Based on what you’ve heard about my situation, do you expect custom work will be required? Do you have an estimate for how many hours might be required? (If your budget is constrained) We’re on a limited budget: is there a way to ensure cost efficiency of the tool(s) we buy?) Any other costs I should be aware of?
How fast can they move?	How long will it take to get set up? (days, weeks, months?) If I need to make adjustments like set up additional sites, change input fields, etc, how much lead time will that take?
Are they compliant?	Are you HIPPA compliant? Is your tool ADA compliant? What kind of language translation support do you provide?
Can I trust this company and this team? You can ask specific questions to get at this, but keep asking this question throughout your evaluations. For example, a smooth sales meeting doesn’t necessarily mean that they’ll be able to deliver, but a disorganized / unaccommodating meeting is a red flag on their operating ability.	How long have you been operating? How many employees are part of the company? How much have you been growing over the last year? What’s the background of your leaders / investors? Do you have examples of success stories? Customer testimonials to share? Customer referrals? In what ways do you think you shine? Who are your competitors, and how are you differentiated from them? (They likely will have a guarded response, but this is a way to understand how they view their offering compared to competitive alternatives.)

Because the main focus area we’ve repeatedly heard from government partners right now is scheduling which includes 1) queue management and reminder tools and 2) patient registration tools, we’ve included additional detailed information for both in the following sections.

And we’ve already evaluated several vendors and are happy to share our candid reviews. We’re also happy to hop on the phone with a vendor and help you evaluate a vendor. Just let us know.