Double check cases before final determination

Estimated impact versus effort

Medium to high impact to PER

Admin effort	Tech effort	Worker burden	Client burden
High effort	Low to medium effort	Medium to high burden	Unknown

Burden is likely due to application processing times taking longer.

Challenges

Across states, quality assurance processes tend to focus on reviewing cases after they've been completed — often, as a way to evaluate a caseworker’s performance. This can feel punitive to workers and misses the chance to catch errors before benefits go out. Where pre-approval review teams do exist, states are navigating real tradeoffs around capacity and timeliness. And when supervisors and QA teams are looking at the same cases, it limits how many total cases get a meaningful second look.

---

Plays

Create a team to review sample cases before final determination.

Through our quantitative research in Arizona, we found their pre-determination QA to be an extremely effective strategy for catching errors before they get to QC. Arizona had assembled a cross-departmental specialized team to perform this QA, requiring team members to pass a policy-based assessment before joining. We found this so effective that it’s a strategy we believe other states should consider piloting.

Our team was able to compare the data from this specialized case review and QA team in Arizona with QC data to uncover additional criteria based on income and benefit amounts that could be used to catch almost two-thirds of errors. This change requires almost zero technical effort and can be done within days.

However, it's important to note, this shift in workflow does affect staff burden by increasing the number of sampled cases. We recommend only doing this approach alongside a similar analysis to find cases that are likely to have zero errors. If states develop criteria for cases that are likely to have zero errors, then you can filter these cases from the QA sample to reduce sample size and workload.

In North Carolina, we recommended they shift QA in counties to before final determination in a small pilot test. This recommendation was based on the specific case elements that are more likely to drive errors, regardless of the caseworker who handled them. The pilot would be designed so the reviewer (either a supervisor or an experienced QA team member) would pick up a case from a caseworker and review it to complete processing. As part of this pilot, the experienced QA team member would also share feedback with the caseworker, highlighting strengths and areas to improve. North Carolina could potentially use this approach to build functionality into their eligibility system to account for a case “pause” and enable supervisors or QA teams to identify cases based on specific criteria.

Make it easier for workers to review their own work before they finalize a case.

A less burdensome way to review cases is to make it easier for caseworkers to check their work during processing. Some examples of this could be: ensuring the case summary or review screen makes potential error sources easy to view (for example, earned income or shelter deductions), or implementing an in-system module to flag key areas for review (for example, missing fields, address change, or student status).

An example screen design for an eligibility system with a window that flags high-risk case elements for the caseworker to review.

---

Key results to track

• Number of errors and error dollars found by QC

  • More errors should be caught by QA, which will lower the number of errors to be found by the QC team.

  • There should be a decrease in error dollars due to high-risk case elements.

• Percentage of pre-determination reviews that find at least one error over the error threshold

  • Increasing the number of errors found is not enough. Simply increasing the QA team’s caseload would also increase the finding of errors, but it also would put often-overwhelmed caseworkers over their capacity to process cases.

  • Instead, states can aim for QA finding 100% of errors before QC, but not having to review cases that were already completed correctly.

  • Finding errors below the error threshold, while good, is not an efficient use of the QA team’s limited time.

• Average time spent sitting in queue for pre-determination reviews

  • If an increase in time is observed, this could indicate the QA team is receiving more cases than they have capacity for and their caseload needs to be reduced.

• For changing when the QA process happens, these are different measurements to track:

  • Number of and error dollars discovered prior to case determination

  • Caseworker job satisfaction, trust, and retention

  • Average case processing time

  • Percentage of untimely processing

What to watch out for

• If a state wants to pilot something similar, we believe assessing any negative impacts to errors, timeliness, or backlogs is key.

• It’s also important to note that using criteria to flag certain case elements and developing changes to screen designs and workflows like these examples can help save time, but they are also susceptible to bias. It’s important to consider if these system interface design and workflow changes are disproportionately affecting certain demographics, especially more vulnerable populations. (Refer to Section 1: Equity questions to keep in mind throughout this work.)