Every AI coding or documentation vendor pitch sounds roughly the same. Fast, accurate, built for home health, easy to implement. The pitch deck rarely tells you what you actually need to know before letting a tool near real patient charts, and that's true no matter which vendor you end up choosing, including us.

Here's the checklist worth working through before signing anything.

Questions about data security and compliance

Ask whether the vendor is HIPAA compliant, and ask them to show it, not just say it. Ask whether they carry a SOC 2 report, which means an outside auditor has actually reviewed their security controls rather than taking the vendor's word for it. Ask whether they'll sign a Business Associate Agreement, since any tool touching PHI needs one, and a vendor who hesitates on this question is telling you something important on its own.

Questions about how accuracy is actually measured

Ask whether the accuracy number they're quoting came from a demo sample or from production charts at scale. The gap between those two things is bigger than most agencies expect, since a handful of clean, handpicked charts will produce a great looking number almost every time, regardless of how the tool performs on your actual messy day to day documentation.

Ask for a breakdown by service line, coding, OASIS, Plan of Care, rather than one blended figure. And ask specifically about precision and recall, not just accuracy: how often a flagged issue turns out to be real, and how many of the real issues actually get caught. A vendor who only has one number to share hasn't necessarily done anything wrong, but it's worth knowing that up front.

Questions about who's actually reviewing the work

Ask whether the tool operates entirely on its own, or whether certified coders and clinicians are part of the actual review process, not just available if something goes wrong. An AI tool working alone has a very different risk profile than one built alongside human review from the start.

This matters even more for ambient AI scribes specifically. Ask how the workflow catches it when a scribe's note is technically accurate but leaves something out, since published research on AI scribes keeps finding that omissions, not outright errors, are the more common failure. If nobody's checking a generated note against the specific items it's supposed to support, that gap can sit there unnoticed.

Questions about outside verification

Ask whether any accrediting body or outside organization has reviewed the vendor's actual process, not just their marketing claims. Self reported numbers are still self reported, even when they're measured honestly, and a vendor who's gone through independent verification, from an accreditation body agencies already trust, is making a different kind of claim than one asking you to just take their word for it.

Questions about how the tool actually fits your workflow

Ask whether implementation means integrating with your existing EMR, or whether your team has to learn an entirely new interface on day one. Ask whether there's a way to see the tool run against your own charts before committing to anything long term, side by side with your current process, rather than being asked to trust it on faith from day one.

Questions about the actual commitment

Ask what happens if it doesn't work out. A vendor confident in their own product should have no problem offering a real way out, not a long contract with no exit ramp. Ask about turnaround time and what support actually looks like day to day, since the sales team you talk to now isn't necessarily who you'll be working with once you're live.

Why this list matters regardless of who you pick

None of these questions are designed to lead you to one specific answer. They're designed to get past the pitch deck and into the details that actually determine whether a tool will hold up on your charts, with your team, under your compliance requirements. Any vendor worth working with should be able to answer every one of these clearly and specifically.

See how we'd answer these questions ourselves

We're happy to walk through our answers to every question on this list, and show you what that looks like on your own charts.