Generative AI can summarise healthcare evidence in seconds. But what happens when AI systems are used to represent what patients say about their care?
Using findings from the 2024 NHS Adult Inpatient Survey, we explored how four generative AI systems represented patient experience evidence and compared their responses with the underlying survey findings.
What did we find?
Our analysis found that:
- Headline survey findings were generally represented accurately
- Person centred perspectives were largely maintained
- Different AI systems could produce different accounts from similar evidence
- Some aspects of patient experience consistently received more attention than others
- Qualitative findings and inequalities were more susceptible to interpretation and reframing
- AI systems often relied on published summaries and webpages, rather than solely on the underlying evidence
What does this mean?
The findings suggest that AI-generated responses should be understood as part of a wider process of evidence translation, rather than as neutral or definitive summaries of patient experience.
Where decisions depend on a detailed and authentic understanding of what patients say about their care, AI-generated summaries should be used alongside the original evidence, not as a replacement for it.