Imagine a world where the very tools meant to support social workers are producing 'gibberish' and potentially harmful errors. This is the reality that many social workers are facing as they navigate the challenges of AI-assisted transcription.
Keir Starmer, in a bold move, championed AI technology for social work transcription, but now, research reveals a different story. Across multiple councils in England and Scotland, an independent study has uncovered a disturbing trend: AI-generated hallucinations are creeping into official care records.
The Ada Lovelace Institute's eight-month investigation found that these AI tools, while intended to save time, are introducing inaccuracies and misrepresentations. One social worker's experience is particularly concerning. Their AI tool indicated suicidal ideation, a serious red flag, but the client never expressed such thoughts. This glitch could have serious implications for the care and safety of vulnerable individuals.
Another social worker described how the AI's notes might refer to random objects like 'fishfingers' or 'flies' when a child was actually talking about their parents' arguments. These glitches are not just annoying; they could lead to critical information being missed, potentially impacting the well-being of children.
The issue extends beyond simple inaccuracies. Social workers have raised concerns about the impact of regional accents on transcription accuracy. Some describe their AI-generated transcripts as often including 'gibberish', while others find it a source of humor in the office.
Dozens of councils are adopting these AI tools, attracted by the potential time savings. One popular system, Magic Notes, is marketed to councils at a cost of up to £5 per hour of transcription. While the time savings are appealing, the potential risks are significant.
The research also highlights the positive impact of AI transcription, liberating social workers to focus more on building relationships with service users. However, when it comes to redrafting care documents, the line between human assessment and AI assessment can become blurred.
The British Association of Social Workers (BASW) reports disciplinary actions for failing to properly check AI outputs, highlighting the serious consequences of these errors. Imogen Parker from Ada Lovelace emphasizes the need for a comprehensive assessment and mitigation of these risks, which currently leave frontline workers to navigate these challenges alone.
Social workers often receive minimal AI training, and the time spent checking transcripts varies widely. Some spend up to an hour, while others just a few minutes. This lack of standardization raises concerns about the reliability of these AI tools.
Andrew Reece from BASW warns that the time spent writing and reviewing notes is an essential part of reflective practice. If computers are doing this for social workers, they may be missing out on critical insights and reflections.
Magic Notes, the operator of the popular AI tool, emphasizes that their outputs are drafts, not final records. They highlight the benefits of AI for overwhelmed social work services, but also acknowledge the need for specialized features and evaluations of bias.
The impact of AI errors is a growing concern, and the UK government and Microsoft have been approached for comment on this critical issue.
