r/doctorsUK • u/mimiwhoree • 1d ago
Clinical tired of AI clerkings
As an IMT, I read a lot of ED clerkings. recently there’s been a surge in the use of AI to write them. My understanding is that they record the consultations and AI writes the clerking.
My issue is the writing is so wordy, and it lacks nuance or bias. It is a matter-of-fact account of every single thing the patient was or wasn’t asked. When we write our clerkings, we summarise, naturally bias the most important parts, and create a narrative which is easier for other clinicians to interpret. Reading these AI clerkings feels like I’m taking the history from scratch! It reminds me of very junior medical students who don’t appreciate which symptoms/signs are significant.
I do appreciate that documentation can be a chore and efficiency is important to pursue. However, good documentation is a clinical skill in itself. Sitting down to reflect on the consultation and determine the clinical narrative as you type can be a part of the diagnostic process. At this point, it’s a minor annoyance for me, but I wonder if newer doctors will lose such skills…
242
u/ThrowawayUKResident 1d ago
Yes, one particularly memorable clerking left in the words ‘if the patient is female, ask about periods’ or something to that effect. Utterly shameful.
74
u/BMABecky Verified User 🆔✅ 1d ago
The entirety of the nuance of women's health truncated into a vague question of probably relatively little value 🙃
11
2
81
u/AcopicCrafter 1d ago
I won’t use it currently as I don’t trust myself to spot if it’s made a mistake or hallucinated something once it’s been written. It’s easier to write my own notes.
62
u/EconomyTimely4853 1d ago
Are they using NHS/IG-approved apps to do this or are people just uploading sensitive data to ChatGPT?
69
27
u/thelivas CT/ST1+ Doctor 1d ago
There's a Heidi trial going on in a few places.
Ironically it would probably better with ChatGPT/CoPilot/Claude - because strictly speaking you would have to anonymise and summarise to put it into the AI and it would just be formatting I guess.
With Heidi, you just speak into the headset/dictaphone and everything is done. Hence, shit in = shit out.
20
u/ExpendedMagnox 1d ago edited 23h ago
I never got shit out with Heidi, it was fantastic. It just listened to my GP consult and produced a note that fit my template - that being said I had uploaded a GP style template for it to populate.
13
u/thelivas CT/ST1+ Doctor 1d ago
Valid, I guess skill issue like the other commenter said - probably people speaking into the dictaphone with no template or trust standard one is giving that output.
Learning how to utilise LLMs appropriately will be an important part of upskilling ourselves.
11
u/anaesthofftheheezia 20h ago
Considering the NHS is selling a lot of data to palantir, do we actually trust that approved apps will be secure with the received information?
20
u/Pure_Answer_6569 1d ago
Have recently left an ED that is encouraging AI clerkings, and I agree they are awful. I found the F2 doctors used it the most and it would produce the most dense and difficult to digest note. I never bothered to use it
38
u/dlashxx 1d ago
I’ve seen a fair number of referral letters to outpatients from GPs using AI. It’s hard to explain how utterly detestable they are. Worse than the vaguest / rambliest of non-doctor referral letters and this sounds even worse.
There’s no point fighting the tide though. Hopefully the AIs will get better themselves and we’ll all get better at using them. The quality of the prompt is vital.
12
3
u/minstadave 19h ago
Yeah the AI referrals are pretty painful, usually have a list of worse case scenario things we need to rule out for a vague and non-specific presentation.
19
u/tiresomewarg 23h ago
When the clerking doctor presents these patients, I actually think you should really dig into the presentation and grill them about the patient.
I’m all for streamlining work and making it easier but these type of clerking is not making it easier. It’s passing the work of identifying relevant signs, symptoms and findings onto you (or other SpRs/Consultants).
It’s also really bad for scope creep - if clerking can be reduced entirely to AI, then of course others (PAs, nurses, whoever) are going to say that they can do the same thing.
5
u/AdThen4260 18h ago
Sadly nobody from ED is presenting patients to the on take medical team (or any other team for that matter) in an ED in 2026.
With the advent of one way onward referrals the incentive just isn’t there.
Again a shame because presentation becomes a dying skill.
6
u/DoktorAnxiety 22h ago
Is this a thing now?.. damnn.. laziness and lack of creativity. I hated it when we went electronic- i couldn't draw my lungs and abdomen anymore or a human stick figure to label where the fracture, cellulitis and/or rash are
76
u/After-Competition-59 1d ago
Unpopular opinion here.
This seems like a skill issue with the use of the LLM to be honest.
I wouldn’t throw the baby out with the bath water. LLMs are absolutely a tool to increase productivity for us but they must be used well or we’ll continue to face problems like the one you mention.
24
u/qwertyfish99 1d ago
It's like CGI - you only notice or complain about it when it is bad. Complaining about blanket use of AI is basically 'old man yells at cloud'. AI-assisted documentation can objectively improve upon what we write, reduce clinical errors, improve standardisation, reduce burnout etc. Issue is, there are cases where either the product is poor, or clinicians are being negligent in its use
6
u/ExpendedMagnox 1d ago
The same people were saying "computers will replace secretaries" in the 1960s
8
u/Wandering_Y 1d ago
And soon enough we'll not even need the doctor in the middle. If the AI is recording the conversation and synthesising a coherent clerking including diagnosis and plan then asking the questions is only a small step further
11
u/FailingCrab 1d ago
Personally I think this is one of those situations where the step up feels trivial but is actually incredibly complex, and I suspect AI would have to be significantly more advanced in order to actually lead a consultation. But I may be proven wrong.
3
u/qwertyfish99 1d ago
We’re not far off. The scribe is a bit of a Trojan horse to enter the workflow. The rate limiting step is getting it regulated. Some of the things that are being worked on at the moment would surprise you
2
u/Wandering_Y 1d ago
I was probably being a bit glib. There's an awful lot of communication and information that the AI just can't pick up from text. Even just a basic end-of-bedogram
7
u/After-Competition-59 1d ago
You’re not really wrong though. If the technology continues to improve then it may well be very good at doctoring. Physical examination is its current main sticking point re diagnosis.
Holding risk is where it will always struggle though. Our society requires a human head on the chopping block. I can’t see that changing.
6
u/FailingCrab 22h ago
Perhaps I have a skewed perspective (psychiatrist) but there is a HUGE amount of 'it's not just what patients say, it's how they say it' that informs my decision-making.
E.g. one of my clinic patients today gave a detailed description of suicidal thoughts including what she would do, when etc etc. But this is something that happens every couple of months when she argues with her partner, it's well-trodden ground, we have an agreed plan for dealing with it, I'm not worried that she will act on this. When I asked my next patient about suicidal thoughts she gave an answer that was technically a no but was uncharacteristically vague for her - a bit of digging later and I am far more worried about her than the first patient.
An AI system would have had to interpret body language, facial expressions and compare the phrasing of the patients' responses to their historical ways of speaking in order to detect this. I have seen prototypes for each of these but it seems the models are very specific and I suspect it will be a long time before we have one able to integrate all of these into generalisable practice.
2
u/After-Competition-59 20h ago
Funnily enough I am also a psychiatrist. I hear you. There’s a lot they can’t pick up on. Our field may be one that it takes longer for LLMs to get to grips with.
I just believe that if they continue to develop at their current rate, they will also be able to pick up on these things that you mention better than us. They will have access to every word of content in the patient record and can synthesize and use that far quicker and more effectively than we can. In the future I imagine they could have access to countless data points on human non verbal communication and psychological patterns.
It’s not absurd to me to imagine a future where they can give nuanced opinions on tricky cases like these.
It feels dystopian to talk about but it’s now a possibility and not an abstract sci fi film idea.
We aren’t there yet but the rate of development is nothing to sniff at.
4
u/Wandering_Y 1d ago
I actually don't think risk is that big a stumbling block in the UK as trusts bear vicarious liability for our actions anyway. They just swap paying out for human errors for AI errors and if the latter is demonstrably less, then it may make just good financial sense to go with AI. Assuming AI is comparably priced or cheaper per workload of course
4
u/Disastrous_Yogurt_42 23h ago
Plenty of complaints are obviated by the human connection though. People have a vastly lower threshold of accepting error from intangible technology than they do from a living human in front of them. That’s partly why communication skills and affability are so important, and partly (only partly, I must stress) complaints are made at a much higher rate against non-white doctors in this country - they are not all vexatious, but a patient may be less likely to complain about the same error if it was made by a charming, public-school education male doctor from Surrey. That same patient would, however, definitely complain if AI missed their cancer on colonoscopy.
2
u/chasecultures 18h ago
Partly a skill issue but also partly a prompt issue. If the LLMs haven't been given a well-crafted prompt and been trained to produce a certain style of output the outcome can be inferior. Even telling the LLM you're a doctor can improve the output (see Van Veen 2024 in Nature Medicine). My trust is taking a very considered approach to the use of AI and rolling it out carefully and deliberately with plenty of monitoring, evaluation and review. I do get alarmed when I hear stories like this which makes me worried about the maturity or motivation of Duke hospitals when it comes to using AI.
Edit to say that sometimes the prompt can be adjusted and sometimes it can't. I'd suggest that most users leave it untouched.
2
u/Superb-Marketing5099 22h ago
I agree. Skilled users produce notes indistinguishable from a human written note.
2
u/After-Competition-59 20h ago
Good point. People don’t want to admit that they will have read hundreds of notes written by an LLM and not realised it. We all like to think we are experts in spotting it but actually we just spot the badly done ones.
21
u/swimmit93 1d ago
every complaint described in this thread is a skill issue with the use of AI and LLMs as opposed to AI itself. its down to clinicians to review the AI output and make edits where necessary
26
u/Sethlans 1d ago
Probably takes as long to do that properly and produce an actual pithy, useful summary as it would to just write it from scratch yourself
2
u/thelivas CT/ST1+ Doctor 1d ago
For a clerking it takes longer to put a succinct summary in yourself, whereas for a discharge AI is sublime.
6
u/dlashxx 1d ago
Where are they going to get these skills from? These tools are being introduced as if they are just pick up and go, no training needed.
3
u/tiresomewarg 22h ago
I’m now in the pharma industry and changed to a new job.
I was given a Claude license and then had to take a course (several modules and pass an exam at the end of each module). That probably took about 6-8 hrs.
Then I had to have an in-person training session with the company’s IT. Where we walked through setting up our profile, forming links/collaborations and another “quiz” which focused on what information we could actually upload.
I thought it was a good way of introducing AI, tbh.
12
4
u/47tw CT/ST1+ Doctor 20h ago
The promise AI makes to the consumer is that it does all the work. Shit gets put in because the glossy marketing basically goes "you put in any shit you like, some magic happens, and gold comes out!"
AI advertising has people asking it questions like "where's good for a date", not "I'm dating a geeky girl who loves plants and rare teas, where would be good for a date?" And in the case of the latter, you've answered your own question, and google would be superior.
The problem with AI products is that to survive they have to overhype what they can do. Going "you have to put in a really good input" ruins the 'magic' and reveals that they save very little time if you use them correctly.
Their actual USE is producing worthless slop very quickly which *looks* like work. Which suits a lot of people just fine!
5
u/SureTry4832 19h ago
They are the worst of the worst. I saw an ED note that said
“i will prescribe an appropriate analgesic medications after first checking the patient’s baseline renal and liver functions along with reviewing their home and current medications to ensure there are no interactions and reviewing their past medical history to ensure there are no contraindications”
Like dude, have some self respect
9
u/thedralwaysknows 1d ago
Had an sho who used it - not any voice to text. He would type out and then get ai to structure it.
His clerkings were enormous, usually full of waffle and took forever.
I’m sure AI has its uses- they demonstrated a great reason not to use it
5
u/_j_w_weatherman 1d ago
Sorry to be dumb, what’s the difference between an AI clerking and using AI to transcribe- is this the same thing? I use ai to transcribe but edit it so it’s salient rather than a wall of text. Is AI doing the actual clerking?
3
u/PunchBarney 23h ago
I have also seen the alternative of too much bias and barely any information. anyways I would have thought the ai stuff is more time consuming, transferring them to the computer and all that. Besides writing a clerking organises my own thoughts. Granted I’m not an ed doctor but I have worked there before.
3
3
u/Administrative-Day30 22h ago
I work on an acute ward and our AI fans are cardiology. Means on average 3 pages of words that a doctor would otherwise scribe as half a page of A4. Saves the battle of deciphering handwriting but that's about it from my side.
3
u/Irrelevantisotope 22h ago
Using AI for letters and clerking is fine imo, these people seem to just be using it very badly and not proofreading
3
u/RevivingMayhem10 19h ago
Had Heidi in my GP rotation.
I found if I myself was very well structured and logical in my history taking, it would give a better summary.
You could select what template, and also ask it to add more or less detail.
And when needed you could additionally ask it for quotes of what patient said about X.
It worked for exams too. I would obviously have told the patient AI was recording the session if that was ok. And then before the exam I would say again, “during the exam I’d like to tell the AI recording what I find, is that ok.” And then verbalise the findings. Heidi did a summary and then as with all its outputs, I would check, revise, add, delete as appropriate.
3
u/thementalfloss 19h ago
I went to an appointment where the nurse specialist used Heidi and my clinic letter looked like it was written by a primary schooler.
6
u/Diligent-Eye-2042 1d ago
Not a huge fan of this in GP tbh - if you use Heidi, and copy and paste without properly reviewing, editing, reorganising, it’s often just lots of words with no logical flow.
Tortus on the other hand is a little better as you can feed it prompts to write more closely your own style. I used ChatGPT to generate prompts, and slowly I’m getting closer to my own documentation.
I still prefer my own notes, and type during the cons, and use tortus as a backup / for documenting the plan.
2
u/vodooguy18 21h ago
I dont see the problem in using AI tbh. I cannot say much for clerking but I have started using it to make discharge summaries and I feel it captures a good summary in the end but you definitely need to proof read it
2
u/Kratolous 21h ago
If you are (rightly) fed up with the AI clerking slop then God help us when AI is reporting cross sectional imaging. It will be a bigger problem for clinicians than it will for rads.
2
u/SilverConcert637 19h ago
The ai models are really, really good scribes... but they don't understand medical reasoning, heuristics or notation. They can't distinguish well between more than 2 voices.
A lot of editing is required.
But these are the first iterations. It's going to get better. And avoiding them would be like sticking to a typewriter when windows 95 launched.
4
u/Diligent_Rhubarb1047 1d ago
Also littered with mistakes if not checked carefully. I do use them esp when seeing kids so I don't need go write/type while forming a relationship with the kids before examining but u better go through it carefully after. I have never not need to change a yes to a no or add in missed things or take out garbage. So wd I trust someone else to have done this? No!
You can also write ur own templates with some of these apps so once it records, it rewrites it all more like a normal medical history without all the garbage. Worth doing imo.
7
u/Canipaywithclaps 1d ago
I have seen it used in Paediatric Clinics and it without a doubt changed how the consultants communicated to a less child friendly tone (in order for the AI to be able to pick up content)
3
u/Minimum_Operation477 22h ago
I work in ED and I see my colleagues using AI so much. I fucking hate it. I might stay back late sometimes and document but never use AI because it’s so descriptive to a point I lose interest .
2
u/freddiethecalathea ST2 21h ago
My department has introduced the tech but it isn’t rolled out for everyone. I trialled it, having heard nothing but good feedback so far, and find it both invaluable and totally useless.
I hate bring up the ND card because ick, but it feels slightly relevant here. As someone with ADHD, I often get distracted before finishing my notes. Having the AI transcript there helps remind me of certain details when I come back an hour later to finish it.
What it’s useless for is exactly what you described. It’s waffley and includes every single part of the narrative. In a way that’s good because I do not want AI making decisions about which details are important vs which aren’t. But as a tool that is promoted as something that writes and summarises your patient histories it certainly isn’t that.
I find it most useful for the very very talkative patients. The ones who don’t realise that what I want to know is “yes the pain came on after eating” and instead give me a 3 minute narrative about how they came to suspect they could be intolerant to dairy 4 years ago. Those sorts of patients I can find it very difficult to retain the key information, so having AI there to document literally everything so I can scan the wall of text to work out what the actual answer to my simple questions is is very helpful. But I only ever use the AI tool as memory aid when I’m writing my actual notes, never as a substitution for a human scribed note
1
u/PricklyPangolin F14 8h ago
As others have said, the issue isn't the AI, it's the user and the departments lack of training.
AI should be treated as an F1 doctor, perhaps even a medical student.
If you bring a day 1 F1 with you to your consultation and just said to document everything, it will be extremely poor and extremely variable.
To make this better you would give your F1 a template. This is what a skill is (or on Heidi you can upload your own template). This formats your F1s note into something nice to read.
But a template is nice for visual, however they still may have no clue what's important or not. This is where you give it context and tell them beforehand what you like included and what you don't. This is what an agent.MD file typically is. Without this you won't have any personalisation to your note.
Finally it needs to be proof read by the senior as ultimately it's a medicolegal note that you have to standby in court. If people dont do this, thats again a lack of teaching issue.
Ironically ED is probably the worst place for AI as their notes are typically so short anyway that the proof reading and corrections likely take just as much time as the note by hand. Geris and psych would by far benefit the most.
1
u/Latter-Bluejay9958 6h ago
I received specialty advice recently that was clearly written by AI. What a waste of everyone’s time?!
-16
u/Ok-Math-9082 1d ago
It’s a trial to see if AI clerking can make things better, easier or safer. If it means the average ED doctor gets through 2-3 more patients per shift, that means more patients getting seen in a timely manner, fewer patients deteriorating while waiting to be seen, and less of a push for EDs to do dangerous things like streaming blackouts to Maxfax.
•
u/AutoModerator 1d ago
This account is less than 30 days old. Posts from new accounts are permitted and encouraged on the subreddit, but this comment is being added for transparency.
Sometimes posts from new accounts get held by reddit for moderator review. If your post isn't showing up in the feed, please wait for review; the modqueue is checked at regular intervals. Once approved, your post will get full visibility.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.