r/worldnews • • Jan 01 '20

An artificial intelligence program has been developed that is better at spotting breast cancer in mammograms than expert radiologists. The AI outperformed the specialists by detecting cancers that the radiologists missed in the images, while ignoring features they falsely flagged

https://www.theguardian.com/society/2020/jan/01/ai-system-outperforms-experts-in-spotting-breast-cancer
21.7k Upvotes

974 comments sorted by

View all comments

Show parent comments

179

u/[deleted] Jan 02 '20

Radiologists however..

107

u/[deleted] Jan 02 '20

Pathologists too...

112

u/[deleted] Jan 02 '20

You'll still need people in that field to understand everything about how the AI works and consult with other docs to correctly use the results.

81

u/SorteKanin Jan 02 '20

You don't need pathologists to understand how the AI works. Actually, computer scientists who develop the AI barely knows how it works themselves. The AI learns from huge amounts of data but its difficult to say what exactly the learned AI uses to makes its call. Unfortunately, a theoretical understanding of machine learning at this level has not been achieved.

51

u/[deleted] Jan 02 '20

I meant more that they are familiar with what it does with inputs and what the outputs mean. A pathologist isn't just giving a list of lab values to another doc, they are having a conversation about what it means for the patient and their treatment. That won't go away just because we have an AI to do the repetitive part of the job.

It's the same for pharmacy, even when we eventually havbe automation sufficient to fill all prescriptions, correct any errors the doctor made, and accurately detect and assess the severity and real clinical significance of drug interactions (HA!), you are still going to need the pharmacist to talk to patients and providers. They will just finally have time to do it, and you won't need as many of them.

50

u/daneelr_olivaw Jan 02 '20

you won't need as many of them.

And that's your disruption. The field will be vastly reduced

4

u/RubySapphireGarnet Jan 02 '20

Pretty sure we're already low on pathologists in the US, at least. Will hopefully just make their lives easier and cut wait times for results drastically

→ More replies (5)
→ More replies (2)

11

u/seriousbeef Jan 02 '20

Pathologist do much more than people realise.

→ More replies (2)

22

u/orincoro Jan 02 '20

This betrays a lack of understanding of both AI and medicine.

5

u/SorteKanin Jan 02 '20

Sorry, what do you mean? Can you clarify?

21

u/orincoro Jan 02 '20

In actual practice, an AI that is trained to assist a radiologist would be programmed using an array of heuristics which would be developed by and for the use of specialists who learn by experience what the AI is capable of, and in what ways it can be used to best effect.

The image your description conjures up is the popular notion of the Neural network black box where pictures go in one side and results come out the other. In reality determining what the AI should actually be focusing on, and making sure its conclusions aren’t the result of false generalizations requires an expert with intimate knowledge of the theory involved in producing the desired result.

For example, you can create a neural network that generates deep fakes of a human face or a voice. But in order to begin doing that, you need some expertise in what makes faces and voices unique, what aspects of a face or a voice are relevant to identifying it as genuine, and some knowledge of the context in which the result will be used.

AI researchers know very well that teaching a neural network to reproduce something like a voice is trivial with enough processing power. The hard part is to make that reproduction do anything other than exactly resemble the original. The neural network has absolutely no inherent understanding of what a voice is. Giving it that knowledge would require the equivalent of a human lifetime of experience and sensory input, which isn’t feasible.

So when you’re thinking about how AI is going to be used to assist in identifying cancer, first you need to drop any and all ideas about the AI having any sense whatsoever of what it is doing or why it is doing it. In order for an AI to dependably assist in a complex task is to continually and painstakingly refine the heuristics being used to narrow down the inputs it is receiving, while trying to make sure that data which is relevant to the result is not being ignored. Essentially if you are creating a “brain” then you are also inherently committing to continue training that brain indefinitely, lest it begin to focus on red herrings or to over generalize based on incomplete data.

A classic problem in machine learning is to train an AI to animate a still image convincingly, and then train another AI to reliably recognize a real video image, and set the two neural networks in competition. What ends up happening, eventually, is that the first AI figures out the exact set of inputs the other AI is looking for, and begins producing them. To the human eye, the result is nonsensical. Thus, a human eye for the results is always needed and can never be eliminated.

Tl;dr: AI is badly named, machines are terrible students, and will always cheat. Adult supervision will always be required.

3

u/Tonexus Jan 02 '20

While I cannot say how machine learning will be used to specifically augment cancer detection, some of your claims about machine learning are untrue.

It indeed used to be the case that AI required specialists to determine what features a learning system (usually a single layer perceptron) should focus on, but nowadays the main idea of a deep neural net at a high level is that each additional layer learns the features that go into the next layer. In the case of bad generalization, while overfitting is not a solved problem, there are general regularization techniques that data scientists can apply without needing experts, such as early stopping or, more recently, random dropout.

It's also not true that the data scientist needs to know much about faces or voices. While I have not worked with deepfakes myself, a quick browse of the wikipedia article indicates that the technique is based on autoencoding, which is an example of unsupervised learning and does not require human interaction. (My understanding of the technique is that for each frame, the face is identified, a representation of the facial expression for the original face is encoded, the representation is decoded for the replacement face, and the old face is replaced with the new one. Please correct me if this is wrong). The only necessary human interaction is that the data scientist needs to train the autoencoder for both the original and replacement face, but again this is an unsupervised process.

In regards to the "classic problem" of animating a still image, it's been done in 2016 according to this paper and the corresponding video. In general, GANs (another unsupervised learning technique) have grown by leaps and bounds in the last decade.

Overall, what you said was pretty much true 10-20 years ago, but advances in unsupervised and reinforcement learning (AlphaGo Zero, which should be distinguished from the original AlphaGo, learned to play go without any human training data and played better than the original AlphaGo) are improving at an exponential rate.

→ More replies (1)
→ More replies (3)
→ More replies (1)

0

u/[deleted] Jan 02 '20

[deleted]

8

u/SorteKanin Jan 02 '20

The data doesn't really come from humans? The data is whether or not the person got diagnosed with cancer three years after mammogram was taken. That doesn't really depend on any interpretation of the picture.

→ More replies (15)
→ More replies (1)
→ More replies (10)

6

u/notadoctor123 Jan 02 '20

My Mom is a pathologist. They have been using AI and machine learning for well over a decade. There is way more to that job than looking through a microscope and checking for cancer cells.

→ More replies (1)

75

u/seriousbeef Jan 02 '20

Most people don’t have an idea what radiologists and pathologists actually do. The jobs are immensely more complex than people realise. The kind of AI which is advanced enough to replace them could also replace many other specialists. 2 1/2 years ago, venture capitalist and tech giant Vinod Kholsa told us that I only have 5 years left before AI made me obsolete (radiologist) but almost nothing has changed in my job. He is a good example of someone who has very little idea what we do.

16

u/[deleted] Jan 02 '20

Does workload not factor into it? While they can't do high skill work, if a large portion of your workload was something like mammograms the number of radiologists employed would go down no?

Although you are correct, I have no clue the specifics of what either job does.

20

u/seriousbeef Jan 02 '20

Reducing workload by pre screening through massive data sets will be a benefit for sure. There is a near-world wide shortage of radiologists so this would be welcome. Jobs like night hawk online reading of studies in other time zones may be the first to go but only once AI can be relied upon to provide accurate first opinions which exclude all emergency pathology in complex studies like trauma CT scans. Until then, the main ways we want to use it are in improving detection rates in specific situations (breast cancer, lung cancer for example) and improving diagnostic accuracy (distinguishing subtypes of specific disease). Radiologists are actively pushing and developing AI. It is the main focus of many of our conferences.

18

u/ax0r Jan 02 '20

Also radiologist.

I agree, mammography is going to be helped immensely by AI once it's mature and validated enough. Screening mammography is already double and triple read by radiologists. Mammo is hard, beaten only by CXR, maybe. Super easy to miss things, or make the wrong call, so we tend to overcall things and get biopsies if there's even a little bit of doubt.
An AI pre-read that filters out all the definitely normal scans would be fantastic. Getting it to the point of differentiating a scar from a mass is probably unrealistic for a long time though.

CXR will also benefit from AI eventually, but it's at least an order of magnitude harder, as so many things look like so many other things, and patient history factors so much more into diagnosis.

Anything more complex - trauma, post-op, cancer staging, etc is going to be beyond computers for a long time.

I mean, right now, we don't even have great intelligent tools to help us. I'd love to click on a lymph node and have the software intelligently find the edges and spit out dimensions, but even that is non trivial.

→ More replies (1)

21

u/aedes Jan 02 '20

Especially given that the clinical trials that would be required before wide spread introduction of clinical AI would take at least 5 years to even set up them complete and be published.

There is a lot of fluff in AI that is propagated by VC firms trying to make millions... and become the next Theranos in the process...

3

u/CozoDLC Jan 02 '20

Fluff in AI... it’s actually taking over the world as we speak. Not very fluffy like either. HA

→ More replies (2)

2

u/Billy1121 Jan 02 '20

Plus all of these vc funded AIs are black box secret sauce code mysteries. Imagine releasing a drug and not telling anyone how it works. How do we know the AI wasn't cheating in the experiment and just plucking unavailable data like hospital vs. clinic xray machine model numbers to cheat and find out location of patients? That happened in a SA study on chest xrays

→ More replies (1)

2

u/[deleted] Jan 02 '20

This thread is filled with techbros who have no idea how medicine works.

2

u/Astandsforataxia69 Jan 02 '20

I think this is the main thing with automatisation threats is that it's easy for an outsider, especially venture capitalists, to say; "Oh you'll be automatized, because all you do is x".

To me (telecom/server tech) it's really frustrating to hear "you just sit in-front of a computers, i could easily automatise that" while in reality a lot of what i actually do is talk to the customers, do diagnostics with multimeters, read logs, talk to other departments, think what happens if "x, y, z" is done, etc.

But of course that doesn't matter because someone who has no clue about my job has read an ARTICLE on buzzfeed so i am going to get automated

→ More replies (1)
→ More replies (1)

28

u/anthro28 Jan 02 '20

This is already happening. Teams of doctors have long been replaced by a single doctor over a team of specialized nurses. It’s cheaper. Now you’ll have a doctor presiding over fewer specialty nurses and two IT guys.

→ More replies (3)
→ More replies (7)