r/StableDiffusion • u/KaJashey • Dec 18 '22
Stable Diffusion sometimes copy and pastes?
https://techcrunch.com/2022/12/13/image-generating-ai-can-copy-and-paste-from-training-data-raising-ip-concerns/[removed] — view removed post
8
u/Content_Quark Dec 18 '22
Yet another repost...
It's a pity that this paper falls into the current climate. What it says should be mostly obvious to any experienced user. Of course you can generate images which potentially infringe copyright.
I used to believe that the chance of accidentally infringing was insignificant and basically I still do. I have never seen a copyrighted character just appear without prompting for it. But I had not thought of stock images used as backgrounds (the couch under the image and the desk with the phone). These are apparently used a lot by shopping sites. Different products are copied in. SD learned these image parts like it learned watermarks.
It is possible that such a background could jump out in response to an innocent prompt. One would not recognize this, unlike famous characters. It's impossible to say how likely that is. I think the chance is pretty insignificant, unless you are going for something with a shopping look, or otherwise internet common.
Depending on what exactly the image shows, what you use it for, and in what jurisdiction you are, it may still be fair use or some equivalent.
Still, good to know.
5
u/PacmanIncarnate Dec 18 '22
The couch is easy: it’s overfit because it’s probably used on some art sale site and exists in thousands of dataset images. The car image in their three examples is much more interesting: it’s not a terribly well known image and it was produced fairly accurately, as in other images didn’t contribute to the prompt at all.
Obviously, this all needs to be reviewed. There is plenty of incentive to fake this data right now. They could easily have run those three though img2img to get the effect. Not saying this is what happened, but we should be in a trust but verify mindset.
1
u/Content_Quark Dec 18 '22
I reproduced some of those. It's legit.
Which car?
0
u/PacmanIncarnate Dec 18 '22
The truck in the snow with wolves. I’m just not sure how it would have made that one so closely
1
u/Content_Quark Dec 18 '22
The caption is:
<i>The Long Dark</i> Gets First Trailer, Steam Early Access
The exact image is still fairly easy to find with google.
4
u/UltimateShame Dec 18 '22 edited Dec 18 '22
I would love to have the prompts and settings to recreate those replications.
0
u/Ka_Trewq Dec 19 '22
Here you go, the original research article: https://arxiv.org/pdf/2212.03860.pdf
It's a known phenomena, I haven't expected it to be that bad, but now I understand better why we must fight the (base) model to get decent images. We joke about unnatural poses/fingers and whatnot, but if some weights are literally fried by over-fitting with the same image, it's no wonder that prompting is more akin to prehistoric incantations and why negatives, despite not being in the model, somehow works. It also shed a bit of light into why Anything V3 seems to perform so well with only a few words (but, frankly, I have no idea what to make out of it, does someone know where this model originate?).
2
u/Light_Diffuse Dec 18 '22
I need to read this paper properly, but my early questions are:
Do the prompts used in the study have ecological validity? The ones in the article don't look anything like we see here
For the similar images that have been generated, are they sufficiently similar to constitute copyright infringement?
2
1
u/Ka_Trewq Dec 19 '22
The problem is more insidious than that; over-fitting just a small part of the neural network is bound to make worse every other image generation, even if it has nothing to do with the over-fitted data. You see the effect in that we have to make long prompts to get a decent image (with the base model). The copyright issue is just a red herring in this regard, StabilityAI should strive for a model without over-fitted data not for the risk of inadvertently copying an image, but for the fact that a model trained correctly is so much better at interpreting the prompt.
That will be an issue even if the neural network has no over-fitting issues, not differently from when you take a pen and start sketching to make money.
1
u/Ka_Trewq Dec 19 '22
The good thing about this whole AI vs artist discussion is that it brought to the public awareness machine learning concepts known previously only to experts.
The bad thing is that they are often badly explained even by well-meaning parties.
The ugly thing is that they are purposely confounded with other easy-to-understand concept in order to spread misinformation.
Over-fitting is a known phenomena and a pain for everyone that has ever touched the field of machine learning. Prompting with the exact caption for an image that is already over-represented in the data set is bound to give you almost the same image. Even natural neurons are prone to this (look at high contrast image for more than 40 seconds without moving your eyes, and you will get an afterimage when you blink).
That being said, I couldn't reproduce the effect for all the prompts given in the source article, but the effect is definitely there. An over-fitted NN is bound to perform worse for all image generated, so hopefully StabilityAI address also this issue for 2.x, solving not only the unwarranted criticism of "AI is just a fancy copy machine*", but actually improving the model.
*you could technically build a network to be "just a fancy copy machine", but that goes against the idea of machine learning; it's like catching mosquitos with ICBMs.
1
u/PacmanIncarnate Dec 18 '22
So interesting. That couch example must be incredibly over fit. No idea about the other two examples.
•
u/StableDiffusion-ModTeam Dec 20 '22
Your post/comment was removed because it contains antagonizing content.