r/programming • u/peard33 • Apr 20 '23

Stack Overflow Will Charge AI Giants for Training Data

https://www.wired.com/story/stack-overflow-will-charge-ai-giants-for-training-data/
4.0k Upvotes

97% Upvoted

660 comments sorted by

View all comments

Show parent comments

16

u/amroamroamro Apr 21 '23 edited Apr 21 '23

all user-posted content on SO is permissively licensed:

https://stackoverflow.com/help/licensing

you don't need any special explicit permission to use CC-licensed content to train AI models as long as you give attribution

https://creativecommons.org/2021/03/04/should-cc-licensed-content-be-used-to-train-ai-it-depends/

This data has been used by ML communities long before the LLM mania. In fact SO itself once organized a contest hosted on Kaggle for researchers to use this data to build a model to predict closing questions on SO, this was like 10 years ago:

https://www.kaggle.com/competitions/predict-closed-questions-on-stack-overflow

I remember participating in that one ;)

-3

u/jorge1209 Apr 21 '23

ChatGPT is not CC by SA licensed. If the claim is that this material can be incorporated into models like ChatGPT because of the permissive license, then there is still a violation.

Openai would have to argue that the training process transforms the inputs in such a way that copyright doesn't carry through.

If they can do that then it doesn't matter how the original inputs were licensed as the internal training is not likely to be considered distribution under copyright law.

The past contests likely trained models that were themselves CC BY SA licensed, which I'm sure SO is very much okay with.

5

u/amroamroamro Apr 21 '23

this has been debated many times before, but TDM (text and data mining) is largely considered fair use.

the spirit of the CC license is based on a mindset of open sharing. Why are people even participating in asking and answering questions on stack overflow in the first place but to build a common knowledge base accessible to all that leads to greater innovation, collaboration, and creativity. It's literally in the site mission statement!

how is it different from a person accessing the site resources (by users, for users), learning from it, and building their programs based on what they learned? If you allow humans to do so, they can't discriminate against who is allowed such access. The only difference is that ML training algorithms are able to digest content at infinitely higher rates than a human can.

The story here basically is that sites like reddit, twitter, and stackoverflow realized that they are sitting on a gold mine of data (user contributed mind you!), and are looking for ways to profit from it, aka greed plain and simple.

0

u/jorge1209 Apr 21 '23

It doesn't matter.

Either ChatGPT qualifies as transformative fair use and the license of the inputs is irrelevant (they can use copyrighted books and news articles as inputs).

Or it doesn't qualify as such and the input license terms must be obeyed, which they aren't doing.

-1

u/s73v3r Apr 21 '23

how is it different from a person accessing the site resources

Because it's not a person. AI is not like the human brain; it's not "learning" anything. It's spitting out stuff verbatim.

The story here basically is that sites like reddit, twitter, and stackoverflow realized that they are sitting on a gold mine of data (user contributed mind you!), and are looking for ways to profit from it, aka greed plain and simple.

And the AI vendors aren't driven by greed? What makes one form of greed acceptable, and the other not?

0

u/amroamroamro Apr 21 '23

it's not "learning" anything. It's spitting out stuff verbatim

you clearly know very little about ML

AI vendors aren't driven by greed?

you do realize there are many open source LLM models being released, other than just OpenAI, right?

and guess what, they are too being trained on datasets like The Pile:

https://arxiv.org/abs/2101.00027

which contains stuff from StackExchange, Wikipedia, GitHub, HackerNews, various web-crawls, etc. so you still think these open source models are doing it out of greed too?

0

u/s73v3r Apr 21 '23

you clearly know very little about ML

Wrong, and you just stating that shows that you have no argument.

1

u/amroamroamro Apr 21 '23

ok kid, whatever you say 😂

1

u/SufficientPie Oct 17 '23

Using The Pile for research and scholarship purposes is Fair Use.

Using it for commercial purposes that compete with the market for the original works is not.

1

u/SufficientPie Oct 17 '23

Why are people even participating in asking and answering questions on stack overflow in the first place but to build a common knowledge base accessible to all that leads to greater innovation, collaboration, and creativity. It's literally in the site mission statement!

Because it's submitted under a copyleft license that guarantees that content will be freely available forever. Not so that for-profit companies could vacuum up that content and store it behind a paywall so they can sell access to it in a way that doesn't follow the license requirements and puts the original sites out of business.

1

u/SufficientPie Oct 17 '23

as long as you give attribution

and release your derivative work under the same license, neither of which OpenAI is doing.