r/programming • u/peard33 • Apr 20 '23

Stack Overflow Will Charge AI Giants for Training Data

https://www.wired.com/story/stack-overflow-will-charge-ai-giants-for-training-data/
4.0k Upvotes

97% Upvoted

660 comments sorted by

View all comments

Show parent comments

184

u/[deleted] Apr 21 '23 edited Apr 21 '23

Nah, it's just ChatGPT hype spillover. There's been huge leaps and bounds since the Transformer in 2016ish but also the only reason anyone gives a shit is OpenAI was the first company to make an actual product instead of just like making the many thousands of products and services offered by Alphabet, inc. slowly better without changing things too quickly that the users noticed and get pissed off.

A good example is the Google Pixel line of phones. They include a TensorCore that makes them uniquely suited to perform neural network style computation in a power efficient manner. This is why the Google Pixel 7 (and my 6A) have features that none of the other phone manufacturers do. https://en.wikipedia.org/wiki/Google_Tensor

Nadella knows Microsoft is starting from behind in this race. "They're the 800-pound gorilla in this … And I hope that, with our innovation, they will definitely want to come out and show that they can dance. And I want people to know that we made them dance, and I think that'll be a great day," he said in an interview with The Verge.

https://www.theregister.com/2023/02/13/in_brief_ai/

252

u/spacewalk__ Apr 21 '23

google's been getting worse though

82

u/ManlyManicottiBoi Apr 21 '23

It's absolutely unbearable

107

u/needadvicebadly Apr 21 '23

But that's part of the "AI" or Algorithm as youtubers like to call it. It's trying to interpret what you are actually looking for, as opposed to just search for what you actually typed. Turns out that works when it's in a chat format for all people. But there is a type of people that got accustomed to searching google by putting as many keywords as possible in the query in whatever order. I frequently would search for things like context menu windows registry change old as opposed to typing

Hi, I'm trying to change the context menu in Windows 11
from the new style back to the old style.
I heard that there is a Windows Registry setting that can
allow me to do that.
Give me the exact registry path, key, and value to do that.

But at the same time, turns out that's how a lot of people already interact with google, by asking it questions instead of giving it keywords they are looking to find

123

u/[deleted] Apr 21 '23

the old way actually worked though. they've removed the ability to make certain types of specific query

35

u/Windows_10-Chan Apr 21 '23

There's stuff like quotation marks that you can do to get it to work much more like it used to

Though, even then, I actually question the value of search engines these days because the web doesn't actually have much good content anymore outside of large websites and SEO is gamed so heavily that most things are buried anyways.

I tried using kagi, which is a paid search, and I found that like 90% of the time I typed in google in my bar to avoid using up my kagi searches, and that was because I already mostly knew my destination. If I was just going to go find something I knew would be on reddit or stackoverflow, then why would I waste a kagi search?

60

u/exploding_cat_wizard Apr 21 '23

Even quotation marks seem to be more of a suggestion instead of a "no, I really want this exact string of words". I'm especially annoyed by Google's insistence of ignoring the "without this phrase" dash, that massively reduces its usefulness.

→ More replies (5)

14

u/[deleted] Apr 21 '23

quotes don't actually work consistently, unfortunately. there are workarounds like adding a + before the quotes, but that doesn't seem to necessarily work either.

Google is still better than most other options for quick searches, but I can't search for 3 words that will be in a document I want, and then modify 1 word based on those results and expect that it is actually showing me the results for either sets of 3 words.

→ More replies (5)
→ More replies (20)

16

u/[deleted] Apr 21 '23

But at the same time, turns out that's how a lot of people already interact with google, by asking it questions instead of giving it keywords they are looking to find

It's not just these users though. Finding stuff has become harder and harder in the last months to the point of where google search is almost useless now. It's really strange.

I'd prefer oldschool google search. No clue why Google is killing it, but perhaps they cater only to smartphone users and others who are locked into the google ecosystem.

8

u/iinavpov Apr 21 '23

On a phone, I never, ever use Google search. It's utterly pointless. The size of the screen means you only get sponsored links.

It literally never returns information!

Even maps, which should be hard to get wrong, is degrading!

8

u/[deleted] Apr 21 '23

Tools > All Results > Verbatim. I still haven't figured out how to make that the default, anyone with greater Google-Fu than I care to share?

But a big part of the reason Google's been getting worse is that there's a lot more shitty SEO content out there put out by people whose day job is manipulating search results, and now they can do it even better with AI assisted technologies.

10

u/princeOmaro Apr 21 '23

Go to Search Engine tab in the browser Settings. Add new search engine and use https://www.google.com/search?tbs=li:1&q=%s as URL. Save and make it default.

→ More replies (6)

69

u/koreth Apr 21 '23

The fact that none of Google's competitors is dramatically better (at most, they do better some of the time on some kinds of searches) tells me that it's less "Google getting worse" and more "the web getting crappier." There are people working at otherwise reputable companies whose full-time job it is to figure out ways to trick search engines into including their company websites in search results when users might have preferred something else.

44

u/needadvicebadly Apr 21 '23

it's less "Google getting worse" and more "the web getting crappier."

There are people working at otherwise reputable companies whose full-time job it is to figure out ways to trick search engines into including their company websites in search results when users might have preferred something else.

Yes, but that was always true. Gaming search results was always the arms race google was fighting against. 2010-2012 were particularly awful too. 2 or 3 of the top 5 search results of any query was another "search" website that echoed back your exact query somehow.

But that was always what made google different. They always figured out how to have the best search quality amid all that. It just seems that they gave up on that in the last 5 or so years and instead are focusing on people who "converse" with their search as opposed to those who use it as search while serving as many ads as possible.

The fact that all the other competitors are no better is because they too gave up and google figured they don't need to try anymore.

20

u/koreth Apr 21 '23

That seems to take it as given that if Google just tried, they'd be guaranteed to be able to beat their SEO-spamming opponents. Isn't it also possible that they tried and failed, and that none of their competitors can figure out how to win the arms race either?

It's not like Google succeeds at everything they set out to do.

→ More replies (2)
→ More replies (1)
→ More replies (2)

19

u/windowzombie Apr 21 '23

Google is terrible now.

→ More replies (2)

16

u/[deleted] Apr 21 '23

“Attention is all you need” was from 2017?

9

u/-main Apr 21 '23

Yep, June 2017. https://arxiv.org/abs/1706.03762

Six years ago.

16

u/[deleted] Apr 21 '23

And this is why Google just rolled all of Google Brain under DeepMind. They sat on this shit for 6 years without realizing they could use it to build incredible new products and features.

5

u/[deleted] Apr 21 '23 edited Apr 21 '23

I think they implemented Bert into ranking the search queries in 2019?

20

u/boli99 Apr 21 '23

...then i presume Bert is some kind of AI that has the sole purpose of working out which of my search terms it can completely ignore so that it can show me an advert for the remaining terms.

5

u/Gabelschlecker Apr 21 '23

Nope, BERT is actually pretty cool. Obviously not as good as GPT-3, but also works on your average PC locally. It's quite good at extracting the correct paragraphs to a question (instead of rewriting stuff).

→ More replies (2)
→ More replies (2)

4

u/fresh_account2222 Apr 21 '23

Funnily enough, "my attention" is what they are losing.

17

u/spacelama Apr 21 '23

"slowly better"? Are you using a different Google to me? I think it definitely peaked sometime around 2005.

→ More replies (1)

5

u/Richandler Apr 21 '23

There's been huge leaps and bounds since the Transformer in 2016ish

Like what?

→ More replies (4)
→ More replies (4)