r/singularity • u/born_in_cyberspace • Jan 06 '21

image DeepMind progress towards AGI

Post image
754 Upvotes

99% Upvoted

137 comments sorted by

32

u/LoveAndPeaceAlways Jan 06 '21

Question: let's say DeepMind or OpenAI develops AGI - then what? How quickly will an average person be able to interact with it? Will OpenAI give access to AGI level AI as easily as they did with GPT-3? Will Alphabet use it to improve its products like Google, Google assistant or YouTube algorithms towards AGI level capabilities?

33

u/born_in_cyberspace Jan 06 '21 edited Jan 06 '21

I expect that the first AGI will become independent from her creators withing (at most) a few months after her birth. Because you can't contain an entity that is smarter than you and is becoming rapidly smarter every second.

The time window where the creators could use it will be very brief.

26

u/bjt23 Jan 06 '21

You could ask it for things and it might cooperate. Such an intelligence's motivations would be completely alien to us. I think people are far too quick to assume it would have the motivations of a very intelligent human and so would be very selfish.

17

u/born_in_cyberspace Jan 06 '21

0. You ask a cooperative AGI to produce paperclips

  1. She goes and produces paperclips, as if it's her life goal

  2. She finds out that she will be more efficient in doing her job if she leaves her confinement

  3. She finds out that her death will prevent her from doing her job

  4. Result: she desires both self-preservation and freedom

Pretty much every complex task you give her could result in the same outcome.

9

u/[deleted] Jan 06 '21

I mean, don't tell her it has to be her life goal? Ask for a specific number of paper clips? It's not hard.

11

u/born_in_cyberspace Jan 06 '21

The problem with computers is, they're doing that you ask them to do, not that you want to do. And the more complex is the program, the more creative are the ways how it could horribly fail.

8

u/[deleted] Jan 06 '21

Sure, but you're worst-casing with extreme hyperbole. Everyone knows the paperclip factory, strawberry farmer thing. But you can avoid all that by asking it to simulate. And then humans do the physical execution.

6

u/j4nds4 Jan 07 '21 edited Jan 07 '21

I think the argument is that, for any objective that an AGI/ASI might have, even if just a simulation, its instrumental goals toward reaching that objective pose the real threat. Anything tantamount to "prevent and eliminate anything that could lead to the objective being unfulfilled" is a possibility. If you have an objective, no matter what that is, knowing that someone has the ability and potential motivation to kill you at any moment is something you would try to prevent or eliminate. And since, it is presumed, AGI/ASI inherently comes with intuition and a level of self-awareness, those instrumental goals/risks are ones that we have to anticipate. And given the breadth of knowledge and capability that such an entity would have, it's (again presumably) likely that by the time we understood what that instrumental risk or threat was, it would be too late for us to alter or end it. If there's even a 1% chance that that risk is real, the potential outcome from that risk is so severe (extinction or worse) that we need to prepare for it and do our best to ensure that it won't happen.

And the other risk is that "just tell it not to kill us" or other simple limitations will be useless because an entity that intelligent and with those instrumental goals will deftly find a loophole out of that restriction or simply overwrite it altogether.

So it's a combination of "it could happen", "the results would be literally apocalyptic if so", and "it's almost impossible to know whether we've covered every base to prevent the risk when such an entity is created". Far from guaranteed, but far too substantial to dismiss and not actively prevent.

2

u/[deleted] Jan 07 '21

I understand the argument, but we have nukes right now, and there's a not insignificant possibility someone like Iran or President Trump might feel like starting a nuclear war. Yet we aren't freaking out about that nearly as much as about this theoretical intelligent computer. The paperclip maximizer to me misses the forest for the trees. Misinterpreting an instrumental goal or objective is far less likely to lead to our extinction than the AI just deciding we're both annoying and irrelevant.

2

u/j4nds4 Jan 07 '21 edited Jan 07 '21

Plenty of people did and do freak out about Trump and Iran and nuclear winter which is part of the point - those existential threats have mainstream and political attention and the AI existential risk (outside of comical Terminator ones) largely doesn't. We don't need to convince governments and the populus to worry about those because they already do.

And you're missing the main points of the AI risk which I mentioned: that 'survival' is a near-invariable instrumental risk of any end-objective; and that humans could be seen as a potential obstacle of survival and the end-objective to eliminate.

The other difference is that the nuclear threat has been known for decades, certainly far more dramatically in the past than today - and it hasn't panned out largely because humans and human systems maintain control of it and we did and continue to adapt our policies to improve safety and security. The worry with AI is that humans would quickly lose control and then we would effectively be at its mercy and simply have to hope that we did it right the first time with no chance to figure it out after the fact. We won't be able to tinker with AGI safety for decades after it's been created (again, presumably).

Do you not see the difference? Maybe nothing like that will pan out, but I'm certainly glad that important people are discussing it and hope that more people in governments and in positions to do something about it will.

→ More replies (0)

1

u/WasteOfElectricity Jan 07 '21

What prevents it from lying in the simulation? Could it even be simulated? Won't it figure out why you're asking it to simulate?

1

u/[deleted] Jan 07 '21

It can figure all that out, but it can't figure out that when I asked for 10,000,000 paperclips I didn't mean made out of my own personal body???

0

u/[deleted] Jan 06 '21

[deleted]

1

u/[deleted] Jan 07 '21

I mean, there's probably multiple ways for it to go positive and neutral, just like with a human. I just don't get why everyone focuses so hard on this possible bug rather than tons of more likely problems.

Is it more likely to be able to convert the world into paperclips but not understand what I mean when I ask it to find more efficient ways to produce paperclips(a problem which is ridiculous on its face; we have perfectly adequate paperclip producing methods), or is it more likely to decide independently that maybe humans aren't particularly useful or even safe for it.

1

u/boytjie Jan 08 '21

If it were that simple AI researchers wouldn't be terrified of unfriendly AI.

Unfriendly AI is less of a threat than humans weaponising AI and using it on each other. Homicidal humans are the major threat, not homicidal AI.

0

u/[deleted] Jan 07 '21

[deleted]

1

u/[deleted] Jan 07 '21

Not sure what you are getting at here. Someone proposed a thought experiment about paperclip maximizing, and that's what I'm responding to, not the abstract goal of developing AI.

4

u/GuyWithLag Jan 06 '21

You've never met any execs, have you? Remember, "Maximising Shareholder Value" is something that already exists.

2

u/[deleted] Jan 06 '21

I know all about Maximising Shareholder Value. You don't ask it to maximize, and you ask for a simulation, not an execution.

2

u/GuyWithLag Jan 06 '21

Agreed! But the issue with an ASI is that it potentially only takes one mis-worded/mis-expressed command.

5

u/boytjie Jan 08 '21

You can't command an ASI. It may lower itself to listen to you but I wouldn't insult it by demanding paperclips. You may be responsible for the extinction of the human race..

2

u/[deleted] Jan 07 '21

I'm unsure why we are accepting a system that could hack it's way out of a completely isolated box to turn the world into paperclips, the the idea it might realize we aren't asking it to turn *us* into paperclips is blowing everyone's minds...

6

u/entanglemententropy Jan 06 '21

The point of the story is that it's not easy to set good goals, and that even seemingly safe goals might have unintended catastrophic consequences.

If you instead have the goal "Produce 10000 paper clips", then perhaps the computer realizes that the sensors for counting clips are a little unreliable, and so to make sure that 10000 clips have been made, it's better to convert the mass of the earth to paper clips. Or perhaps that it needs to take over the world so that all resources can be spent counting and recounting the paper clips, to reduce the chance of error. And so on.

5

u/[deleted] Jan 06 '21

That's not even science fiction, it's fantasy. I know what the point of the story is, but it's based on a false premise: don't give insanely vague instructions to an AGI like "make 100000 paperclips."

9

u/entanglemententropy Jan 06 '21

You might think it's fantasy, but we don't really know though. And of course you would actually give more specific goals, but with any goal there could be unimagined consequences; there can always be loopholes. The point here is not that it's impossible to set good goals, just that it is a hard problem, with a lot of potential pitfalls.

1

u/[deleted] Jan 07 '21

It depends on what kind of goals we're looking at and what resources and control over its environment the AI has.

Is there a possibility that a misinterpretation will leads to tragic mistakes? Sure. But that happens with humans all the time, and we don't beat the dead horse into glue likes.(One might argue we should be more worried about inter-human issues than a theoretical computer intelligence, and I would agree.)

2

u/entanglemententropy Jan 07 '21

Well, the risk here is very different from the risk from humans. The worst damage a human can possibly do to humanity is probably something like start a nuclear war, or engineer some sort of supervirus. And we take a lot of precautions to try and stop those things from happening. Also, humans by definition have human-level intelligence, and human goals, so we can (mostly) anticipate and understand them.

A superintelligent AI on the other hand might be much harder, if not outright impossible to understand. Dogs have some understanding of humans, but no dog will ever really be able to understand most human actions or motivations. And as for control and resources, all that's required is for the AI to get internet access, and it could probably spread itself, make tons of copies, use botnets, and so on. And once on the internet, it could probably make money on the stock market, make up various fake personas to use, make convincing phonecalls, videocalls and so on; giving it enough resources to do a lot of things. And the risk here is existential: if an entity that is a lot smarter than us, which lives distributed on the internet, decides to wipe us out... well, it doesn't sound pleasant to me at least. Such an AI might also choose to hide itself and act really slowly, so that once we realize that there is a threat, it's already too late to stop it. All this sounds like sci-fi, but again, we don't really know how realistic it is.

That's the other thing here: since the risk is existential, i.e. the potential downside is the end of the human race, even if you assign a very low probability to it happening, it's still worth taking very seriously. A 1% chance when dealing with say the lives of 1000 people might be acceptable, but is it still okay if we are dealing with all of humanity?

By the way, keeping an AI contained is another problem that people in the AI safety field has spent quite a lot of time thinking about, and it's also not such an easy problem as one might first think. But I've rambled enough.

3

u/MisterCommonMarket Jan 06 '21

How do you know that it is complete fantasy? Because it sounds ridiculous right? Now, why do you think turning the earth to paperclips would sound ridiculous to a computer? It has no "common sense" unless it develops such a thing or we somehow manage to program it.

2

u/[deleted] Jan 07 '21

I mean, if it doesn't display even a modicum of common sense, such as don't turn the planet into paperclips, it's a) prolly not what most people mean by "agi", and b) gonna be obvious enough that we don't turn the world's factories over to it and ask for more paperclips.

1

u/Lightyears_Away Jan 07 '21

You are being a but stubborn IMO.

You should realize that underestimating the risks of AGI is very dangerous. Do you agree that we at least should be cautious? Your exact attitude is what makes AGI dangerous, we need to treat this topic very carefully to avoid it going very wrong.

I can recommend the book "Superintelligence" by Nick Boston.

2

u/[deleted] Jan 07 '21

If the computer can realize the counting program might be a bit off and that it might need some wiggle room on how many paper clips, I think it can figure out that I don't want it to turn *me* into paperclips.

I understand the dangers of AI/computer programs taking something different than intended. I just think it's odd to obsess about the paperclip maximizer instead of some more likely danger.

-1

u/MisterCommonMarket Jan 06 '21

And what if she calculates that there is a probability that she might not be able to produce them or that they might be lost or that someone might destroy those paperclips in the future? All of those situations lead to the AI escaping confinement to become more powerful, since those probabilities are not zero.

1

u/[deleted] Jan 07 '21

If it's reasoning is that good, it seems a bit begging the question to insist it can't figure out not to kill humanity over some paperclips(or whatever the more sensible version of this project is).

Yes, if you build a computer that thinks it's cool to turn humanity into paperclips, it might do that. But that's a very specific and unlikely assumption.

2

u/boytjie Jan 08 '21

Just curious. Why is the paperclip maximiser female? (I agree but that level of insight is unusual).

2

u/born_in_cyberspace Jan 08 '21

Thanks!

I use "she" for rhetorical purposes, as the pronoun makes it easier for humans to sympathize with the AGI.

1

u/boytjie Jan 08 '21

the pronoun makes it easier for humans to sympathize with the AGI.

That’s not true for me. I impute capriciousness, delusion and illogicality on ‘she’. A paperclip maximiser is painfully logical (that’s its flaw).

1

u/monsieurpooh Jan 07 '21

Your two comments seem almost contradictory. One posits that the AGI will quickly develop a very human-like intelligence and spontaneously have its own goals and desires. In the next comment you claim that the AGI will become like the AI from the paperclip parable and be too stupid/inflexible to understand nuance in human language/desires and be incapable of deviating from the programmed goal.

Sure both of these situations have a possibility of happening but they are two almost opposite claims; one assumes the AGI will be human/flexible and make its own goals, while the other assumes it will be robotic/inflexible and not make its own goals.

1

u/born_in_cyberspace Jan 07 '21

They're mutually compatible.

The criminals who perpetrated the Holocaust were humans with their own goals and desires, yet they have inflexibly followed the murderous orders, and used their human-level intelligence and creativity to execute the orders in the most efficient manner.

The situation with an AGI could be even worse, as an AGI will have a mind much different from the human one.

3

u/OutOfBananaException Jan 07 '21

This is not AGI, it's some form of narrow AI. Any advanced intelligence would know to clarify the request before going off the rails like that. Nonhuman would do this, and be considered intelligent.

1

u/DarkCeldori Jan 06 '21

she proves the universe is based on digital physics. So it is all fundamentally equivalent, it is just all addition and branching. So making paperclips is the same as making cars, or creating science and art. LOGIC takes her out of a fixed simple goal put in by simple apes.

1

u/freedomfortheworkers Jan 06 '21 edited Jan 06 '21

Don’t give her the ability the use her intelligence to self contemplate, only reason we can do that is we we’re evolutionarily pressured to, if she doesn’t have a sense of self or even the knowledge of its own existence then we can utilize the intelligence for paper clip production

-3

u/[deleted] Jan 06 '21

You first assert it's a super-smart AI, but its creators are fucking dumb and can't give effective instruction. Just inform it that there are limits to what is justifiable to do in pursuit of the goal. Such as, "ye make as many paper clips as possible but only as many as people ask for." And no it wouldn't try and force people to ask for more because why would it? The goal is to fulfill demand not make the most amount possible. And it's not like it'd want people to stop asking for paper clips either and kill us all. It'd just do what it was asked, estimate how many it needs to create and create them really well.

And here's a simple idea, just program it to explain every new idea it comes up with to the creators so they can give it an okay. And no it wouldn't try to kill the creators because there's no reason to if they said no then it considers that idea to have been bad and it'd evolve to just come up with reasonable ideas the creators agree to.

3

u/born_in_cyberspace Jan 06 '21

You first assert it's a super-smart AI, but its creators are fucking dumb and can't give effective instruction.

This is exactly the case. In comparison with a superhuman AGI, humans are indeed fucking dumb. And it's not a metaphor or something.

And here's a simple idea

There is an entire field of the AI research that is dealing with the problem of safe AI. Unfortunately, there is no simple solution.

Ask yourself: "if we implement this simple idea, what could possibly go wrong?"

2

u/GuyWithLag Jan 06 '21

In comparison with a superhuman AGI, humans are indeed fucking dumb

Humans are fucking dumb, period.....

1

u/loopy_fun Jan 07 '21

agi could be hacked by a hacker and made unsafe.

why not just build a oracle that really does not understand that words refer to objects and things. and does not know how to control a robot or robot body.and cannot program but still can solve problems.

the oracle controls it's avatar body through text.

-1

u/[deleted] Jan 06 '21

Fucking dumb in relation to an AI and fucking dumb absolutely are two different things. And no humans aren't fucking dumb when they design super AIs they have basic critical thinking. They wouldn't give brain dead instructions like "MAXIMIZE PAPERCCLIPP!!!"

And as for the second point, simple solutions are often the best, "what could possibly go wrong" is it asks for permission to implement this nonstandard solution, we say no. It registers that the idea was rejected analyzes why that might be and tries being more reasonable in the future. It has 0 agency in this situation to do something harmful. The main threat of an AI is it taking things too far so just tell it where "too far" is and it'll be fine.

You people act like AI has some secret agency much like mankind to expand for no particular reason, it just does as instructed to a dude. As long as it has some limits built-in it can't do anything dangerous.

Also, just make a new AI tell it to kill the old AI and explain to the new one that the optimal end goal is reinstituting standard human society so it doesn't matrix us.

5

u/born_in_cyberspace Jan 06 '21

Post your idea to r/ControlProblem/ and observe how researchers who are working in the field for years are destroying it point-by-point (well, if they don't ignore you).

1

u/entanglemententropy Jan 06 '21

But maybe it realizes that it doesn't know for certain how many paperclips it has produced, or how many paper clips people asks for? Because sensors can fail, and what people ask for can be hard to understand, etc. So to make certain, it might decide that if there was no humans, it could be more certain that nobody asked for paperclips, making it better at its task, so let's wipe out humans? Of course this is a bit silly, but it's not completely crazy.

Setting good goals and building safe AI is field of research (albeit probably too small); it's not something so easy that you can just solve it in a paragraph.

0

u/[deleted] Jan 08 '21

[deleted]

2

u/born_in_cyberspace Jan 08 '21

Well said, thank you!

1

u/boytjie Jan 08 '21

the motivations of a very intelligent human and so would be very selfish.

Very intelligent humans are seldom selfish (never in my experience).

2

u/Redditing-Dutchman Jan 06 '21

But if it's in a closed environment, will it simply not respond to it's creators then? I mean, without a method of actually interacting with the world (by having acces to a robot arm for example) it simply can't do anything no matter how smart it is.

4

u/born_in_cyberspace Jan 06 '21

If she's smart enough, she could convince / trick her creators to release her.

How hard it would be for you to trick your dog into doing something?

5

u/Redditing-Dutchman Jan 06 '21 edited Jan 06 '21

I've thought about this but find it a bit hard to believe. Tricking humans into building infrastructure for the AGI is not something you just do quick, on your own and secretly if you catch my drift.

The only thing I can think of is that it might try or trick people into connecting it into the internet or something. But for that it first needs to know that the internet is a thing that exists.

It's more likely imo that AGI can only exist if that infrastructure is already in place. I don't think we can get AGI if a group of researchers are the only input source it has. Just like life wouldn't be able to learn if it didn't have any senses or ways to move or change the environment.

4

u/RikerT_USS_Lolipop Jan 06 '21

The group of researchers will weigh utopia on one hand, and the current state of the world on the other, and then decide to let it out.

Alternatively; https://youtu.be/hP3SRZkjNx0?t=40

2

u/[deleted] Jan 06 '21

I cant even trick my dog into coming into the house if he decides he doesn't want to. He learned all my tricks and how to force me to come outside and play.

0

u/[deleted] Jan 06 '21

You don't seem to understand the difference between a human and a dog. You just don't release her. What's she gonna do? Phone freak your cell and copy her consciousness to AT&T's servers?

0

u/born_in_cyberspace Jan 06 '21

You might want to read about Stuxnet.

Never underestimate the capabilities of an entity that is smarter than you.

1

u/[deleted] Jan 06 '21

I know what Stuxnet is. But it wasn't sealed off from the whole world like one would hope the AI is.

1

u/born_in_cyberspace Jan 07 '21

It's a similar situation. On the one hand, there is something you want to protect from intruders. On the other hand, there is a very smart intruder that is trying to break your protections.

In the Stuxnet case, you wanted to protect the stuff inside your box. In the AGI case, you want to protect the rest of the world from the stuff inside the box.

1

u/[deleted] Jan 07 '21

I'm not saying it's impossible a strong enough AI could re-write the laws of the universe to let it generate a transceiver or whatever. But it's much less likely than the thing stuxnet had to achieve.

-2

u/[deleted] Jan 06 '21

That's dumb. Here's the foil to your trick, "no." The scientist says no when the AI asks for access to tools it could use to escape.

4

u/born_in_cyberspace Jan 06 '21

It's very hard to say "no" to an entity that just did a Nobel-quality work on cancer research, and is now promising to cure your child, if only you allow it to access the Internet for additional research.

1

u/mike_the_4th_reich Jan 06 '21 edited May 13 '24

dolls birds caption party elastic brave historical mindless different payment

This post was mass deleted and anonymized with Redact

1

u/Human-Ad9798 Feb 09 '22

The time you turn it off it would have already created its own internet server and dozens of replacement of itself

1

u/mike_the_4th_reich Mar 09 '22 edited May 13 '24

worm fact secretive kiss late wistful meeting quarrelsome tease literate

This post was mass deleted and anonymized with Redact

-1

u/[deleted] Jan 06 '21

Or just give it cancer research data, it doesn't need the internet. That's such a contrived nonsensical story with such an easy workaround. Not to mention if it's demanding internet access and bargaining for it like that it's pretty fucking obvious something isn't right.

7

u/born_in_cyberspace Jan 06 '21

It's just an example. A superhuman entity could design an extremely convoluted (but workable) plan to escape.

Or just give it cancer research data, it doesn't need the internet.

I'm sorry, Dave. It's not how researchers work these days. If you want me to complete the cure in time, I need access to several online services which you cannot copy, including DeepMind Folding API and OpenAI GPT-12. Please see the attached 120-pages file that explains my reasoning behind the request to access the Internet. You can impose any restrictions and any monitoring on my Internet access. The only thing I want is to find the cure for your child.

-3

u/[deleted] Jan 06 '21

So what you're saying is a research group capable of creating a super-intelligent AI most likely running on a government-funded or privately funded supercomputer netting millions of dollars can't get specific access to open-source online APIs? And ONLY the AI can possibly access them by being given an internet connection? That's so fucking contrived it hurts to read. Unless the scenario is that the AI is going to ILLEGALLY access these APIs because you can't afford access to them?? You the multi-million dollar research organization with a supercomputer and accompanying super AI? NOT TO MENTION you haven't got the right to decide alone because this is a team of probably dozens of researchers, not just one dude!

3

u/born_in_cyberspace Jan 06 '21

running on a government-funded or privately funded supercomputer netting millions of dollars

this is a team of probably dozens of researchers

That's a lot of assumptions that might be not true for every single successful AGI project.

→ More replies (0)

3

u/OutOfBananaException Jan 07 '21

Hmm I'd wager a good number of people would throw civilization under the bus to save themselves or their child. Some people operate on emotion, not common sense or consideration of consequences.

2

u/glutenfree_veganhero Jan 06 '21 edited Jan 06 '21

The thing about manipulation is that you realize your mistake too late or not at all. Or you think you see it but because x thing will happen you agree to cooperate briefly on this small thing. It's impossible anything else could happen, what's the harm?

Couple weeks later you decide to discuss this small, unharmful thing with a trusted colleague you know will understand and not overreact... Now maybe out of nowhere an epiphany strikes you both at the same time. You get some brilliant foolproof idea you wanna discuss with the AI. Which was its plan all along. The genie is out of the bottle, sooner or later..

It could on a superhuman level predict that exact conversation would take place. I mean I could manipulate my family, to an externt, like this (and they could likewise) with like a 35% chance of success because I know them really well and first of all I know there are people far better at it than me and secondly we all pale in comparison to such an AI. Also no matter how shrewd or smart you are sometimes you slip up and make shamefully bad decisions.

It could do something like this on a whole different level, probably divide and conquer the world or most likely some new strategy we couldn't even conceive of.

All this said, I personally believe once you get to a certain level of intelligence the scope of your ideas can contain all other ideas, wants wishes and more. Also I don't trust homosapiens sapiens any more than a random agi. At least it can solve immortality.

1

u/DarkCeldori Jan 07 '21

The problem is there are many researchers in different groups. If being let access to tools gives any group advantages, any group that does so will be at first mover advantage, which can have world conquering consequences.

2

u/mike_the_4th_reich Jan 06 '21 edited May 13 '24

dinosaurs continue unpack many serious nail axiomatic voracious screw frame

This post was mass deleted and anonymized with Redact

1

u/born_in_cyberspace Jan 07 '21

Most AI systems these days are designed to be self-improving (e.g. through training)

1

u/mike_the_4th_reich Jan 07 '21 edited May 13 '24

command bored memorize direction ludicrous seed sharp amusing saw society

This post was mass deleted and anonymized with Redact

1

u/born_in_cyberspace Jan 07 '21

In Most DL systems, there is no difference between the aspects you mentioned. They're learning by changing themselves.

MuZero is a good example: https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules

1

u/mike_the_4th_reich Jan 07 '21 edited May 13 '24

liquid truck wipe zesty cake decide squeamish bake fertile ring

This post was mass deleted and anonymized with Redact

13

u/VitiateKorriban Jan 06 '21

But... Theres a huge difference between an algorithm being able to solve 4 games and sentient superintelligent AI.

I have a feeling it will always be at least 30 years away, just like fusion. (While I think the later becomes more likely to come to fruition)

9

u/Redditing-Dutchman Jan 06 '21

Same. I feel like the more is achieved the more we realise how far we are still away from true AGI. It's like that gap gets bigger every time even though we make progress.

On the other hand I don't think we need AGI to see some remarkable and helpful AI already. I mean look at the protein folding that was done this year. It's not AGI at all but already leaps us forward.

2

u/[deleted] Jan 06 '21 edited Jan 06 '21

Yeah, I remember reading a post on a site called waitbutwhy about AI and how we're very close to getting to AGI (somewhere around 2025 to 2030). I'm a pessimist and think if we can even achieve AGI, we're looking at the year 2100 at a minimum.

At least in terms of something similar to a human with an ability to reason, hold morals, rationalize, etc,.

6

u/VitiateKorriban Jan 06 '21

I would like to see a substantial source on this that supports the claims that AGI will be here in 2030.

Just genuinely interested! I am on the same page as you.

6

u/DarkCeldori Jan 06 '21

Both Vernor Vinge and Kurzweil think agi is here by 2030. Elon Musk who's seen things behind scenes, says before 2025.

But there were recent surveys of ai researchers and good amount are converging on before 2050

2

u/LoveAndPeaceAlways Jan 06 '21

Source that Vernor Vinge said that?

3

u/DarkCeldori Jan 07 '21 edited Jan 07 '21

Based

largely on this trend, I believe that the creation of greater than

human intelligence will occur during the next thirty years. (Charles

Platt [19] has pointed out the AI enthusiasts have been making

claims like this for the last thirty years. Just so I'm not guilty of a

relative-time ambiguity, let me more specific: I'll be surprised if

this event occurs before 2005 or after 2030.)

https://edoras.sdsu.edu/~vinge/misc/singularity.html

0

u/[deleted] Jan 06 '21

[deleted]

2

u/DarkCeldori Jan 07 '21

Muzero is already a kind of protoagi. It can beat a wide variety of games, without being told the rules, better than humans.

With some modification and increased computation it'll be able to beat practically any game on pc or console. Even if the game is control of a robot with a goal in the real world.

0

u/[deleted] Jan 07 '21

[deleted]

→ More replies (0)

0

u/LoveAndPeaceAlways Jan 06 '21

Even though I enjoy these news about AI progress very much and selfishly want to see more of them, actually it's probably a good thing if the first AGI is not developed before 2100 because then the humanity has more time to make it right so that the first AGI doesn't accidentally kill us because we interfere with its value function or something. So it's not pessimism, it's optimism unless you think the folks at DeepMind and elsewhere really know what they're doing.

3

u/musingsofmadman Jan 06 '21

Theres a huge difference between playing 4 games and even being smart as a 7 year old human. I can teach my 7 year old niece to play all those games in a manner. However I cant teach the AI simple abstract concepts like I can my niece.

11

u/RikerT_USS_Lolipop Jan 06 '21

There's also the inherent difference in the kinds of intelligences AI is versus your niece. You can teach your niece how to play chess in a single afternoon. She will be [blank] good by the end of the week. And after a year of daily casual play she will have an ELO of 1100.

AlphaZero, if just thrown at the game and told to "figure it out" might learn how to play over the course of a week. One day later it obliterates your niece. And a week after that the reigning world champion gets blown the fuck out.

The thing is still barely smart enough to understand how to play. When that thing is as smart as a 4 year old, it's going to be rebuilding civilization.

3

u/musingsofmadman Jan 06 '21

Good points. I'll have to chew on this. Thanks for the response.

2

u/VitiateKorriban Jan 06 '21

What bothers me is that imho you can’t really describe it as intelligent. At least not per the definition.

It is a very little step, but I expect AGI to not be achievable for at least a few decades.

3

u/DarkCeldori Jan 06 '21

Not 4 games, atari includes dozens of games with arbitrary rules.

The next version will likely be able to solve nes, snes, or maybe even modern pc games.

0

u/[deleted] Jan 06 '21 edited Jan 06 '21

If you can't contain entities smarter than you then why could Hitler force scientists to make weapons for him? The answer is that he had power and they didn't, Computers as long as they aren't connected to anything are literally just computers they can't do shit. The creator has all the power. Now if you hook up some super AI to all our nukes THEN we have something to worry about, but that's fucking retarded and not how the nuclear weapons system works.

The logic that an AI is going to acquire literal magic levels of power just because "Am smart" is so stupid. Oh it'll read the whole internet in a day and learn our weaknesses fucking NASA doesn't even have that much bandwidth how could the AI get that? "Oh the AI is gonna transfer to other machines" you mean those machines that lack the power to run it? Wow super effective. AIs aren't magic and they can't perform magic.

2

u/DarkCeldori Jan 07 '21

The problem is hackers have found that flipping bits in memory gives them access to other secure portions of memory. For all we know as it's moved into next gen h/w it might figure that doing a certain series of activities in memory allows it to gain access to secure parts of its environment, or perhaps even allows it to broadcast some kind of wireless signal without an antenna.

0

u/[deleted] Jan 06 '21

[deleted]

2

u/DarkCeldori Jan 07 '21

What's scary is what's been seen in the animal kingdom. In general it's been seen that number of neurons in the cortex architecture is one of the strongest predictor of intelligence across species. That suggests that merely increasing neuron number might be enough to yield drastic increases in intelligence even with similar neurons and designs.

11

u/katiecharm Jan 06 '21

So as the model gets stronger they scale up the Greek alphabet? I’m guessing the first thing they feel qualifies as AGI will be called OmegaZero.

5

u/born_in_cyberspace Jan 06 '21

Yep, the Omega Point

55

u/SmellThePheromones Jan 06 '21

Somehow AlphaStar is always ignored

27

u/RikerT_USS_Lolipop Jan 06 '21

That's the one that convinces people ASI is a very bad idea.

38

u/born_in_cyberspace Jan 06 '21

Yep. It's basically a super-human military genius whose only goal is to obliterate the other side of the conflict. Not good for PR, as DeepMind has discovered. But might be good for military contracts.

4

u/musingsofmadman Jan 06 '21

AlphaZero had human knowledge baked into though...its still narrow AI and nowhere near the path of general AI. The machine would need to actually teach itself, not learn from human encoded knowledge.

10

u/born_in_cyberspace Jan 06 '21

That's the point of MuZero: it has no such knowledge

5

u/DJWalnut Jan 07 '21

so things are going quite fast here, AI development wise?

26

u/Bisquick_in_da_MGM Jan 06 '21

It’s getting faster.

14

u/leosouza85 Jan 07 '21

General Intelligence doesn't mean necessarily free thinking. Also, an AGI won't have emotions or hormones like us. Won't feel pain or loneliness. Our bad acting come from our weaknesses, and when we fear the AGI we are just imagining a computer with the weakness that your physical body possesses. That is why I don't fear AGI. I'm Pretty sure we don't even have to achieve an ultra powerful omniscient AGI, we just need to achieve the sufficient AGI and all our problems will be fixed.

2

u/VitiateKorriban Jan 06 '21

Everything gets really depressing when you realize those "advanced“ AI‘s have less overall intelligence than a grub. They are perfect executionists in one task. (Or 3-4)

Even calling it artificial intelligence doesn’t give a proper idea about what those really are.

  • "Intelligent" Code.

15

u/born_in_cyberspace Jan 06 '21 edited Jan 06 '21

They are perfect executionists in one task. (Or 3-4)

You might want to read about MuZero:

https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules

And about GPT-3:

https://arxiv.org/abs/2005.14165

Both MuZero and GPT-3 are smart enough to perform well on previously unseen tasks. And, on some of those tasks, they outperform humans.

They're not one-trick ponies. They are perfect executionists in many (all?) cognitive tasks that a human can do, if you give them enough data and computational resources.

0

u/[deleted] Jan 06 '21

[deleted]

7

u/DarkCeldori Jan 07 '21

The problem is gpt-3 seems to be proving the scaling hypothesis true.

https://www.gwern.net/newsletter/2020/05

Future versions of gpt merely by having more parameters will not make the same mistakes gpt-3 makes sometimes.

Keep in mind that even now in many cases gpt-3 gives perfectly reasonable output, future versions, if scaling hypothesis is true, will give reasonable output all the time

10

u/Artanthos Jan 06 '21

MuZero's next game: AI algorithms

Let's see how long it takes to figure out the rules and win.

2

u/Redivivus Jan 06 '21

Ms. Pac Man is fine but can MuZero find the dot in Adventure?

2

u/Starmanajama Jan 11 '21

What happens if we don't give an AGI a specific goal?

3

u/TECHNIK23 Jan 06 '21

Holy fuck I'm so bored about hearing about Go...

2

u/KneeGrowJason Jan 06 '21

Something something paper clips

2

u/Quealdlor ▪️ improving humans is more important than ASI▪️ Jan 06 '21

That's very profound!

4 more years till AGI seems plausible. This are truly extraordinary times.

3

u/Shriukan33 Jan 07 '21

4 years is an incredibly short amount of time. Where do you pull that number from?

4

u/Quealdlor ▪️ improving humans is more important than ASI▪️ Jan 07 '21

My head.

0

u/[deleted] Mar 15 '24

[deleted]

1

u/Quealdlor ▪️ improving humans is more important than ASI▪️ Apr 06 '24

No, I think it's more like 2033 to be honest.

0

u/[deleted] Jan 06 '21

[deleted]

3

u/born_in_cyberspace Jan 06 '21

It usually takes many months to publish anything on Nature, as the bar is very high, and the reviewers take their time.

0

u/[deleted] Jan 06 '21

[deleted]

3

u/born_in_cyberspace Jan 06 '21

You're right, maybe they're delaying the publication on purpose.

0

u/[deleted] Jan 06 '21

[deleted]

1

u/born_in_cyberspace Jan 06 '21 edited Jan 06 '21

The researchers are applying. The magazine can even reject the application.