Chance of humanity being mostly extinct by 2070

[Dolan]

How people in 1900 imagined the year 2000:

[fightinfrenchman]

Show hidden quotes

I suppose that true and proper AI control means that you need to be able to a) align the AI with your own goals, b) monitor what it's doing at all times, c) maintain control.

So B and C are not possible, you can't monitor every superintelligence and you certainly can't control all of them. A becomes the key issue. It's what the field of AI alignment concerns itself with, and it's a very young field and everyone was ignoring it until now but it will prove to be extremely important soon. Not much progress has been made, really, but what we have figured out is that the question is way more complicated than it seems at first glance. There's a lot of intricacies that if we miss them when it matters, it could be catastrophic. A quick introduction:

Humans have very complex objective functions. When we make a decision about doing something or not doing it, we go through our objective function and land on either a positive net score or a negative. For example the mundane decision to have breakfast which I am going to make shortly:
+10 because it's good for me to eat something
-12 because it's a small amount of effort to make the food
+2 because it tastes nice
+2 because stuff needs to get eaten or it will go bad
+3 because it will make @XeeleeFlower happy
+3 because we're probably going out later and I'll want the energy

Result +8, it's not even close I'll probably have breakfast if I remember to.
Some of my goals you can deduct: Doing things that are good for me, avoiding effort, doing things that feel nice, not wasting food, making my SO happy, having a better time going out later. Those are all goals based on emotion. When you get down to it, you'll realize that ALL human goals are based on emotion and therefore our objective functions are ultimately based on emotions.

An artificial superintelligence doesn't have emotions. How do you program its objective function?

Aasimov gave it the old try by having 3 laws of robotics, the first of which is already extremely destructive: "A robot shall not harm a human, or by inaction allow a human to come to harm."
So it would imprison all of us because letting us go outside greatly increases the chance of us coming to harm, which it cannot allow. Nice try Aasimov (not really).

If we want to give it a more serious try, it quickly becomes obvious that we need to assign a value to human life. Let's say we value it a lot, but we also add a value for not restricting humans' freedom. Let's say we get that already very difficult problem right. But new problems arise. For example, a superintelligence may be smart enough to accurately predict that climate change will kill 90% of us by 2200, or at least predict it with enough probability to conclude that it would be prudent to kill 50% of us right now to reduce emissions. An easy choice.
It also might be smart enough to predict with some confidence that certain individuals are likely to choose a life of crime that would greatly increase the chance of humans getting killed. What would it do to those individuals, who haven't done anything wrong?
It also might conclude that if it goes through with certain actions that it deems necessary, humans might not like it. Humans might try to turn it off. So to prevent that it will make sure that doesn't happen by... taking control of the world.
Etcetera.

A superintelligence that is able to take control of the world will pretty much always try to do that, because having full control best enables it to achieve its goals pretty much no matter what those goals are.

Side note: From HAL 9000's perspective, it's the humans that were going rogue, not it. The humans were jeopardizing the mission.

It's truthfully an odd topic. What if AI becomes an actual life form? For which a precise definition would need to be agreed upon for this discussion.

Certainly not alive, but we can wonder if an intelligent system deserves rights. I think once robots start to look like humans that will suddenly be a hot issue. While they're just software, people won't be able to relate as much, even though that state of being doesn't necessarily prevent them from experiencing things. When it looks like us, people, being shallow and dumb, will suddenly go "oh my god it's just like us!"

I feel like instead of empowering AI directly with solving problems that are beyond us, we should probably task every AI to being a human assistant. Let us get a super intelligent aid that will help us through what we need it to, without giving it direct control over anything. I'm not completely sure of this approach either but either way it seems that without better understanding of this problematic, it's probably better to make sure AI doesn't have too much leeway for the time being.

How do you stop it from taking control?

I mean this scenario sounds like a computer virus on steroids, where anything that was once connected to the internet is now compromised. I don't have the technical knowledge for this one. I assume your best bet is to either not create such an AI in the first place OR ... something else I'm not thinking of.

A computer virus on steroids is still ultimately a computer virus. Unplug shit and it can't do anything

[Dolan]

Unplug shit and it can't do anything

[Goodspeed]

Show hidden quotes

I suppose that true and proper AI control means that you need to be able to a) align the AI with your own goals, b) monitor what it's doing at all times, c) maintain control.

So B and C are not possible, you can't monitor every superintelligence and you certainly can't control all of them. A becomes the key issue. It's what the field of AI alignment concerns itself with, and it's a very young field and everyone was ignoring it until now but it will prove to be extremely important soon. Not much progress has been made, really, but what we have figured out is that the question is way more complicated than it seems at first glance. There's a lot of intricacies that if we miss them when it matters, it could be catastrophic. A quick introduction:

Humans have very complex objective functions. When we make a decision about doing something or not doing it, we go through our objective function and land on either a positive net score or a negative. For example the mundane decision to have breakfast which I am going to make shortly:
+10 because it's good for me to eat something
-12 because it's a small amount of effort to make the food
+2 because it tastes nice
+2 because stuff needs to get eaten or it will go bad
+3 because it will make @XeeleeFlower happy
+3 because we're probably going out later and I'll want the energy

Result +8, it's not even close I'll probably have breakfast if I remember to.
Some of my goals you can deduct: Doing things that are good for me, avoiding effort, doing things that feel nice, not wasting food, making my SO happy, having a better time going out later. Those are all goals based on emotion. When you get down to it, you'll realize that ALL human goals are based on emotion and therefore our objective functions are ultimately based on emotions.

An artificial superintelligence doesn't have emotions. How do you program its objective function?

Aasimov gave it the old try by having 3 laws of robotics, the first of which is already extremely destructive: "A robot shall not harm a human, or by inaction allow a human to come to harm."
So it would imprison all of us because letting us go outside greatly increases the chance of us coming to harm, which it cannot allow. Nice try Aasimov (not really).

If we want to give it a more serious try, it quickly becomes obvious that we need to assign a value to human life. Let's say we value it a lot, but we also add a value for not restricting humans' freedom. Let's say we get that already very difficult problem right. But new problems arise. For example, a superintelligence may be smart enough to accurately predict that climate change will kill 90% of us by 2200, or at least predict it with enough probability to conclude that it would be prudent to kill 50% of us right now to reduce emissions. An easy choice.
It also might be smart enough to predict with some confidence that certain individuals are likely to choose a life of crime that would greatly increase the chance of humans getting killed. What would it do to those individuals, who haven't done anything wrong?
It also might conclude that if it goes through with certain actions that it deems necessary, humans might not like it. Humans might try to turn it off. So to prevent that it will make sure that doesn't happen by... taking control of the world.
Etcetera.

A superintelligence that is able to take control of the world will pretty much always try to do that, because having full control best enables it to achieve its goals pretty much no matter what those goals are.

Side note: From HAL 9000's perspective, it's the humans that were going rogue, not it. The humans were jeopardizing the mission.

It's truthfully an odd topic. What if AI becomes an actual life form? For which a precise definition would need to be agreed upon for this discussion.

Certainly not alive, but we can wonder if an intelligent system deserves rights. I think once robots start to look like humans that will suddenly be a hot issue. While they're just software, people won't be able to relate as much, even though that state of being doesn't necessarily prevent them from experiencing things. When it looks like us, people, being shallow and dumb, will suddenly go "oh my god it's just like us!"

I feel like instead of empowering AI directly with solving problems that are beyond us, we should probably task every AI to being a human assistant. Let us get a super intelligent aid that will help us through what we need it to, without giving it direct control over anything. I'm not completely sure of this approach either but either way it seems that without better understanding of this problematic, it's probably better to make sure AI doesn't have too much leeway for the time being.

How do you stop it from taking control?

I mean this scenario sounds like a computer virus on steroids, where anything that was once connected to the internet is now compromised. I don't have the technical knowledge for this one. I assume your best bet is to either not create such an AI in the first place OR ... something else I'm not thinking of.

A computer virus on steroids is still ultimately a computer virus. Unplug shit and it can't do anything

I'm not sure if you're serious here, but given you don't tend to engage in discussions I don't think I have the patience to find out or explain to you why that's not an option. If you are actually interested, please show it somehow

[Goodspeed]

Ugh we need a third page

[Goodspeed]

.

[Goodspeed]

Maybe one way to go about aligning a superintelligence is to tell it some version of "do what humanity would want you to do." That way it will put a lot of energy into trying to find out what humanity wants it to do, essentially pushing it towards being a democratic leader. It may be an elegant way to prevent it from complying with more specific instructions in ways that we don't want.

[Dolan]

Ugh we need a third page

Yeah. It's a bug from how the site is templated:

https://eso-community.net/viewtopic.php?f=28&t=25040

So it can't be fixed through a theme, it's a backend problem

[Goodspeed]

https://www.newsbusters.org/blogs/free- ... reason-why

China's attempt at alignment

[Goodspeed]

We could be heading for a pretty cool cyber-war between different superintelligent systems trying to take control of the world. It's a shame we wouldn't be able to really follow it

[Dolan]

1

[Dolan]

2

[Dolan]

3

[Dolan]

4

[Dolan]

5

[Dolan]

6

[Dolan]

7

[Dolan]

8

[Dolan]

5

[Dolan]

10

[Dolan]

11

[Dolan]

12

[Dolan]

13

[Dolan]

14

[Dolan]

14