What is more important: IQ or mechanics?

[Garja]

It's just a matter of time

[Jaeger]

Show hidden quotes

I watched the entire deepmind presentation of alphago and that was still a form of brute force. No it does not calculate all possibilities, but it still has an insane amount of information and plays an insane amount of games versus itself. In that sense its still brute force; its basically solving a puzzle by trying all possibilities.

Ye the learning process is done that way. How would a machine learn a set of complex rules from scratch by itself?

I dont quite know what you are trying to say. My point is that AI shouldn't be praised to much because it can literally only trial and error games at the moment. For chess you can calculated everything, for Go trial and error seems like an excellent thing to do as the amount of possibilities is still limited and all information is available. But how the fuck would you literally trial and error a game like aoe3, where there are a million of options that all require different strategic decisions. Again the AI will probably learn some things, but as shown by the Dota bot the AI currently is not able to actually innovate strategically. We humans are still superior in that aspect.

Trial and error is not trivial. It's the same way humans learn things. The difference between deep learning and previous machine learning techniques is that now AIs are capable of learning and improving over time by themselves. Previously AI worked with just the subset of rules given by humans.
The AI that is challenging humans in Dota already won vs mid tier players. Now it is being improved to try win vs pros. Eventually they it will beat pros, it's just a matter of time.

Of course trial and error is not trivial. But humans do not just learn through trial and error! Some of the greatest breakthrough's I've had in aoe3 where things I thought of in the shower. I won a series against sircallen because I thought of a strategy while lying on the beach. That's an unique human feature: The ability to actually truly understand the game and use that knowledge to solve problems in that game. The AI can't do this, it can only do trial and error. It's very impressive how good the AI can do trial and error, and I'm sure anything trial and error related will be a thing that AI can get better at than humans.

However, what I am pointing out is that solving strategic concepts using trial and error is still extremely hard. I'm nuancing the AI's abilities to actually solve a game like aoe3 with trial and error. Like I tried to explain, alphaGo used some neat tricks for Go that aren't necessarily machine learning related. It just gained a database of winning board states and I believe it's starting point was to build towards these winning board states and then learned how to effectively build towards those states. This is not something that can easily be applied to an RTS game, because there are no discrete board states but rather there is a continuous spectrum of states. That means the AI has nothing to gain from building towards a state in a specific recorded game by precisely positioning units in a certain way because that isn't really relevant in an RTS game. Despite that, the AI did prove to be reasonable at dota. However, it's only reasonable. I watched those games, that AI excells at anything that humans do intuitively. But it is also really bad at the strategic side of things.

If we consider this knowledge, we should conclude that the AI is quite far away from solving aoe3 strategically. Especially interesting is having a bot that has limited mechanics learn to solve a strategic game like aoe3. You can't abuse a trick like what was used for Go to lay a foundation and in aoe3 strategic understanding of the game is really important. Army movement tactics and such - things that are more intuitive and things that humans also mostly learn through trial and error - are not nearly as important in aoe3. This should mean that for an AI to excell at aoe3 it needs to do something that AI hasn't done before: actually trial and error so well to the point where it starts solving strategic concepts. But pure trial and error is a brute force technique. I don't think the current capabilities of this AI can brute force aoe3's strategy yet. Right now the human brain is more competent, because we can solve strategic concepts while lying on the beach or lying on a bed thinking about the game or by arguing about the game on a forum.

But when you have an idea on the beach, aren't you still doing trial and error, but just in your head? I want to elaborate more but I kindda need to know what the idea that came to you was.

[BrookG]

Even with an immense amount of time if you "teach" the incorrect variables and give weights at the wrong features, the AI will be unable to solve the problem of playing optimally the RTS. In chess you can determine the movements of every piece and calculate the possibilities of them moving per situation. Plus calculate future moves and consider whether they open the way to victory. I am not quite sure if AlphaZero uses HMM or what other method.

[Garja]

Even with an immense amount of time if you "teach" the incorrect variables and give weights at the wrong features, the AI will be unable to solve the problem of playing optimally the RTS. In chess you can determine the movements of every piece and calculate the possibilities of them moving per situation. Plus calculate future moves and consider whether they open the way to victory. I am not quite sure if AlphaZero uses HMM or what other method.

Again, deep learning AIs are not given any variable or weight aside from the very basic rules of the game itself.
For chess this meant the size of the board, what the legal moves of pieces are and the winning condition (checkmate). For RTS games this would mean unit stats, mechanics, terrain attributes, other variables like those and, finally, winning condition. Anything in between is learned by the AI itself by playing agaisnt itself. Alphazero figured out the game of Chess by itself, from scratch.
By providing the mentioned basic inputs, it is still entirely possible to determine the movement of each unit and calculate all variants for every instant in RTS games. It's just on a much larger scale than in a turn based game (real time is still a sequence of turns in the end). But since it took only 4hrs for Alphazero to reach a level necessary to destroy the former best AI (this one already unbeatable for humans) it will probably take something like 5-10 years to solve a game like Dota or at least reach a level that is enough to beat the best human players. Also, perhaps, it would need other qualitative methods of learning to complement its main way of learning the game.
From what I've read AlphaZero uses MCTS driven by a Convnet trained with renforcement learning.

[BrookG]

By variables I mean the features that you need to feed the AI, but in some situations the basics are not always enough. By adding extra rules or norms of how the game should be played may not lead to expected results. Then again it misses the purpose of an AI playing the game itself because the human factor is even more significant if that is the case.

[Garja]

Well make an example of variables you need to feed the AI with. Right now I can't think of anything that is necessary to give the AI other than the basics.

[BrookG]

Ι'm giving importance to that point, because I've been working on a dataset that is going to train a system, and making choices about it, is more complicated in other situations unlike chess or games with a simple set of rules.

[ListlessSalmon]

Well Alphazero (and the other thing they made to play lots of other games like 2d atari games etc) learnt by making random moves and then finding patterns from what resulted in victory (or score in 2d games case), so it played a really really large number of games but still an infinitesimal fraction of the total space of possible games (so the prior description of "brute force" is wrong- though its possible he just mean "lots of processing power").

But with something like aoe3 presumably both the number of possible "turns" and the number of possible individual moves per "turn" (think clicking anywhere on the screen as well as pressing every possible button in any combination of that) seems extremely vast and so it seems like it might be harder to apply to that, in addition you have that it is a limited information game which makes things more complicated.
I'm (obviously) nothing like an expert on this but I had seen experts on this saying (in relation to starcraft) that probably some substantial new developments would be needed for competent self-taught AIs in complex limited information rts games, so presumably it would be quite a ways off. Though the consensus until just before AlphaGo was that Go was too hard as well so who knows.

I'm sure it will come eventually (assuming no catastrophic collapse of human civilisation or similar) as I'm pretty sure we'll get AGI at some point and so necessarily a competent rts AI.

[Garja]

Ι'm giving importance to that point, because I've been working on a dataset that is going to train a system, and making choices about it, is more complicated in other situations unlike chess or games with a simple set of rules.

I'm not by any mean an expert but I think there are several ways deep learning works?! I mean, for some applications it seems they need to fuel the machine with data (for financial models for example) but for Chess it basically built its own dataset by playing against itself. So Idk, really.

[BrookG]

Ι'm giving importance to that point, because I've been working on a dataset that is going to train a system, and making choices about it, is more complicated in other situations unlike chess or games with a simple set of rules.

I'm not by any mean an expert but I think there are several ways deep learning works?! I mean, for some applications it seems they need to fuel the machine with data (for financial models for example) but for Chess it basically built its own dataset by playing against itself. So Idk, really.

Other than the "rules" you have to give a winning or losing premise; what is a good or a bad move. This is a general methodology. Then you have models of probability, evolutionary, grammar-based etc. For complicated RTS games the data is more difficult to pin for them to fuel the machine. As for the creativity you were discribing @momuuu, indeed computers aren't very creative and need to follow guidelines. We are superior in the sense that we can combine much more inputs than computers can. Our imagination is the limit. It is an interesting mind exercise, to learn our own capabilities and see how our brains work.

[Garja]

Ι'm giving importance to that point, because I've been working on a dataset that is going to train a system, and making choices about it, is more complicated in other situations unlike chess or games with a simple set of rules.

I'm not by any mean an expert but I think there are several ways deep learning works?! I mean, for some applications it seems they need to fuel the machine with data (for financial models for example) but for Chess it basically built its own dataset by playing against itself. So Idk, really.

Other than the "rules" you have to give a winning or losing premise; what is a good or a bad move. This is a general methodology. Then you have models of probability, evolutionary, grammar-based etc. For complicated RTS games the data is more difficult to pin for them to fuel the machine. As for the creativity you were discribing @momuuu, indeed computers aren't very creative and need to follow guidelines. We are superior in the sense that we can combine much more inputs than computers can. Our imagination is the limit. It is an interesting mind exercise, to learn our own capabilities and see how our brains work.

I can't help but it looks to me that the way you are describing is more typical of other AIs like former chess machines where devs were to give them inputs about the intrinsic value of a piece, and stuff like that so they could tell if a move is good or bad.
Deep learning AI does it by its own, in fact Alphazero is completely capable of giving up lot of material in chess and considering it good, without actually the need of a concrete forcing line that justifies it.
Basically deep learning machines understand what's good and what's bad solely based on the initial rules and the winning condition (or payout scheme), both of which are known info.

[BrookG]

Basically deep learning machines understand what's good and what's bad solely based on the initial rules and the winning condition (or payout scheme), both of which are known info.

I was thinking that a move that leads to the winning condition is a good move. Consider flappy bird AI: you can say that a bad move is if you hit the pipe. Ofc rts games are more fluid and there are several ways to define a winning condition

[n0el]

That's what deep learning does though, there's one win condition and it learns the moves that lead to that.

[momuuu]

Well Alphazero (and the other thing they made to play lots of other games like 2d atari games etc) learnt by making random moves and then finding patterns from what resulted in victory (or score in 2d games case), so it played a really really large number of games but still an infinitesimal fraction of the total space of possible games (so the prior description of "brute force" is wrong- though its possible he just mean "lots of processing power").

But with something like aoe3 presumably both the number of possible "turns" and the number of possible individual moves per "turn" (think clicking anywhere on the screen as well as pressing every possible button in any combination of that) seems extremely vast and so it seems like it might be harder to apply to that, in addition you have that it is a limited information game which makes things more complicated.
I'm (obviously) nothing like an expert on this but I had seen experts on this saying (in relation to starcraft) that probably some substantial new developments would be needed for competent self-taught AIs in complex limited information rts games, so presumably it would be quite a ways off. Though the consensus until just before AlphaGo was that Go was too hard as well so who knows.

I'm sure it will come eventually (assuming no catastrophic collapse of human civilisation or similar) as I'm pretty sure we'll get AGI at some point and so necessarily a competent rts AI.

In what sense is huge trial and error not still brute force? If I ask you to solve a puzzle to get some 5 digit code, you can either solve the puzzle or just try all options. I do agree that the AI's trial and error is quite efficient and it actually does some learning, but it's still relative brute force; it doesn't actually simplify situations that much. But I think you agree with that but don't quite agree on the terminology, because your conclusion is the same as mine; aoe3 is still too complex right now for such an AI to solve.

Of course the problem with Go in the past was that older AI's didn't work intuitively but by either trying to calculate everything or by hardcoding some rules. Hardcoding rules is just not great, but for Go calculating everything is actually virtually impossible. It required this breakthrough of a learning algorithm to solve. However, I'd say that for a game in continuous space rather than discrete space it might be extremely hard to push the alpha-go technology to the point where it can solve this sort of game. And then there's still the problem that this deeplearning hasn't actually showcased strategic insight yet. That's interesting too.

[ListlessSalmon]

In what sense is huge trial and error not still brute force? If I ask you to solve a puzzle to get some 5 digit code, you can either solve the puzzle or just try all options. I do agree that the AI's trial and error is quite efficient and it actually does some learning, but it's still relative brute force; it doesn't actually simplify situations that much. But I think you agree with that but don't quite agree on the terminology, because your conclusion is the same as mine; aoe3 is still too complex right now for such an AI to solve.

Let me just take a standard definition of brute force in relation to this area:

"In computer science, brute-force search or exhaustive search, also known as generate and test, is a very general problem-solving technique and algorithmic paradigm that consists of systematically enumerating all possible candidates for the solution and checking whether each candidate satisfies the problem's statement."

It should be immediately clear that what we're talking about is not brute force.

To go to your puzzle combination description, its worth considering the number of games of Go AlphaZero played against itself as a fraction of the total possible games of go. I do not know the precise values for either of these numbers, but the most commonly stated value for the possible games of Go is 10^700.

Assuming your 5 digit code is just 10 number digits as a possibility for each digit thats 10^5=100,000 possibilities. So if the computer in your scenario tried precisely one combination it would have tried 1/10^5 of the search space.

If we take the 10^700 value as correct, then for AlphaZero to have looked at a higher fraction of the search space than trying precisely one combination in your scenario it would need to have played at least 10^695 unique games of Go. I think you would agree that it did not do that. So as it seems quite hard to describe a method to solving your problem by trying one and only one combination as "brute force" it seems quite strange to describe to describe AlphaZero as "brute force".

Probably a better way to describe it is as a relatively slow (at learning) person (who is nonetheless very good at calculation) learning Go by themselves in a hyperbolic time chamber. So I agree that a large amount of learning games are necessary but brute force seems a terrible way to describe it.

On an unrelated note I think for a futuristic aoe3 AI to actually be interesting for human gameplay you'd need some kind of limitations on its mechanics, if it had perfect unit control for every unit (and the ability to control every single unit individually if necessary) I would imagine gameplay would be rather different than human gameplay. Kinda like how sc2 bots with perfect mechanics can do crazy stuff like this:

https://www.youtube.com/watch?v=3PLplRDSgpo

[Garja]

And then there's still the problem that this deeplearning hasn't actually showcased strategic insight yet. That's interesting too.

In chess and in Go Deepmind's AIs showed plenty of strategic insight. This is why human players consider it revolutionary and are trying to learn from the way AlphaZero and AlphaGo play.

[Garja]

On an unrelated note I think for a futuristic aoe3 AI to actually be interesting for human gameplay you'd need some kind of limitations on its mechanics, if it had perfect unit control for every unit (and the ability to control every single unit individually if necessary) I would imagine gameplay would be rather different than human gameplay. Kinda like how sc2 bots with perfect mechanics can do crazy stuff like this: [spoiler=]https://www.youtube.com/watch?v=3PLplRDSgpo[/spoiler]

Ahah that video is hilarious. But I think the problem there is that most attacks have a projectile instead of doing instant damage.
Also SC2 is full of units that don't necessarily counter each other but can beat each other depending on micro (e.g. marines vs banelings). AOE3 mostly employs big damage multipliers for its counter system.

[[Armag] diarouga]

Haha lol at that sc2 video.

[momuuu]

In what sense is huge trial and error not still brute force? If I ask you to solve a puzzle to get some 5 digit code, you can either solve the puzzle or just try all options. I do agree that the AI's trial and error is quite efficient and it actually does some learning, but it's still relative brute force; it doesn't actually simplify situations that much. But I think you agree with that but don't quite agree on the terminology, because your conclusion is the same as mine; aoe3 is still too complex right now for such an AI to solve.

Let me just take a standard definition of brute force in relation to this area:

"In computer science, brute-force search or exhaustive search, also known as generate and test, is a very general problem-solving technique and algorithmic paradigm that consists of systematically enumerating all possible candidates for the solution and checking whether each candidate satisfies the problem's statement."

It should be immediately clear that what we're talking about is not brute force.

To go to your puzzle combination description, its worth considering the number of games of Go AlphaZero played against itself as a fraction of the total possible games of go. I do not know the precise values for either of these numbers, but the most commonly stated value for the possible games of Go is 10^700.

Assuming your 5 digit code is just 10 number digits as a possibility for each digit thats 10^5=100,000 possibilities. So if the computer in your scenario tried precisely one combination it would have tried 1/10^5 of the search space.

If we take the 10^700 value as correct, then for AlphaZero to have looked at a higher fraction of the search space than trying precisely one combination in your scenario it would need to have played at least 10^695 unique games of Go. I think you would agree that it did not do that. So as it seems quite hard to describe a method to solving your problem by trying one and only one combination as "brute force" it seems quite strange to describe to describe AlphaZero as "brute force".

Probably a better way to describe it is as a relatively slow (at learning) person (who is nonetheless very good at calculation) learning Go by themselves in a hyperbolic time chamber. So I agree that a large amount of learning games are necessary but brute force seems a terrible way to describe it.

On an unrelated note I think for a futuristic aoe3 AI to actually be interesting for human gameplay you'd need some kind of limitations on its mechanics, if it had perfect unit control for every unit (and the ability to control every single unit individually if necessary) I would imagine gameplay would be rather different than human gameplay. Kinda like how sc2 bots with perfect mechanics can do crazy stuff like this:

https://www.youtube.com/watch?v=3PLplRDSgpo

Okay far enough. I'd still say that on the spectrum from human brain to actual brute force this is still pretty far from the human brain. It still does rely heavily on the capability of machines to very quickly process data and has a low amount of actual learning capability. I wasn't quite aware of the precise brute force definition, which makes it so that I can't legitemately call this brute force. But it's definitely still something similair.

[momuuu]

Btw, I think deepmind is set to solve sc2 with a real apm limitation. I think this is their concession that they are unable to solve this precise strategic thing yet. I think nobody doubts that deepmind could currently get really good at sc2 if there was no limit on its apm, and even with high apm it can probably outmicro almost all players. It's thus more interesting to try to force the AI to actually do good strategic things. I am curious what it brings, but like I said I'm also scared that this development will solve RTS games and take away some of their magic.

[[Armag] diarouga]

Btw, I think deepmind is set to solve sc2 with a real apm limitation. I think this is their concession that they are unable to solve this precise strategic thing yet. I think nobody doubts that deepmind could currently get really good at sc2 if there was no limit on its apm, and even with high apm it can probably outmicro almost all players. It's thus more interesting to try to force the AI to actually do good strategic things. I am curious what it brings, but like I said I'm also scared that this development will solve RTS games and take away some of their magic.

I think it is on sc1.

[ListlessSalmon]

Ahah that video is hilarious. But I think the problem there is that most attacks have a projectile instead of doing instant damage.
Also SC2 is full of units that don't necessarily counter each other but can beat each other depending on micro (e.g. marines vs banelings). AOE3 mostly employs big damage multipliers for its counter system.

Right its very very different because of the game differences, but just in terms of trying to understand things about the game as played by humans a perfect AOE3 bot without mechanically limitations there would still be things done differently that would not really give useful information to humans.

Just as a motivating example consider like a scenario where there is a decision point of sending some upgrade card for the attack of some ranged units (where importantly some enemy unit will now die in one less shot) or some other card. Presumably a computer with genuinely perfect mechanics would value the upgrade card higher than a human in that scenario would, because the human is going to waste some of it to overkill, whereas the computer won't. That's obviously a very minor point but if it was a close situation and the AIs micro advantage makes the difference in the decision then that wouldn't tell humans "the upgrade was the right decision there" but "if your control is perfect that was the right decision there".

Being bad at the game I can't come up with interesting specific examples but I'm sure some must exist, to just throw one possibility out, perhaps longbow would be stronger because a perfect mechanics AI would be able to entirely avoid the redoing the setup thing even with a very large longbow mass.

[momuuu]

Ahah that video is hilarious. But I think the problem there is that most attacks have a projectile instead of doing instant damage.
Also SC2 is full of units that don't necessarily counter each other but can beat each other depending on micro (e.g. marines vs banelings). AOE3 mostly employs big damage multipliers for its counter system.

Right its very very different because of the game differences, but just in terms of trying to understand things about the game as played by humans a perfect AOE3 bot without mechanically limitations there would still be things done differently that would not really give useful information to humans.

Just as a motivating example consider like a scenario where there is a decision point of sending some upgrade card for the attack of some ranged units (where importantly some enemy unit will now die in one less shot) or some other card. Presumably a computer with genuinely perfect mechanics would value the upgrade card higher than a human in that scenario would, because the human is going to waste some of it to overkill, whereas the computer won't. That's obviously a very minor point but if it was a close situation and the AIs micro advantage makes the difference in the decision then that wouldn't tell humans "the upgrade was the right decision there" but "if your control is perfect that was the right decision there".

Being bad at the game I can't come up with interesting specific examples but I'm sure some must exist, to just throw one possibility out, perhaps longbow would be stronger because a perfect mechanics AI would be able to entirely avoid the redoing the setup thing even with a very large longbow mass.

I honestly think with even better micro ranged units and ranged infantry in particular will become much stronger even. Right now you can kind of make cav, but if someone's going to perfectly pull track and never overkill anything then a composition like musk huss just wont work.

[ListlessSalmon]

Given the previous discussion in here its perhaps worth noting there's some kind of Deepmind sc2 livestream tomorrow hosted by Artosis and Rotterdam: https://www.teamliquid.net/forum/starcr ... ion-jan-24

No idea precisely what it will be, I would have assumed not something substantial (like a full 1v1 program capable of beating reasonable players) but who knows.

[momuuu]

Given the previous discussion in here its perhaps worth noting there's some kind of Deepmind sc2 livestream tomorrow hosted by Artosis and Rotterdam: https://www.teamliquid.net/forum/starcr ... ion-jan-24

No idea precisely what it will be, I would have assumed not something substantial (like a full 1v1 program capable of beating reasonable players) but who knows.

I just realized I really dont want them to have a bot that solved sc2. It takes away some of the magic.