SC2 Deepmind Demonstration

This is for discussions about news, politics, sports, other games, culture, philosophy etc.
No Flag deleted_user0
Ninja
Posts: 13004
Joined: Apr 28, 2020

Re: SC2 Deepmind Demonstration

Post by deleted_user0 »

Mr_Bramboy wrote:Nice demonstration. Its APM wasn't human at all though. At some point during fights, you could clearly see AlphaStar blinking multiple stalkers individually in the same second while its APM spiked to 1500+. I think the biggest advantage these AIs have is their mouse accuracy, even with limited APM. As far as I understand it, AlphaStar can click anywhere on the screen whenever it wants to. That takes away both the burden of mouse accuracy and mouse speed. Imagine a CSGO AI with perfect mouse accuracy and you have what is basically a perfect aimbot. All in all, it's terrifying to see that AI has come so far that it clean sweeps a pro player even with limitations. The futuristic dystopia of robots taking over the world is becoming more likely with the day.


its actually interesting, when there wasn't any fight going on, you could see TLO spamming 1500 apm, while AS was using 100-200 apm max. But when there was a fight, you saw TLO's apm drop to 500, while AS ramped it up to 700. It's EPM must be way higher than that of a human, SC2 pro's especially love to spam clicks, but many of those are wasted.

AlphaStar was also restricted in the speed of its actions to keep it in line with human pros, such as having a 350 millisecond delay time between perceiving information and issuing commands. The AI also played with an average APM (actions per minute) of 280, compared to 390 for MaNa and 678 for TLO.

TLO and MaNa faced a number of different AlphaStar 'agents'—versions of the AI that trained through matches against each other in an internal league (combined with initial, imitation learning from human replays). With accelerated training, the agents were able to accrue around 200 years of real-time StarCraft II training over 14 days. The agents in the demonstration showed different preferences for strategies—for example one opted for mass Disruptors while others chose mass Blink-Stalkers.

While AlphaStar defeated both TLO and MaNa 5-0 in previous recorded matches, MaNa was able take a victory in a final, live exhibition game against a new agent. While the first ten agents agents were effectively able to 'see' the entire map at once (NOT a maphack—more akin to a max zoom-out), the new agent was given restrictions to mimic human player's field-of-vision limitations during a game. While this new agent still had an estimated MMR of over 7000, MaNa was able to defeat its mass-Stalker strategy with careful scouting and an overpowering army of Immortals.
United States of America XeeleeFlower
Retired Contributor
Xeelee Patron
Posts: 1650
Joined: Aug 28, 2016
Location: Netherlands

Re: SC2 Deepmind Demonstration

Post by XeeleeFlower »

I was disappointed that they didn't show the confidence of winning level in all the games. I thought that was really interesting. I would've loved to have seen how confident it was that it was going to win the game that they played live. When did it know that it was defeated?

Something that kept bothering me, that you all pointed out as well, was how they kept talking about the APM is lower than that of the pros, yet during battles, it's APM spiked to 1500. A human cannot micro at that level of efficiency.

Something I thought was really cool was how different versions had different strategies that they preferred.
Time is wise and our wounds seem to heal to the rhythm of aging,
But our past is a ghost fading out that at night it’s still haunting.

http://www.galactanet.com/oneoff/theegg_mod.html
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

So in the tenth game rotterdam even remarked the micro was not humanly possible. That was a bit unfortunate because it overshadowed the AIs strategical competence. In the end they failed to properly model human apm in my opinion. If they can improve this model by trying to analyze a human's actual epm this would be more interesting. I think it maybe needs to actually half its epm or so.
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

I don't think they even limited the APM at all. When asked about the limitations, they responded with a story that was supposed to show us that an APM limit "wasn't needed". There was a graph that showed A*'s average APM was not as high as the humans' average APM, which in their mind probably meant it wasn't necessary to limit it. It's a shame they weren't told, or ignored whoever did tell them, that this is no indication because the high APM spikes in battles, plus the fact that the AI's APM is 100% effective, actually means it is vastly outperforming humans in the mechanics department.

The fact that the AI currently has no "module" that decides how to spend its limited APM (since it isn't actually limited), they will need to build this still. That'll be some work, and will reset the agents' training back to zero. Still it should be quite doable, and with the feedback they are getting they should be able to figure out that it's needed.

Another issue is that any one agent would be exploitable. They need to build in an "anti-predictability" module that forces the AI to mix up its strategy. Or, ideally, the AI would learn that it has to change it up through reinforcement learning. After all, when playing against the same opponent thousands of times, they will start blind-countering you. Ideally that would happen in the training games, too. Their current way of training doesn't really support this, and I think they should switch to "playing against itself a gazillion times" training rather than this league where the agents play against other agents. But they probably have good reasons for doing it this way.

So yeah there's a lot of work to do still but the AI did show understanding of basic strategic concepts. That and the big difference between the agents that got 1 week of training and the ones that got 2 weeks, was the most promising part of the demo imo.
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

I was reminded of aoe2's AI though. That AI is hardcoded but very competent and strategically pretty good actually, without getting any resource hacks. If this AI was toned down apm wise, I'd start to wonder how much of an upgrade it actually was compared to a 'state of the art' AI (programmed by one dude I think).
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

2 things on that. First, the AoE2 AI doesn't stand a chance against top level players. It's not close. Second, there's a very big difference between hard coding strategic concepts and build orders, and having the AI learn them itself. The fact that the AI got this good without hard coding anything is the main point of this research.
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

Goodspeed wrote:2 things on that. First, the AoE2 AI doesn't stand a chance against top level players. It's not close. Second, there's a very big difference between hard coding strategic concepts and build orders, and having the AI learn them itself. The fact that the AI got this good without hard coding anything is the main point of this research.

Its to put this in perspective. How much more can this technology achieve than a hardcoded AI based on 20 year old software? This showcase didnt really show that this concepts is much better on a strategic level than just hardcoding strategies.

So put in perspective the self learning AI technology doesnt seem revolutionary in the context of RTS games yet. Its basically a mechanically amazing player with only okayish understanding of the game. We already knew AI could master micro mechanics for example. Based on this exhibition Id say that strategically you might be better off hard coding the AI to do certain strats. Compare that to AlphaGo, which achieved something truly remarkable in the AI space that cant be done with any other technology. Thats the point I was trying to make.
No Flag kami_ryu
Retired Contributor
Posts: 2196
Joined: Jan 2, 2017

Re: SC2 Deepmind Demonstration

Post by kami_ryu »

-- deleted post --

Reason: on request (off-topic bulk delete)
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

momuuu wrote:
Goodspeed wrote:2 things on that. First, the AoE2 AI doesn't stand a chance against top level players. It's not close. Second, there's a very big difference between hard coding strategic concepts and build orders, and having the AI learn them itself. The fact that the AI got this good without hard coding anything is the main point of this research.

Its to put this in perspective. How much more can this technology achieve than a hardcoded AI based on 20 year old software? This showcase didnt really show that this concepts is much better on a strategic level than just hardcoding strategies.
That's not the point though. DeepMind doesn't care about making the best possible Starcraft bot. It wants to make an algorithm that learns how to do a task on its own, preferably better than humans. This demo showed that the concept works not just for turn-based scenarios, but real-time ones (where there is a nearly infinite number of possible actions to do at any given moment) as well. It also shows it works for imperfect information scenarios.

Oh and the answer to your question is much more, because it is already much better than any hardcoded SC2 AI and we are still at the beginning stages of AlphaStar. Btw AoE2HD's AI isn't 20 years old.
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

I think it might be good if you tried to understand what I wrote down.
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

I just don't think your comparison between a machine learning algorithm and a bunch of if-else statements is relevant, especially considering the point of this research. I also don't agree with your notion that you'd be better off hard coding strategic concepts; A* showed significantly better strategic "understanding" than any hardcoded AI in RTS history.
User avatar
Netherlands Mr_Bramboy
Retired Contributor
Donator 01
Posts: 8219
Joined: Feb 26, 2015
ESO: [VOC] Bram
Location: Amsterdam

Re: SC2 Deepmind Demonstration

Post by Mr_Bramboy »

kami_ryu wrote:
Mr_Bramboy wrote:Nice demonstration. Its APM wasn't human at all though. At some point during fights, you could clearly see AlphaStar blinking multiple stalkers individually in the same second while its APM spiked to 1500+. I think the biggest advantage these AIs have is their mouse accuracy, even with limited APM. As far as I understand it, AlphaStar can click anywhere on the screen whenever it wants to. That takes away both the burden of mouse accuracy and mouse speed. Imagine a CSGO AI with perfect mouse accuracy and you have what is basically a perfect aimbot. All in all, it's terrifying to see that AI has come so far that it clean sweeps a pro player even with limitations. The futuristic dystopia of robots taking over the world is becoming more likely with the day.


I don't see why it's surprising.

A pocket calculator with 50s technology can already do basic math faster than any human. This is a continuation of that.

A pocket calculator is hardcoded to do what it's supposed to do. AlphaStart figured out what to do by trial and error all by itself. That's the key difference between these two, and why it's so impressive.
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

Goodspeed wrote:I just don't think your comparison between a machine learning algorithm and a bunch of if-else statements is relevant, especially considering the point of this research. I also don't agree with your notion that you'd be better off hard coding strategic concepts; A* showed significantly better strategic "understanding" than any hardcoded AI in RTS history.

I just don't think you got the comparison. You just read what you think I'm saying, and not what I'm actually saying..
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

Or maybe I just don't agree with what you're saying. Seem possible? For example you said you might be better off hard coding strategic concepts. I don't think you'd get anywhere near the performance of A* doing that.
Feel free to elaborate if you think your point didn't come across.
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

You really should learn to read better. Every word in a sentence has a purpose. When your disagreement comes from blatantly ignoring parts of a sentence or entire sentences, you just failed to read.
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

I don't think I ignored anything, I just think you are vastly understating A*'s performance, especially when you compare it to AoE2HD's "AI". Maybe you missed how well A* picked its fights, how it consistently got economically ahead in the early game by investing in economy at the right times, how it knew when to be aggressive, how it knew to wait for the enemy to get out on the map before engaging, to surround his army, etcetera. Anyway if you still think I'm misunderstanding you...
Feel free to elaborate if you think your point didn't come across.
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

Goodspeed wrote:Or maybe I just don't agree with what you're saying. Seem possible? For example you said you might be better off hard coding strategic concepts. I don't think you'd get anywhere near the performance of A* doing that.
Feel free to elaborate if you think your point didn't come across.

momuuu wrote:I was reminded of aoe2's AI though. That AI is hardcoded but very competent and strategically pretty good actually, without getting any resource hacks. If this AI was toned down apm wise, I'd start to wonder how much of an upgrade it actually was compared to a 'state of the art' AI (programmed by one dude I think).

momuuu wrote:Its to put this in perspective. How much more can this technology achieve than a hardcoded AI based on 20 year old software? This showcase didnt really show that this concepts is much better on a strategic level than just hardcoding strategies.

If you read closely I don't think I'm saying that a hard coded AI would be better. I'm saying that strategically this AI was lackluster. Especially once you consider that for example the aoe2 AI is capable of countering your unit composition.

Now read this, and actually read every word of it:
Based on this exhibition Id say that strategically you might be better off hard coding the AI to do certain strats.

Wouldn't an AI coded to understand the counter system have done better in that last game? I sure do feel like it, as massing stalkers vs mass immortals is just stupid. Or you know, making Oracles to counter a warp prism..

Compare that to AlphaGo, which achieved something truly remarkable in the AI space that cant be done with any other technology. Thats the point I was trying to make.

Now read this. I don't even know how you can disagree with this statement. AlphaStar didn't actually show anything new, we've already seen the dota AI show the same skills and that AI was much more limited even (no communications at all between the 5 AI controlled units in a team game). So what did this show us? That if you're a mechanical god that you will be good at starcraft? I'd agree yes. It didn't even show that an AI can be a mechanically awesome player, because we've already seen that before.
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

If you read closely I don't think I'm saying that a hard coded AI would be better. I'm saying that strategically this AI was lackluster.
Yes and I disagree. Is that really so hard to believe? I mean it wasn't Sun Tzu reincarnate but I think its understanding of strategic concepts was actually very impressive, especially considering we're still in the early stages of its development.
Especially once you consider that for example the aoe2 AI is capable of countering your unit composition.
You cherry picked a thing that's very easy to hard code. Unit x is countered by unit y. Kid could do it. Examples of things that are extremely difficult to hard code are how to engage in fights properly, or when to engage in fights. When to attack and when to defend. When it's safe to expand. A*, when it came to this, seemed to have impeccable decision making, although it was a small sample.
Wouldn't an AI coded to understand the counter system have done better in that last game? I sure do feel like it, as massing stalkers vs mass immortals is just stupid. Or you know, making Oracles to counter a warp prism..
No, because it would have been severely worse than A* in other areas.

Given the difference between the 1-week agent and the 2-week one, I'm sure that with a little more training and some tweaks in how they do the training the agents will start to recognize the unit counter system and use it to their advantage. Currently the agents are likely to discard specialized units because they only work in niche situations and with the league system the agents are naturally drawn to strategies that are as universally applicable as possible. That is probably why the stalker and phoenix, both very versatile units, were so popular among all of the agents.
Compare that to AlphaGo, which achieved something truly remarkable in the AI space that cant be done with any other technology. Thats the point I was trying to make.
I'm pretty sure you can't make an AI like A*, or one that outperforms it, by "hard coding". Besides AlphaGo is a finished project. They only just started on this one.

Last I checked, in Brood war, there are bots which go over 10.000 APM and they are still getting demolished by professional human players.
User avatar
Netherlands Goodspeed
Retired Contributor
Posts: 13002
Joined: Feb 27, 2015

Re: SC2 Deepmind Demonstration

Post by Goodspeed »

Also, small correction: I was wrong about them not limiting APM. They did limit it, just not enough I guess. From the AMA:
We consulted with TLO and Blizzard about APMs, and also added a hard limit to APMs. In particular, we set a maximum of 600 APMs over 5 second periods, 400 over 15 second periods, 320 over 30 second periods, and 300 over 60 second period. If the agent issues more actions in such periods, we drop / ignore the actions. These were values taken from human statistics. It is also important to note that Blizzard counts certain actions multiple times in their APM computation (the numbers above refer to “agent actions” from pysc2, see https://github.com/deepmind/pysc2/blob/ ... alculation). At the same time, our agents do use imitation learning, which means we often see very “spammy” behavior. That is, not all actions are effective actions as agents tend to spam “move” commands for instance to move units around. Someone already pointed this out in the reddit thread -- that AlphaStar effective APMs (or EPMs) were substantially lower. It is great to hear the community’s feedback as we have only consulted with a few people, and will take all the feedback into account.
User avatar
Great Britain Horsemen
Jaeger
Posts: 2998
Joined: Sep 24, 2018

Re: SC2 Deepmind Demonstration

Post by Horsemen »

When do we get to see the Deepthroat demonstration?
User avatar
No Flag fightinfrenchman
Ninja
Donator 04
Posts: 23505
Joined: Oct 17, 2015
Location: Pennsylvania

Re: SC2 Deepmind Demonstration

Post by fightinfrenchman »

Horsemen wrote:When do we get to see the Deepthroat demonstration?


Did it take you four days to come up with this joke?
Dromedary Scone Mix is not Alone Mix
Image
No Flag kami_ryu
Retired Contributor
Posts: 2196
Joined: Jan 2, 2017

Re: SC2 Deepmind Demonstration

Post by kami_ryu »

-- deleted post --

Reason: on request (off-topic bulk delete)
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

Goodspeed wrote:
If you read closely I don't think I'm saying that a hard coded AI would be better. I'm saying that strategically this AI was lackluster.
Yes and I disagree. Is that really so hard to believe? I mean it wasn't Sun Tzu reincarnate but I think its understanding of strategic concepts was actually very impressive, especially considering we're still in the early stages of its development.
Especially once you consider that for example the aoe2 AI is capable of countering your unit composition.
You cherry picked a thing that's very easy to hard code. Unit x is countered by unit y. Kid could do it. Examples of things that are extremely difficult to hard code are how to engage in fights properly, or when to engage in fights. When to attack and when to defend. When it's safe to expand. A*, when it came to this, seemed to have impeccable decision making, although it was a small sample.
Wouldn't an AI coded to understand the counter system have done better in that last game? I sure do feel like it, as massing stalkers vs mass immortals is just stupid. Or you know, making Oracles to counter a warp prism..
No, because it would have been severely worse than A* in other areas.

Given the difference between the 1-week agent and the 2-week one, I'm sure that with a little more training and some tweaks in how they do the training the agents will start to recognize the unit counter system and use it to their advantage. Currently the agents are likely to discard specialized units because they only work in niche situations and with the league system the agents are naturally drawn to strategies that are as universally applicable as possible. That is probably why the stalker and phoenix, both very versatile units, were so popular among all of the agents.
Compare that to AlphaGo, which achieved something truly remarkable in the AI space that cant be done with any other technology. Thats the point I was trying to make.
I'm pretty sure you can't make an AI like A*, or one that outperforms it, by "hard coding". Besides AlphaGo is a finished project. They only just started on this one.

Last I checked, in Brood war, there are bots which go over 10.000 APM and they are still getting demolished by professional human players.

Sorry, but you literally dont read what I say...
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

Did the AI do a good job at understanding the counter system and basic build order or did it do worse at that than what humans can code into the AI? It did worse than what humans can do.
Is the AI in that sense not a leap forward? It is not a leap forward.
Did this demonstration show anything special then? No, we already saw OpenAI do the same thing but better in dota2.
What did this demonstration then show? Starcraft 2 can be won by mechanics.
Should we be impressed by that? No, unless you're a simple mind.
Netherlands momuuu
Ninja
Posts: 14237
Joined: Jun 7, 2015
ESO: Jerom_

Re: SC2 Deepmind Demonstration

Post by momuuu »

Goodspeed wrote:
If you read closely I don't think I'm saying that a hard coded AI would be better. I'm saying that strategically this AI was lackluster.
Yes and I disagree. Is that really so hard to believe? I mean it wasn't Sun Tzu reincarnate but I think its understanding of strategic concepts was actually very impressive, especially considering we're still in the early stages of its development.

You should appreciate something for what it is. Not realizing you can split units to defend a warp prism or make phoenixes to kill it, or that you probably shouldn't go full stalker against full immortal is straight up terrible. Thats sub gold level strategic understanding. It might have done okay in other aspects, such as army movement or build orders (the former is generally refered to as tactics in strategy games like these, and the latter was probably mostly the result of augmented learning), but that doesn't make this excusable. Especially once you get your facts straight and realize that AI can already do these things well.

Especially once you consider that for example the aoe2 AI is capable of countering your unit composition.
You cherry picked a thing that's very easy to hard code. Unit x is countered by unit y. Kid could do it. Examples of things that are extremely difficult to hard code are how to engage in fights properly, or when to engage in fights. When to attack and when to defend. When it's safe to expand. A*, when it came to this, seemed to have impeccable decision making, although it was a small sample.

I didn't cherry pick a thing that's very easy to code, because that's the entire point. It's very bad at something that's very easy to hard code. It's good at things that aren't easy to hard code, but we already knew (or at least should have known if you had followed AI developments in this area) that an AI can be good at these things. It seems entirely reasonable to 'cherry pick' on that one thing it was blatantly fucking terrible at because that was the main flaw. How can you even call this cherry picking, when I'm literally pointing at a fucking glaring problem and stating that it's a big deal.

Wouldn't an AI coded to understand the counter system have done better in that last game? I sure do feel like it, as massing stalkers vs mass immortals is just stupid. Or you know, making Oracles to counter a warp prism..
No, because it would have been severely worse than A* in other areas.

I never said you need to hardcode everything though, you just assumed that I did. In other posts it isn't even implied, as I even refer to it as "the AI" rather than "an AI".. This AI was bad at this strategic concepts while that was the primary thing it needed to be good at to be impressive or 'revolutionary'. Again, if you had followed AI developments you'd have realized that without showcasing that it can outsmart humans it doesn't actually show anything new. This was just the same thing as openAI but then half a year later in a slightly different game. Nothing new.

Given the difference between the 1-week agent and the 2-week one, I'm sure that with a little more training and some tweaks in how they do the training the agents will start to recognize the unit counter system and use it to their advantage. Currently the agents are likely to discard specialized units because they only work in niche situations and with the league system the agents are naturally drawn to strategies that are as universally applicable as possible. That is probably why the stalker and phoenix, both very versatile units, were so popular among all of the agents.
[\quote]
And once it does manage that, I'll be impressed. But for now it didn't really seem to figure out much on a strategical level. I wonder how much of its strategical understanding of build orders comes from the augmented learning and how much it discovered by itself. It seems way too early to conclude that deepmind can improve much more. 200 years of learning and it didn't figure out that you can split your army against a warp prism.. That isn't very promising. The potential might not be as huge, especially if you consider the possibility that most of its strategic understanding comes from augmented learning.

Compare that to AlphaGo, which achieved something truly remarkable in the AI space that cant be done with any other technology. Thats the point I was trying to make.
I'm pretty sure you can't make an AI like A*, or one that outperforms it, by "hard coding". Besides AlphaGo is a finished project. They only just started on this one.

Last I checked, in Brood war, there are bots which go over 10.000 APM and they are still getting demolished by professional human players.

openAI. Check it out. Also, I think currently A* mixed with some hard coding would be better than a raw A*. That's the main thing. Strategy was its big handicap in this sort of game. That's the new thing an AI should achieve. It didn't achieve that. Instead it showcased that an AI can be good at the things that humans do based on intuition (positioning, when to move out and things like that). But that's not something new.

Who is online

Users browsing this forum: No registered users and 8 guests

Which top 10 players do you wish to see listed?

All-time

Active last two weeks

Active last month

Supremacy

Treaty

Official

ESOC Patch

Treaty Patch

1v1 Elo

2v2 Elo

3v3 Elo

Power Rating

Which streams do you wish to see listed?

Twitch

Age of Empires III

Age of Empires IV