Civ Matchup Data: did you lose because you suck or matchup sucks?

[dutchdude117]

The hard working boys at SunBros Discord https://discord.gg/kBARf5XXzp
Have done it again!

Hopefully as time goes on and more and more data is collected, we can see better and better results.

No surprise to me is that Dutch is a bottom tier civ

[iron_turtle]

Can someone play aztec-ethiopia and hausa-haude matchup for science?

[kevinitalien]

id like to see same data but only with player above 1800 or 1900 elo, cuz when i see dutch with an insane winrate and otto with a bad winrate its weird to me xD

[hidden_blaze]

I'm not really sure how much info we can draw from this data. I think there are several problems:

1. A lot of the games you considered are played by low elo players with an elo of like 1100ish. I think in that elo range, matchups aren't all that important, it's more about doing basic things right. Most players probably don't know how to play a civ to its potential and how to exploit their opponent's civs weaknesses. If I cannot herd properly or let my TC idle for 5 minutes, I probably didn't lose because of the match up. In low elo, I think it matters much more how easily playable a civ is and not how strong it objectively would be.

2. You considered a lot of games where the elo gap between both players it quite big. Again, here I would say the match up becomes much less important, since the better player will always have a big advantage, even if they play a civ that isn't good against the civ their opponent is playing.

3. For the most matchups, it seems like there isn't enough data to back up the win rate. For example, you claim that Dutch are a bad civ with a lot of bad matchups, however in the majority of the matchups they played there are less than 20 games happening. So according to the data, Lakota vs. Dutch is a bad match up for Dutch, yet there were only 3 games played between them where Lakota won 2 and Dutch won 1. That doesn't really tell you anything about the matchup.

4. Generally, I feel like a lot of civs aren't figured out that well. When I play Italy for instance, I actually win a lot of games simply because my opponents don't know what to do against them and I can go age IV unpunished. When I play British, I lose a lot more games because every opponent I run into knows exactly what to do against them.. We simply know a lot more about legacy civs than about DE civs. So imo a lot of DE civs may have either the advantage of that the opponent doesn't know what to do against them or the disadvantage of the players not knowing what to do with them, and neither is exactly correlated with how strong the civ actually is.

5. For some civs, I'm not sure if it makes sense to just give a mean winrate. Ports for instance probably have a much higher winrate on water maps than on land maps. If we'd take 10 Port games on water maps, from which they'd win 8, and 10 more games from land maps, on which they only win 2, in total they'd seem like a perfectly balanced civ that won 10 out of 20 games, so they'd have a 50% win rate. They wouldn't be balanced at all though, they'd be too strong on water maps and too weak on land maps. The same concept might be possible if you differentiate between tp and no tp maps or livestock and no livestock maps, some civs are much stronger or weaker on those. Since DE QS doesn't let you pick your civ after you see the map, civs are just thrown on a map randomly that might or might not suit them.

So yeah, imo to get reliable data, you'd need to have a bigger pool of games between stronger players (I'd say an elo of at least 1600 should suffice) where both players have around the same elo. Even then, some civs may seem stonger or weaker simply because they can play on certain maps. And even then, a lot of civs won't be played as often, so some matchups won't take place at all or only very rarely, making it impossible to draw data for these particular matchups. On a game with such a small player base such as AoE3DE, it's probably impossible to get reliable data that holds all these factors constant.

[Don Nameless]

I knew japan win russia only on De feel the opposite like russia received op units to counter the strong units of japans. Dutch with meh percentage honestly doesn't convince me. India overrated in my opinion. Portugal score 60 weird i heard from everyone was op and now it's only 60? I still think that on De the Aztecs beat the Chinese and China has very little chance of winning. Russia against Aztecs sincerely never seen that Russia has a great advantage, let's say the Aztecs have the advantage more than Russia. Russia against Spain, Spain is favored more than Russia.
If H2o were here it would demonstrate how the Portuguese are much stronger but he's not here and so we're satisfied.
Anyway good job but I would like to see more reliable graphs with more games played in all match ups and 2k plus games, 1800/1900 meh it doesn't seem like there are really that many good players just people abusing broken things or civ a lot strong.
It's just my opinion not a law

[iron_turtle]

Although this data is too early to draw any significant conclusions, it should only get better from here. Also, how is otto the highest picked civ? I would have guessed brits or france.

[Kanoo]

How does Brits have among the least good matchups?

[callentournies]

How many games are being added to the raw data each day?

[Garja]

This has to be done with like top 20% elo for a very extensive number of games or it's kinda useless.
Good effort nonetheless.

[iCourt]

Is this manually entered or is it pulling game server data?

[dutchdude117]

Question 1: I would like to see same but with X data exluded or included
Response 1: Great! I would too. Hopefully the guys at SunBros can do that, but if you dont want to wait you can check out the data your self (its all on that spread sheet) and see if you can figure it out.

Question 2: There isnt enough data yet? will this be updated?
Response 2: I agree, there isnt enough data yet. I don't know when it will be updated but im pretty sure it will be updated. You can check out the discord for more details on that.

Question 3: Is this manual or pulling game server data.
Response 3: I believe this is scraping data from live games being played. You'll have to check the discord for more details.

[callentournies]

Is this manually entered or is it pulling game server data?

It's pulled... Dori recently got match history to work on the aoe3 companion app, and the # games in the civ grid went from like 80 (manually entered over 2 weeks) to 3k overnight ish. Presumably hellpunch is working with that. But THAT game data has to be manually entered into the civ grid data IIRC to update it -- it isn't automatically updating.

If I'm reading the 'cord right, ~half (or less) of all games played meet the >1100 Elo, <100 Elo difference, longer than 4-mins criteria. Approximately can add ~2k games/day to this data, but more likely it'll be mass updated about once a week w/14k new games or something.

[richard]

Nice Thread!

Though I doubt that this is an effective way to determine which civ "wins" a matchup.

IMO to effectively determine this, it would be better to let two very good players play several games between two civs performing pre-thought-out build orders. The "favored" civ A should play a potentially game-winning buildorder repeatedly and the lower civ B, knowing what will be coming, should try several builds to hold it. If Civ B cant find a way to hold civ A s best build, then it s Civ-A-favored.

Example: Let Ottomans perform a well executed FF several times and Russia try to find a way to hold it. Then you will see the matchup is Ottomans-favored.

The Statistics you did are not sufficient here I think. For example, if one player did a stupid mistake in one game which loses him the game, you cant count that game. It should be removed from Data. But in your data, you dont know what happened in the games at all. Thats not enough information to draw conclusions about civ balance.

[firstaim]

I think pickrate more or less can show the civ strength too, because people are mostly preferring to use a stronger civ to win the game.
Moreover I also think 1100 elo will be too low for the data, I have seen many people under elo 1300 using for example Sweden or British, when they were facing Haude rush, they were still sending 4torps/700w even 3rd card is not sending any army, assuming they can build more houses to outboom their opponent.

Thanks for the effort to make the table.

[RefluxSemantic]

You really have to also include standard deviation here. Data is useless if you dont do a proper statistical analysis. You'll be surprised by the number of games you need to estimate winrate within a 1% confidence intervall.

[aligator92]

I really appreciate the effort put into this!

But I think it is counter-productive. As Garja said, this data is irrelevant for the top players. And for anybody below 1700 or even 1800 ELO balance is just not that relevant. When I was more active I was around 1700 and people (including me) would drop games left and right because they would miss a massive raid, forget their market techs, not scout, take fights with half their army, delete 30 Cav into a choke point or gernerally do bad build orders. You cannot blame balance in these cases. Having this kind of data available just gives people another chance to blame something that is out of their hands instead of realizing that they just played poorly.

[hellpunch]

You remember those matches because they were matches where those relevant events occured, its not always like that. Above 1100 elo is already a good starting point as people around that elo already have grasps of the game, even if not 'pro like', also useless to reduce the elo any point further right now as there aren't enough matches. Maybe in a month. I have 12k matches, already with this elo setup, remainings are only 5.6k matches...

[aligator92]

You remember those matches because they were matches where those relevant events occured, its not always like that. Above 1100 elo is already a good starting point as people around that elo already have grasps of the game, even if not 'pro like', also useless to reduce the elo any point further right now as there aren't enough matches. Maybe in a month. I have 12k matches, already with this elo setup, remainings are only 5.6k matches...

The events don't need to be as drastic as throwing away 30 units but they will still lose you the game no matter what MU. Just this week somebody ranted to me about the MU but when I checked the map in the end it turned out that he had not herded properly and had like 15 vills on berries and that was on 1600 ELO and it happens all the time. People at that level are just not consistent enough to blame balance. Sure some MUs might be favourable but the main decider between victory and defeat is the performance in that game. The amount of actually unwinnable MUs is tiny even at 1700 ELO. Let alone 1100.

[hellpunch]

Then the same can be said by a 2k player for a 1.7k player 'yeah he did that and this and should have done that, can't balance with that', so where do we stop? we just gather the data of top 5 players and balance from 50 matches? If the player has a basic understanding of the game and his civ, its fine to start balancing from there. The balance isn't suppossed to be for 20 people, its supposed to be for the majority of the people that play the game. Anyway later on, it should have a filter by elo system, when enough matches are gathered.

[firstaim]

The events don't need to be as drastic as throwing away 30 units but they will still lose you the game no matter what MU. Just this week somebody ranted to me about the MU but when I checked the map in the end it turned out that he had not herded properly and had like 15 vills on berries and that was on 1600 ELO and it happens all the time. People at that level are just not consistent enough to blame balance. Sure some MUs might be favourable but the main decider between victory and defeat is the performance in that game. The amount of actually unwinnable MUs is tiny even at 1700 ELO. Let alone 1100.

Some elo may be not accurate he can ask sb else to play instead of himself but the percentage is not high, at least above 1400~1500 most of them know the basic concept, may be some people are relying on OP civ only (I have seen a 1300 player using Lakota no matter what opponent using, he was just spamming BR), may be some people have super APM but no concept, this is only few people, however 1100~1300 I have seen a quite large percentage those players still want to boom under hard rush.

[gibson]

You really have to also include standard deviation here. Data is useless if you dont do a proper statistical analysis. You'll be surprised by the number of games you need to estimate winrate within a 1% confidence intervall.

Given the nature of the situation I dont think it would ever be possible to estimate winrate withint a 1% confidence interval lol

[dutchdude117]

I can say "definitively" that it is the match up that sucks for me

[fei123456]

Why do the data in this table doesn't match sometimes? For example, it says Spain has 63% winrate against French, while French has 55% against Spain: that's impossible.