Civ Matchup Data: did you lose because you suck or matchup sucks?

General forum about Age of Empires 3 DE. Please post strategy threads, recorded games, user-created content and tech support threads in their respective forum.
User avatar
United States of America dutchdude117
ESOC Community Team
Posts: 237
Joined: Apr 24, 2015
ESO: Dutchdude117
Location: USA California

Civ Matchup Data: did you lose because you suck or matchup sucks?

  • Quote

Post by dutchdude117 »



The hard working boys at SunBros Discord https://discord.gg/kBARf5XXzp
Have done it again!

Hopefully as time goes on and more and more data is collected, we can see better and better results.

No surprise to me is that Dutch is a bottom tier civ ;) ;)
User avatar
India iron_turtle
Skirmisher
Posts: 139
Joined: Jan 25, 2021
ESO: iron_turtle

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by iron_turtle »

Can someone play aztec-ethiopia and hausa-haude matchup for science?
User avatar
French Southern Territories kevinitalien
Lancer
Posts: 671
Joined: Oct 31, 2015
ESO: KEVINITALIEN
Clan: PLOP

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

  • Quote

Post by kevinitalien »

id like to see same data but only with player above 1800 or 1900 elo, cuz when i see dutch with an insane winrate and otto with a bad winrate its weird to me xD
Image
Image
Image Image
No Flag hidden_blaze
Musketeer
Posts: 58
Joined: Apr 7, 2020

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

  • Quote

Post by hidden_blaze »

I'm not really sure how much info we can draw from this data. I think there are several problems:

1. A lot of the games you considered are played by low elo players with an elo of like 1100ish. I think in that elo range, matchups aren't all that important, it's more about doing basic things right. Most players probably don't know how to play a civ to its potential and how to exploit their opponent's civs weaknesses. If I cannot herd properly or let my TC idle for 5 minutes, I probably didn't lose because of the match up. In low elo, I think it matters much more how easily playable a civ is and not how strong it objectively would be.

2. You considered a lot of games where the elo gap between both players it quite big. Again, here I would say the match up becomes much less important, since the better player will always have a big advantage, even if they play a civ that isn't good against the civ their opponent is playing.

3. For the most matchups, it seems like there isn't enough data to back up the win rate. For example, you claim that Dutch are a bad civ with a lot of bad matchups, however in the majority of the matchups they played there are less than 20 games happening. So according to the data, Lakota vs. Dutch is a bad match up for Dutch, yet there were only 3 games played between them where Lakota won 2 and Dutch won 1. That doesn't really tell you anything about the matchup.

4. Generally, I feel like a lot of civs aren't figured out that well. When I play Italy for instance, I actually win a lot of games simply because my opponents don't know what to do against them and I can go age IV unpunished. When I play British, I lose a lot more games because every opponent I run into knows exactly what to do against them.. We simply know a lot more about legacy civs than about DE civs. So imo a lot of DE civs may have either the advantage of that the opponent doesn't know what to do against them or the disadvantage of the players not knowing what to do with them, and neither is exactly correlated with how strong the civ actually is.

5. For some civs, I'm not sure if it makes sense to just give a mean winrate. Ports for instance probably have a much higher winrate on water maps than on land maps. If we'd take 10 Port games on water maps, from which they'd win 8, and 10 more games from land maps, on which they only win 2, in total they'd seem like a perfectly balanced civ that won 10 out of 20 games, so they'd have a 50% win rate. They wouldn't be balanced at all though, they'd be too strong on water maps and too weak on land maps. The same concept might be possible if you differentiate between tp and no tp maps or livestock and no livestock maps, some civs are much stronger or weaker on those. Since DE QS doesn't let you pick your civ after you see the map, civs are just thrown on a map randomly that might or might not suit them.

So yeah, imo to get reliable data, you'd need to have a bigger pool of games between stronger players (I'd say an elo of at least 1600 should suffice) where both players have around the same elo. Even then, some civs may seem stonger or weaker simply because they can play on certain maps. And even then, a lot of civs won't be played as often, so some matchups won't take place at all or only very rarely, making it impossible to draw data for these particular matchups. On a game with such a small player base such as AoE3DE, it's probably impossible to get reliable data that holds all these factors constant.
User avatar
Italy Don Nameless
Skirmisher
Posts: 183
Joined: Dec 12, 2022
Location: Italy

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by Don Nameless »

I knew japan win russia only on De feel the opposite like russia received op units to counter the strong units of japans. Dutch with meh percentage honestly doesn't convince me. India overrated in my opinion. Portugal score 60 weird i heard from everyone was op and now it's only 60? I still think that on De the Aztecs beat the Chinese and China has very little chance of winning. Russia against Aztecs sincerely never seen that Russia has a great advantage, let's say the Aztecs have the advantage more than Russia. Russia against Spain, Spain is favored more than Russia.
If H2o were here it would demonstrate how the Portuguese are much stronger but he's not here and so we're satisfied.
Anyway good job but I would like to see more reliable graphs with more games played in all match ups and 2k plus games, 1800/1900 meh it doesn't seem like there are really that many good players just people abusing broken things or civ a lot strong.
It's just my opinion not a law
Attachments
As H2O said
As H2O said
User avatar
India iron_turtle
Skirmisher
Posts: 139
Joined: Jan 25, 2021
ESO: iron_turtle

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by iron_turtle »

Although this data is too early to draw any significant conclusions, it should only get better from here. Also, how is otto the highest picked civ? I would have guessed brits or france.
User avatar
No Flag Kanoo
Skirmisher
Posts: 174
Joined: Jan 7, 2020

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by Kanoo »

How does Brits have among the least good matchups?
thebritish wrote:
19 Jan 2016, 09:58
So, you are saying that if i watch H20's rec, i can beat anyone below H20's level because i know his BO?
Rainbow Land callentournies
Howdah
Posts: 1699
Joined: May 6, 2021
ESO: esuck

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by callentournies »

How many games are being added to the raw data each day?
If I were a petal
And plucked, or moth, plucked
From flowers or pollen froth
To wither on a young child’s
Display. Fetch
Me a ribbon, they, all dead
Things scream.
User avatar
Italy Garja
Retired Contributor
Donator 02
Posts: 9752
Joined: Feb 11, 2015
ESO: Garja

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by Garja »

This has to be done with like top 20% elo for a very extensive number of games or it's kinda useless.
Good effort nonetheless.
Image Image Image
User avatar
United States of America iCourt
Retired Contributor
Posts: 700
Joined: Jan 14, 2016
ESO: iCourt
Location: Monterey, California

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by iCourt »

Is this manually entered or is it pulling game server data?
User avatar
United States of America dutchdude117
ESOC Community Team
Posts: 237
Joined: Apr 24, 2015
ESO: Dutchdude117
Location: USA California

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by dutchdude117 »

Question 1: I would like to see same but with X data exluded or included
Response 1: Great! I would too. Hopefully the guys at SunBros can do that, but if you dont want to wait you can check out the data your self (its all on that spread sheet) and see if you can figure it out.

Question 2: There isnt enough data yet? will this be updated?
Response 2: I agree, there isnt enough data yet. I don't know when it will be updated but im pretty sure it will be updated. You can check out the discord for more details on that.

Question 3: Is this manual or pulling game server data.
Response 3: I believe this is scraping data from live games being played. You'll have to check the discord for more details.
Rainbow Land callentournies
Howdah
Posts: 1699
Joined: May 6, 2021
ESO: esuck

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by callentournies »

iCourt wrote:
02 Feb 2023, 16:27
Is this manually entered or is it pulling game server data?
It's pulled... Dori recently got match history to work on the aoe3 companion app, and the # games in the civ grid went from like 80 (manually entered over 2 weeks) to 3k overnight ish. Presumably hellpunch is working with that. But THAT game data has to be manually entered into the civ grid data IIRC to update it -- it isn't automatically updating.

If I'm reading the 'cord right, ~half (or less) of all games played meet the >1100 Elo, <100 Elo difference, longer than 4-mins criteria. Approximately can add ~2k games/day to this data, but more likely it'll be mass updated about once a week w/14k new games or something.
If I were a petal
And plucked, or moth, plucked
From flowers or pollen froth
To wither on a young child’s
Display. Fetch
Me a ribbon, they, all dead
Things scream.
User avatar
Germany richard
Dragoon
Posts: 342
Joined: Feb 24, 2015
Location: Germany

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

  • Quote

Post by richard »

Nice Thread!

Though I doubt that this is an effective way to determine which civ "wins" a matchup.

IMO to effectively determine this, it would be better to let two very good players play several games between two civs performing pre-thought-out build orders. The "favored" civ A should play a potentially game-winning buildorder repeatedly and the lower civ B, knowing what will be coming, should try several builds to hold it. If Civ B cant find a way to hold civ A s best build, then it s Civ-A-favored.

Example: Let Ottomans perform a well executed FF several times and Russia try to find a way to hold it. Then you will see the matchup is Ottomans-favored.

The Statistics you did are not sufficient here I think. For example, if one player did a stupid mistake in one game which loses him the game, you cant count that game. It should be removed from Data. But in your data, you dont know what happened in the games at all. Thats not enough information to draw conclusions about civ balance.
Macau firstaim
Crossbow
Posts: 48
Joined: Nov 28, 2020

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by firstaim »

I think pickrate more or less can show the civ strength too, because people are mostly preferring to use a stronger civ to win the game.
Moreover I also think 1100 elo will be too low for the data, I have seen many people under elo 1300 using for example Sweden or British, when they were facing Haude rush, they were still sending 4torps/700w even 3rd card is not sending any army, assuming they can build more houses to outboom their opponent.

Thanks for the effort to make the table.
No Flag RefluxSemantic
Gendarme
Posts: 5996
Joined: Jun 4, 2019

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by RefluxSemantic »

You really have to also include standard deviation here. Data is useless if you dont do a proper statistical analysis. You'll be surprised by the number of games you need to estimate winrate within a 1% confidence intervall.
User avatar
Germany aligator92
Howdah
Posts: 1519
Joined: Feb 27, 2015
ESO: aligator92

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

  • Quote

Post by aligator92 »

I really appreciate the effort put into this!

But I think it is counter-productive. As Garja said, this data is irrelevant for the top players. And for anybody below 1700 or even 1800 ELO balance is just not that relevant. When I was more active I was around 1700 and people (including me) would drop games left and right because they would miss a massive raid, forget their market techs, not scout, take fights with half their army, delete 30 Cav into a choke point or gernerally do bad build orders. You cannot blame balance in these cases. Having this kind of data available just gives people another chance to blame something that is out of their hands instead of realizing that they just played poorly.
Italy hellpunch
Crossbow
Posts: 15
Joined: Jul 15, 2022

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by hellpunch »

You remember those matches because they were matches where those relevant events occured, its not always like that. Above 1100 elo is already a good starting point as people around that elo already have grasps of the game, even if not 'pro like', also useless to reduce the elo any point further right now as there aren't enough matches. Maybe in a month. I have 12k matches, already with this elo setup, remainings are only 5.6k matches...
User avatar
Germany aligator92
Howdah
Posts: 1519
Joined: Feb 27, 2015
ESO: aligator92

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by aligator92 »

hellpunch wrote:
03 Feb 2023, 14:52
You remember those matches because they were matches where those relevant events occured, its not always like that. Above 1100 elo is already a good starting point as people around that elo already have grasps of the game, even if not 'pro like', also useless to reduce the elo any point further right now as there aren't enough matches. Maybe in a month. I have 12k matches, already with this elo setup, remainings are only 5.6k matches...
The events don't need to be as drastic as throwing away 30 units but they will still lose you the game no matter what MU. Just this week somebody ranted to me about the MU but when I checked the map in the end it turned out that he had not herded properly and had like 15 vills on berries and that was on 1600 ELO and it happens all the time. People at that level are just not consistent enough to blame balance. Sure some MUs might be favourable but the main decider between victory and defeat is the performance in that game. The amount of actually unwinnable MUs is tiny even at 1700 ELO. Let alone 1100.
Italy hellpunch
Crossbow
Posts: 15
Joined: Jul 15, 2022

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by hellpunch »

Then the same can be said by a 2k player for a 1.7k player 'yeah he did that and this and should have done that, can't balance with that', so where do we stop? we just gather the data of top 5 players and balance from 50 matches? If the player has a basic understanding of the game and his civ, its fine to start balancing from there. The balance isn't suppossed to be for 20 people, its supposed to be for the majority of the people that play the game. Anyway later on, it should have a filter by elo system, when enough matches are gathered.
Macau firstaim
Crossbow
Posts: 48
Joined: Nov 28, 2020

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by firstaim »

aligator92 wrote:
03 Feb 2023, 17:03
The events don't need to be as drastic as throwing away 30 units but they will still lose you the game no matter what MU. Just this week somebody ranted to me about the MU but when I checked the map in the end it turned out that he had not herded properly and had like 15 vills on berries and that was on 1600 ELO and it happens all the time. People at that level are just not consistent enough to blame balance. Sure some MUs might be favourable but the main decider between victory and defeat is the performance in that game. The amount of actually unwinnable MUs is tiny even at 1700 ELO. Let alone 1100.
Some elo may be not accurate he can ask sb else to play instead of himself but the percentage is not high, at least above 1400~1500 most of them know the basic concept, may be some people are relying on OP civ only (I have seen a 1300 player using Lakota no matter what opponent using, he was just spamming BR), may be some people have super APM but no concept, this is only few people, however 1100~1300 I have seen a quite large percentage those players still want to boom under hard rush.
User avatar
Tuvalu gibson
Ninja
ECL Reigning Champs
Posts: 13601
Joined: May 4, 2015
Location: USA

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by gibson »

RefluxSemantic wrote:
03 Feb 2023, 08:50
You really have to also include standard deviation here. Data is useless if you dont do a proper statistical analysis. You'll be surprised by the number of games you need to estimate winrate within a 1% confidence intervall.
Given the nature of the situation I dont think it would ever be possible to estimate winrate withint a 1% confidence interval lol
User avatar
United States of America dutchdude117
ESOC Community Team
Posts: 237
Joined: Apr 24, 2015
ESO: Dutchdude117
Location: USA California

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by dutchdude117 »

I can say "definitively" that it is the match up that sucks for me ;)
Attachments
image.png
User avatar
China fei123456
Jaeger
Posts: 3288
Joined: Apr 23, 2015
ESO: fei123456
Location: Alderaan

Re: Civ Matchup Data: did you lose because you suck or matchup sucks?

Post by fei123456 »

Why do the data in this table doesn't match sometimes? For example, it says Spain has 63% winrate against French, while French has 55% against Spain: that's impossible.

Who is online

Users browsing this forum: No registered users and 2 guests

Which top 10 players do you wish to see listed?

All-time

Active last two weeks

Active last month

Supremacy

Treaty

Official

ESOC Patch

Treaty Patch

1v1 Elo

2v2 Elo

3v3 Elo

Power Rating

Which streams do you wish to see listed?

Twitch

Age of Empires III

Age of Empires IV