Balanced MU

[BrookG]

I didn't say these games are related to any way with a tournament. Hypothetically, let's have two players, e.g. Lordraphael and Mitoe, play a certain MU 400 times, to have a significant result, and the winrates are 60-40, we can say that the 60% civ is somewhat equal to the other, that is according to @Garja . Personally, I would set a lower boundary, like 55-45, @momuuu was saying a 52-48 is more preferable. Clearly normalising 91 pairs of winrates is adding even more to the problem. My other point that is linked to the first is how many games do we need to have a significant with little errors winrates result. Having many games and players of similar skill will minimize the factors of only one player doing many mistakes, map screws and different map situations (tp, non-tp, water, etc) or others.

[Mitoe]

@momuuu was saying a 52-43 is more preferable.

Sooo... 5% of the games are draws?

[Garja]

@BrookG
As I said a gigantic number of games if we want to make conclusions based solely on the outcome of each game.
For example, look at jp elo. It has a fair amount of games played and you can even filter the games for map and pr range (25-50). But it is still quite inaccurate I'd say it needs some additional interpretation to reflect the state of balance.

[BrookG]

The problem with jpelo is that players are of various levels and we can clearly blame maps and cheats too. I don't see updating patch yearly or twice per year, or even a tournament can reflect the desired balance. The same works for the tournaments. Obtaining various data from that had me thinking about their credibility. EP is vastly more consistent and balanced. Do we have however some destination point or no? Statistics although being a useful tool, can't help us yet in our current state.

[WickedCossack]

@momuuu was saying a 52-43 is more preferable.

Sooo... 5% of the games are draws?

It's 40-40.

(SC2 reference for the one person that might recognise it!)

[momuuu]

@momuuu was saying a 52-43 is more preferable.

Sooo... 5% of the games are draws?

Probably a typo. I thought +-5% would be a point of OP. Something smaller like +-3% would seem absolutely fine to me. So a civ with average winrate between 47% and 53% is a civ I wouldnt touch.

Anyways, the main point of this thread is that it is very hard to measure balance. Brook and I did some basic statistics and it turns out that even with a sample size of 400 you have a ~5% error in your calculated winrate. So if you say a civ with more than 55% winrate is OP and should be nerfed, and civ between 40 and 60% winrate cant statistically be concluded to be imbalanced. I think you need about 1600 samples for a 2.5% accuracy, which means that civs with more than 57.5% winrate are beyond what one might consider balanced. Even then, the variance is relatively large.

[n0el]

That is the biggest problem, there aren't large enough data sets.

[Jaeger]

That is the biggest problem, there aren't large enough data sets.

U want to play? (check ESO plz)

[n0el]

That is the biggest problem, there aren't large enough data sets.

U want to play? (check ESO plz)

Not home. Left it open probably

[gibson]

Blow me