Loading...
  OR  Zero-K Name:    Password:   

Mathematical balance metrics in competitive multiplayer games

45 posts, 3802 views
Post comment
Filter:    Player:  
Page of 3 (45 records)
sort
8 years ago
So my thesis have gotten delayed and I can't make up my mind of what to focus on.
My thesis advisor said I should make my thesis easier to read and aim at social study students. He also asked if I could include other games in my analysis but I insisted it wasn't really necessary because my results are just mathematical and I don't know how I would relate the game unit stats to each other between different games other than just showing them.
I'd like to compare games and try to compare their strategic depth but that is much larger scope and would be too much for a bachelor thesis but I'd like to do it in a later master thesis.

Basically my thesis describes simple unit metrics like DPS/cost, DPS*health/cost because I couldn't find anything about "Mathematical balance metrics" in the scientific literature.
Originally I intended that my work should only be about Planetary Annihilation because I was so hyped about PA and PA having only 1 faction would be interesting from balance perspective as what makes the strategic depth diverse and dynamic have to come from the unit balance rather than factional strengths.
However my interest for PA died. I don't know if it were the guilt of the undone thesis that made me quit or if it was something else.

So now I'm thinking about comparing stats from Zero-K to PA although I'm not really sure how much impact that will have in the thesis as it is likely to only result in a graphic or 2 and I'm not sure if it is the thesis scope to compare the games to any depth.

So what tools can you recommend to pull out stats from Zero-K?
Can you recommend any graphic tool that automatically produces graphics from the data extracted?

For PA I used https://github.com/speth/planetary-annihilation-db to generate a webpage with desired stats.
My page: http://people.dsv.su.se/~akbj7812/pa-db/
Example of stat table: http://people.dsv.su.se/~akbj7812/pa-db/unitmatchup/assault_bot/stat.html

Here is a rough draft of the thesis if you are interested: http://people.dsv.su.se/~akbj7812/Thesis.pdf
+9 / -0


8 years ago
Depends on what kind of stats you want.

Here's the bit that generates the unitguide, which should be a nice indication of how you can obtain unit definition values.

Statistical data on actual unit performance ingame has been shot for around a year, and my attempts to repair it were in vain.

Math and game theory article being easier to understand for 'social study students' sounds like some kind of a jab.

+3 / -0

8 years ago
The static unit guide generation tool is probably decent enough at extracting these but remember many of these values are not what they are in the files due to post-processing.

A widget could read the real ingame values and write them somewhere but I don't think any reliable widget for that exists. Maybe check out the space+click menu.
+0 / -0
You might do better to try write this with something with simpler mechanics like starcraft... Zero K has a ridiculous number of factors that are going to be really hard to computate - projectiles that generally miss, model size and shape, turn rate, turret rate, area of effect. I don't see how you will begin to capture that with a mathematic model
+1 / -0
I know this wasn't the question, but I was surprised that there was no mention of Hazardous Software (of Achron fame) using automated unit tests to verify game balance during development.

It also seems like a natural extension to apply some of these mathematical models to a game (zk maybe) and then see if those can predict the observed "metagame" preferences of players (no idea how to usefully measure this) but I might be misunderstanding the scope. In fact one might go from the other direction by gathering some sort of popularity metric and then seeing which mathematical properties correlate best with them.

I'm neither a mathematician nor an academic, so my apologies if these are actually useless ideas.
+2 / -0

8 years ago
You could make software that downloads ZK replays and gathers information from them (I think you can just create your own widget that gathers info and run each replay to do so - also possible to automatize).
One thing I'd do is try to calculate if units made cost. Basically how much metal they killed/how much they cost, and you can use that as a balance metric (although it only shows how effective people are at using units, it's still an interesting metric).
The way you calculate this can also be interesting (scientifically) as units can be healed, multiple units can damage a unit that got killed, and there are some utility units as well.
You can then compare results over time (as balance changes were made) and the difference of units on skill levels and game types (1v1, teams, etc.).
For looking at raw stats you can analyze the metrics you suggested (value = DPS*health/cost) and similar, and you can actually compare games with that in mind. Example: The variance of the unit value in ZK and SC2 is such and such and "units-made-cost" variance is ... for low/high ranked players, etc.
You can provide analysis as how patches have influenced unit balance and also do such comparison between games.
Anyway, lots of room for contributions.
+0 / -0
Math balancing was tried in Spring ages ago and the consensus was that it does nothing more than give ballpark figures when it comes to relevant values.

Like IK said, there are tons of values which interact with each other (and with the user!) in ways that make it very difficult to create a decent mathematical model.
[Spoiler]

JPrankgajop: ZK has a modstats tool that does exactly that: gather stats from games. It is broken at the moment though.
+1 / -0
the scope is very wobbling. my experience tells me, that you have to somehow narrow the scope and make it gritty (at which i was not so successful with several works so far, costed me much more unneeded time investment to the work).
what is the final goal/vision the thesis shall target? how to win, what amount of tech stats or unit usages lead to what statistical outcome?

this is a very complex task. i am not even sure how i would start here, as gaming has so many dimensions and the technical parameters are only one part of it. how much part has the human creativity in this (cover this too?)?

you write rock-paper-scissors (RPS) principle, also overkill stats. most simple start is RPS with oneshot / no overkill, then you can compare easiest way. look for units close to this as a start maybe, then broaden along the lines of the unit classes. hardes part are classification of units which dont fit into classes.

what are your plans to stich these into equations:
information/information denial
time windows to react to known things
macro gaming
unit micro
element of surprise
timeing to react and teamwork
unit combinations to a certain situation
unit placement on the battlefield (and influence by unit speed...)
singleplayer / teamplayer coordination
etc

i would also write a subchapter about self-target your thesis from the side (how you say that in proper english, i hope you understand), how a human can alter the game mechanics with unintended usage, see creative trolling in zero-k.

i feel im drawing the ideas too far. im glad if i can offer more support later on, keep us updated.

also, im 87,37% sure you name is not godde godde. you cannot fool me!
+1 / -0
8 years ago
EErankAdminAnarchid, PLrankAdminSprung, JPrankgajop
About statistics:
I really like to avoid statistics and just work with raw numbers and assume the simulation in a whitebox where all values are known.
True, verifying the simulation with statistics would be more scientific but it would get real complex really soon when you work with stuff like:
Is 1 ingame second 1 realtime second? If they differ, which one should I choose?
Is DPS calculated correctly? There is so much stuff that could prevent units from dealing full DPS even in perfect conditions. Like rounding off Rate of Fire, damage or damage falloff.
Is income and cost correct? It was just recently discovered that income was done every 16th frame while half a second is 15 frames.

USrank[GBC]1v0ry_k1ng
I've had similar discussions in the PA forums. https://forums.uberent.com/threads/defining-mathematical-unit-balance-metrics.56287/
Basically I just want to get the numbers right for well defined conditions. With different and more advanced metrics you can calculate the likeliness of dodging dodge projectiles, how much time the player has to react and so on. Also I don't think that Starcraft is that much easier initially, because combat there is also complex with overkill, stutterstep kiting and unit behaviors depending on micro that changes the effectiveness of the units drastically. In the later phases though if bullet physics should be included, then, yes, Starcraft would be much easier. But its long way to get there. I prefer working with TA derivatives.

USrankkaen
I haven't found any papers or any actual descriptions of how the "strategy space" is evaluated in Achron so I can't really include it. Do you have any?

DErankAdminmojjj
Basically I haven't delved into the "game theory" except in the background(notice that game theory is mostly about economics rather than games). I don't plan to include it in my bachelor thesis. Maybe a bit in my master thesis.
+0 / -0

8 years ago
quote:
Is 1 ingame second 1 realtime second? If they differ, which one should I choose?
Is DPS calculated correctly? There is so much stuff that could prevent units from [...]
Is income and cost correct? It was just recently discovered that income was done every 16th frame while half a second is 15 frames.

Why would that matter for comparing the units in the game? Any measure can be rescaled anyway (that's basically what units do). Aside from that, I have yet to see somebody notice the minute difference of 16/15.
+0 / -0
Company of Heroes 2 uses "Mathematical balance metrics."

http://www.gamasutra.com/blogs/PeterQumsieh/20150115/233644/Balancing_Multiplayer_Games__Intuition_Iteration_and_Numbers.php

If you’re going aim your article at game developers you need to give them practical “gems” that they can uses in development. I don’t want to sound harsh but right now your paper reads like one of the many throw away articles on gamasutra.

Not all games have HP some are instant kill like Chess or use physics based damage. You restrict yourself to HP based game which is fine. I think you need to present a balancing algorithm restricted to this damage model. It does not have to be a full balancing system and can make assumptions such as average damage to account for misses.

Is this a PHD or only a master's paper?

Another recent article I found interesting.
http://gamasutra.com/blogs/KeithBurgun/20150406/240462/Videogames_Are_Broken_Toys.php
+1 / -0
8 years ago
quote:
Why would that matter for comparing the units in the game? Any measure can be rescaled anyway (that's basically what units do). Aside from that, I have yet to see somebody notice the minute difference of 16/15.
It matters in Time To Kill and how much time the player has to micro for example. Yeah, Time conversion is easy though and it basically only matters when the player reaction time is involved. But my point is that proving unit stats with statistics is hard and something that I rather not delve into.
+1 / -0
USrankJasper
Good links!

About restrictions: I need to restrict myself. The scope cannot be too large. It's only a bachelor thesis.

quote:
...I think you need to present a balancing algorithm restricted to this damage model. It does not have to be a full balancing system and can make assumptions such as average damage to account for misses.
I'm not making a balance algorithm. I'm only making "balance metrics". The point is that those metrics should be tools to assess the balance. Deciding the balance is up to the game developer and the balance design for that game. I don't believe in "perfect balance" especially when dealing with games where players have symmetrical options. In regards to fairness, balance is completely arbitrary when the players have the same options. However the difficult stuff is to keep the game interesting, deep and to maintain many viable options/strategies. Now I don't think that there is just 1 "balance algorithm" for all types of balance designs.
+0 / -0
DErankAdminmojjj
quote:
what are your plans to stich these into equations:
information/information denial
Information/information denial is mostly a metagame cost benefit analysis IMO. It's pretty much game theory 101.
Lets take nuke/anti-nuke in ZK for example. There are some options:
1. Make anti-nuke without scouting.
2. Scout the enemy to look for nukes.
3. Attack the enemy so they can't spare the resources to make a nuke.
4. Don't bother about it as a nuke won't cripple you if it hits you.

From the start of the game, 4 and 3 is pretty dominant as you don't have something worth nuking and if the enemy tries to rush a nuke you can probably kill them with your army.
As economies grow nukes becomes more and more accessible and can be built in less time. Scouting the enemy might be still be cheaper than making an anti-nuke so that can be a good alternative to making an anti-nuke outright and you also gain other useful information.
Once scouting becomes more and more expensive, making an anti-nuke is a cheaper alternative than scouting.

Denial of scouting
Spending resources on preventing being scouted means that the enemy have to guess what you are making and increases the likeliness of them making the wrong counter. Are you making a nuke, a detriment, or a Starlight?



quote:
time windows to react to known things
Once scouted it's a matter of how long a counter requires, and of the players execution and skill which in the end affects the metagame greatly. For example if the players can estimate how long it takes them to build an anti-nuke and the enemy to build a nuke, they can decide how frequent they are going to scout or when they are going to make an anti-nuke blind. So game theory evaluating different builds and such.


quote:
macro gaming
unit micro
This seems like its decided by the players skill. Like can you fight on several fronts at once? Can you keep both microing and expanding your economy? Does your enemy have inherent weakness in micro or macro that you can exploit?
Mostly metagaming/game theory.
About different types of unit micro:What I plan to do is to write metrics that sets upper and lower limits. Like if your units all divide fire equally without any overkill they will be so and so effective. Unit X can kite several enemies of type Y even when they are approaching from different directions at an angle of Z.


quote:
element of surprise
This is a lot of things. But mostly metagaming. Can you surprise your enemy? Are you performing a strategy that the enemy have never seen?

quote:
timeing to react and teamwork
Mm... goes mostly into player skill and execution. Like 3 people might be able to perform micro better on 3 fronts than 1 person.

quote:
unit combinations to a certain situation
There are a lot of unit combinations in Zero-K. Although I guess you can evaluate stuff like tanking for other units pretty easily and then implement other specific metrics. Like if you have a Reaper and a Banisher and the Reaper tanks all the damage at first they will a have a cumulative strength of (ReaperDPS+BanisherDPS)*ReaperHealth + BanisherDPS*BanisherHealth

quote:
unit placement on the battlefield (and influence by unit speed...)
This is pretty tricky. Since positioning depends a lot on all the other units and map features, information, kiting, height advantage and resources positioning.
I think that one key to analysing this is to convert Cost to TimeCost. TimeCost is basically how much time of your economy is required to produce a unit.
So say for example that you are rushing 4 Scorchers at the enemy. They will have a relatively high TimeCost as your economy is small at the start of the game. If your enemy makes economy at first instead and then makes 4 scorchers of his own, his Scorchers will have a lower TimeCost as his economy is bigger at that time which means that even if the 4 rushed Scorchers make cost in metal, they might not make cost in TimeCost.
But on the other hand 1 Dart can threaten and harass several positions at once and to counter the single Dart, at least 2 Darts or some other defensive units has to be built so even if the Dart does no damage it can still make TimeCost by forcing the enemy to make defenses and slow down their economic growth.
+2 / -0
There were lot's of interesting musings including various efficiency models described on original caspring.org site.
I think that Detrino has those archived somewhere.

Sadly, I would say that the use of mathematical balance is quite limited in Zero-K game, because usual simplifications don't take into account physical complexities.

Stuff like turret turning/aiming delays, pathing differences, various types of inaccuracies (inacurracy when moving, when firing at moving target), clumping/chaining etc. combined with landscape make pure math comparison only useful as a rough guide.

To get data it's best to use unitguide or ingame unit stats. Work was spent on making basic data like DPS as accurate as possible.

If you get interested into real world stats, tens of thousands historical games were recorded in the "modstats" system, those include real world performance of each unit type vs each other unit type.
+0 / -0

8 years ago
Math balance is super useful when first making a game, you want an algorithm that can inform your baseline stats and remove outliers, assigning weights to various attributes. While many of those weights are arbitrary, you can adjust them in response to playtesting to try and keep it all in a formula so when you add or change something, it informs you where or how to trade it off.

I've been working on Istrolid with treeform recently, this is how I did the initial balance pass but the balance has diverged by empirical feedback since then, I'd like to plug everything into a database and try to find weights that come near our values (Or possibly find outliers again) just to have another pass of that.

Evil4Zerggin did the most comprehensive mathematical analysis of game balance I've ever seen back in CA, I think we brought this up last time we discussed this and you said it was outside of the scope of your thesis, but it was really interesting stuff. It looked at "DPS over expected lifetime" by looking at how much DPS a unit would output before it ran out of HP vs another units DPS. From this you could add in the 'bonus damage' taken when outranged vs the time it takes to close that range. A unit with 10m/s speed and 100m range vs a unit with 5m/s speed and 150m range will take 10 seconds of DPS before it closes and needs more DPS over lifetime for free to compensate (Say, with 10s worth of DPS in extra HP). This doesn't treat range or movement as a flat bonus or multiplier, but actually calculates how much free damage a unit will take in various scenarios. You'll notice that free damage becomes infinite if you are both faster than and have more range than your enemy.

You can also do even more complex things like figure out at what speeds and projectile speeds (or if you want to be more complex, hitvolumes and turnrates too) you can dodge projectiles and how close you have to be before it becomes impossible. There is a sharp slope there between trivial at long range and impossible at point blank where it comes down to micro, but knowing where that slope lies can be useful. Some units have to get into the 'impossible to dodge' end of the slope in order to be in range to fire back, in which case dodging only matters for the 'free' damage (and can slow the unit down enough that the free damage becomes infinite).

Anyway probably outside the scope but fun stuff.
+0 / -0


8 years ago
like most problems, this one could probably be solved with enough ravagers
+3 / -0
quote:
I've been working on Istrolid with treeform recently, this is how I did the initial balance pass but the balance has diverged by empirical feedback since then, I'd like to plug everything into a database and try to find weights that come near our values (Or possibly find outliers again) just to have another pass of that.
I've never heard of that game. Seems like an interesting concept. I think Wargame:Red Dragon does something similar where you preselect units before you go into battle.
I think simple balance metrics is really useful initially. However I hope that by using more advanced metrics according to the design goals of the game you can also use them later to evaluate the balance.

quote:
Evil4Zerggin did the most comprehensive mathematical analysis of game balance I've ever seen back in CA, I think we brought this up last time we discussed this and you said it was outside of the scope of your thesis, but it was really interesting stuff. It looked at "DPS over expected lifetime" by looking at how much DPS a unit would output before it ran out of HP vs another units DPS.
As I said earlier. I don't want to work with statistics. I'd rather work with just raw unit stats.

quote:
From this you could add in the 'bonus damage' taken when outranged vs the time it takes to close that range. A unit with 10m/s speed and 100m range vs a unit with 5m/s speed and 150m range will take 10 seconds of DPS before it closes and needs more DPS over lifetime for free to compensate (Say, with 10s worth of DPS in extra HP). This doesn't treat range or movement as a flat bonus or multiplier, but actually calculates how much free damage a unit will take in various scenarios. You'll notice that free damage becomes infinite if you are both faster than and have more range than your enemy.
This is exactly what I do in the column "units lost on approach" for the unit Dox in this matchup table. http://people.dsv.su.se/~akbj7812/pa-db/unitmatchup/assault_bot/stat.html
"Outranging time" is how long the slower unit will outrange the the faster one(negative values is if the Dox is being outranged). "units lost on approach" is how many units that will be lost on approach. Negative values is how many Doxes that will be lost on approach. Positive values means that the Dox will kill that many enemy units.

quote:
You can also do even more complex things like figure out at what speeds and projectile speeds (or if you want to be more complex, hitvolumes and turnrates too) you can dodge projectiles and how close you have to be before it becomes impossible. There is a sharp slope there between trivial at long range and impossible at point blank where it comes down to micro, but knowing where that slope lies can be useful. Some units have to get into the 'impossible to dodge' end of the slope in order to be in range to fire back, in which case dodging only matters for the 'free' damage (and can slow the unit down enough that the free damage becomes infinite).

Anyway probably outside the scope but fun stuff.
Yes this is a bit more advanced but otherwise it is perfectly in the scope. I just need to derive a formula. :P
+0 / -0
This is incredibly similar to things I have done, for recreation and for my Degree. I will read the rough draft of your paper and give a better response when I have time.

Edit: Wait, that is a rough draft? I might have overestimated the scope of this project. My paper for graduation for a bachelors was well over 100 pages.
+0 / -0
8 years ago
quote:
This is incredibly similar to things I have done, for recreation and for my Degree. I will read the rough draft of your paper and give a better response when I have time.
Cool! Could I read your paper?

quote:
Edit: Wait, that is a rough draft? I might have overestimated the scope of this project. My paper for graduation for a bachelors was well over 100 pages.
Basically we have just been asked to write a scientific paper as it would be published. Scientific papers can be short or long. Like if you do a survey, you don't need 100 pages to present the results.
+0 / -0
Page of 3 (45 records)