Combat results

[Vev]

Hi Verrucosus,

Sounds great you wish to help. The odds based on attacking strength vs defense is accurate. e.g.

green impact rover vs disciplined synthmetal infantry on flat = 4 * 7/8 * 5/4 : 2 = 35/8 : 2 which is roughly 3:2

Now this is assuming equal hits at 10 each. The way they used health to modify the odds is wacky and they do it by straight multiplication of the odds. This approximation is not a truely accurate one.

There is a hypotheses I would like to test. Have combat done with 3:2 odds combats but with all units at 1 health remaining. Then do it with all units at 2 health remaining. I am trying to prove or disprove that multiple damages can be done in one round.

I have modified the problem to crunch through combinations of number of rounds fought and also restricted the output.

[Verrucosus]

Hello again,

as suggested, I tested a combat situation where a green impact rover attacks a disciplined synthmetal garrison on flat terrain.

1st test: Both units have just 1 hitpoint left.

In that situation there is a single round of combat the loser of which is destroyed. Since both units are equally damaged the odds of winning that single round equal the unit strengths as follows:

Impact Rover: 4 * 0,875 * 1,25 = 4,375
Synthmetal Garrison: 2
Strength Ratio = Odds = 35 : 16

This is exactly the prediction given by the "odds" message. The probability of victory for the rover is about 69 %, that of victory for the garrison is about 31 %.

As a test, I resolved this combat situation 20 times, of which the rover won 14. This was exactly as expected, so I decided not do any further testing.


2nd test: Both units have just 2 hitpoints left.

In this situation it will take either two or three rounds to resolve the combat. Assuming that there are no hidden modifiers the probabilities are as follows:

Probability of 2:0 victory for the rover: 0.69 * 0.69 = 0,47 (47 %)
Probability of 2:1 victory for the rover: 0.31 * 0.69 * 0.69 * 2 = 0,3 (30%)
Probability of 1:2 defeat for the rover: 0.69 * 0.31 * 0.31 * 2 = 0.13 (13 %)
Probability of 0:2 defeat for the rover: 0.31 * 0.31 = 0.1 (10 %)
Total probability of rover victory: 77 %
Total probability of rover defeat: 23 %

As a test, the combat situation was resolved 100 times. There was not a single combat where I was able to observe multiple damage inflicted during a single round. This supports Sikander's suggestion that the appearance of multiple hits is just the result of the computer displaying the combat as no more than three or four exchanges with each exchange actually comprising several different rounds of combat. This would explain why there are no "multiple hits" when the combat cannot exceed three rounds anyway.
The test results did not match the probabilities calculated above:

2:0 victories for the rover: 57
2:1 victories for the rover: 23
1:2 defeats for the rover: 8
0:2 defeats for the rover: 12
total rover victories: 80
total rover defeats: 20

The rover won more often than expected and, more importantly perhaps, the chance of 2:1 victoriy for either unit was lower than expected.

Of course, the simplest explanation would be that 100 tests do not really "test" the probabilities calculated above. Putting that (entirely justified) objection to one side for a moment, it is tempting to try to recalculate the probabilities by introducing a hidden combat modifier in the form of a stength penalty to the unit with less hit points as follows:

If the rover wins the first round, its chance of winning the second is 35 : 16/2 or 81 %. The garrison's chance of winning the second round is 19 %.
If the rover loses the first round, its chance of winning the second is 35/2 : 16 or 52 %. The garrison's chance of winning the second round is 48 %.

The probabilities would be recalculated as follows:
Probability of 2:0 victory for the rover: 0.69 * 0.81 = 0.56 (56 %)
Probability of 2:1 victory for the rover after winning the first round: 0.69 * 0.19 * 0.69 = 0.09 (9 %)
Probability of 2:1 victory for the rover after losing the first round: 0.31 * 0.52 * 0.69 = 0.11 (11 %)
Total probability of 2:1 victory for the rover: 20 %
Probability of 1:2 defeat for the rover after winning the first round: 0.69 * 0.19 * 0.31 = 0.04 (4 %)
Probability of 1:2 defeat for the rover after losing the first round: 0.31 * 0.52 * 0.31 = 0.05 (5 %)
Total Probability of 1:2 defeat for the rover: 9 %
Probability of 0:2 defeat for the rover: 0.31 * 0.48 = 0.15 (15 %)

These calculations do not fit the test results perfectly, but they would explain why three round combats are a lot rarer than one would expect if the odds of winning a round were unaffected by the results of previous rounds. If they are correct, the "odds"-message is correct even for damaged units in the sense that it gives an accurate probability for the first round of combat.

Regards,
Verrucosus

[Vev]

Nice. Collecting raw data is not for everyone, not at least for an extended period of time. But I really appreciate your efforts as any data is useful data. I've created a program that tries to simulate combat rounds. There are still a lot of parameter tweaking to do to try to find out why the results are as such. Your effort will help a lot to figure out what is happening. What would be nice is to figure what sort of experiment best tests the current hypothesis.

[Vev]

Verrucosus :
Okay as you might have gathered the 2 hit experiment was to test multiple damage theory. The testing you had done disproved it. Now, using my program to model the outcomes I trial them at batches of 100 rounds. A majority (50%+) gives outcomes close to the estimated expected values. However within 8/9th trial I could get results similar to what you gained from SMAX. I could also gains results that gives close to the number of winning outcomes similar to was recorded on the Excel spreadsheet within 10 goes. The results begin to average out with interations of 1000 rounds and many magnitudes higher. But some strange is still going on.

I might have to not just come up with most likely outcomes, but also a standard deviation of other outcomes.

From this I was thinking how do we try to automate as much as possible the data collection from SMAX?

One way is to set all the units up and let the computer do the attacking. So set once, repeat many. To speed things up, set the morale of all the units to be elite so you don't have to worry about morale upgrade. Unless you can disable that pop-up message completely.

A problem with that is the computer will not attack if the odds are poor. This can be overcome by giving the computer the better odds of winning.

Also we can save time altering the battle odds by changing the rules in Alphax.txt to set the armour and weapon to other values.

Although I haven't confirmed it yet, my friend said mindworms will attack regardless of odds. But the down side with that is you cannot set the lifecycle of the native lifeforms. You have to individually change their morale.

[Kody]

Sorry that was an over simplification I made.

Mindworms will attack any normal adjacent units regardless of the odds. I think they ignore native life (I've only read that, someone should double check), and if they're 2 squares away they will move in a direction at random. The chance of a mindworm attacking a unit is unmodified by defensive strength or whether it's a civilian unit (from the tests I've done, so the worms don't specifically target your colony pod).

From what I've observed they seem to always attack adjacent bases if given the choice of a base or a unit. Someone else I've talked to observed a mindworm that moved one square through fungus and didn't attack a base.

Anyway I think in the map editor you can change a normal unit's control to "alien". So it'll ignore battle odds and should attack all the time.

[MariOne]

Nice work Verrucosus.
Did you look into my "combat test" spreadsheet?
It reports data collected from 100-runs tests in 4 different situations.
Reducing the variables as you did by lowering the HPs is a very good approach to pinpoint and isolate the factors influencing the event.
Yet, you see from my test that by simply giving a *fixed* extra bonus modifier to the stronger untis, without changing it from round to round as HPs were progressively lost, the theoretical curve pretty much matched the observed results for partial damages too.
What you could INDEED see from my tests when many HPs were available, was an *alternating* hi-lo outcome for even/odds remainin HPs.
For instance, in the first tab (2 v 1fusion), the observed 6, 4 & 2 HP left bars for the laser attacker were higher than theory, while the 7, 5, 3 & 1 HP left were lower.

Same in the other tabs. I didn't give inportance to that, dismissing it as a fluke due to the few runs.

You might be onto something, but I would suggest you that while reducing to 2 HPs might help you pinpoint the factors, it might at the same time make you miss a bigger picture...
Keep on testing, I'm really interested in your work and I like your approach.

_____

Vev, I fail to follow you in your testing setup assumptions.
The way I did my tests, was to set up with the scenario (Ctrl+K a normal game to make it easier) a combat situation with the 2 units of the desired strengths and features side to side.
Then I just attcked, noted down, RELOADED (Ctrl+L), attacked, noted down, reloaded, ... and so on.

HOW would you let SMAC automatically repeat hundreds times a combat in the same situation, without reloading and without your intervention? And how would you record the data?

Of course it's SMAC (or SMAX) that we want to observe, so a simulator you might program can be useful to produce *comparative* data to devise a theory about the observed specimen, but you can't collect data from your simulator and say "I've collected SMAC data", as until we understand how SMAC itself works we have nothing to *substitute* its original behavior... indeed

[Verrucosus]

As far as I remember my pod-popping expeditions, mindworms will not attack your native unit unless attacked by it. For example, if your mindworm finds an infested pod and causes four mindworms to appear, they will leave your unit alone unless you use it to attack them. (Sometimes you have to because the stupid worms keep hanging around and block important squares.)

[Vev]

I did set up 100 vs 100 units in a single player save file, set other side to have atrocity vendetta and do 'End Turn' and wait for all 100 combats to resolve itself then next turn count up the remaining hit points. That needs a bit of tweaking to get it right. The great part about that is you could get 1000 combats in ten cycles.

I'll try to find the save file.

[Vev]

Here is the file, all 94 units set up ready to combat the other 94 units. Do nothing and hit 'end turn' and the other side will take care of matters. Save the file after you finish for later perusal when counting up the health remaining.

There are only one other opponent left and the map is tiny.

I tried to reduce the amount of pop-ups as possible. e.g. morale improvements, first battle won. The great thing is that you could give singularity tech and use workshop to upgrade all to that engine.

One my very slow P2 400 it takes 4-5 minutes to go through all 94 combats. I think it is mainly delays for animation that is slowing things down.

You could use Command Report to instantly see how many of the units survived to get a picture of the combat odds.

The point of all this is so that I could get my hands on 1000+ combat results.

Save File

[Verrucosus]

Vev, that's an intriguing way of testing. I'll have a look at the file over the weekend.

MariOne, thanks for your encouraging comments. I have to say that I just followed Vev's suggestion in reducing the hitpoints. The tests themselves were conducted by the method you have described (reloading the same scenario time and time again).
I had looked at your test results earlier, but (being a law student) I am not used to read data sheets and had some difficulty to figure out what the figures exactly meant. After having done a bit of testing myself now, I'll study your results again.

I agree that by reducing the hitpoints the testing is limited to very short combats which might tweak the results. What I liked about the approach is that, apart from testing whether there are multiple hits (which was Vev's concern), a "hidden" combat modifier for losing hitpoints (if there is one) is larger for the first hitpoint lost the fewer hitpoints the units have at the start of the combat. If you start with healthy units, the unit losing the first round would receive only a 0.9 multiplier to its combat strength (assuming fission power) rather than the 0.5 multiplier I used in my recalculation. The loser's chance to win the second round and restore the original odds again for the third would be significant. As a result, I'm afraid that it would take a much larger number of tests to reveal a hidden modifier in a combat with healthy units.

I will not have the time to do any additional testing myself before next year, but one idea you could try would be to repeat the test reported above with more uneven odds (for example disciplined chaos vs. synthmetal infantry units on flat terrain). This time we should record only the three-round-combats, but for those combats also the order in which the hits are inflicted.

Probabilities according to Sikander's model:
2:1 victory for chaos inf. after winning the first round: 0.8 * 0.2 * 0.8 = 0,128 (13 %)
2:1 victory for chaos inf. after losing the first round: 0.2 * 0.8 * 0.8 = 0,128 (13 %)
1:2 defeat for chaos inf. after winning the first round: 0.8 * 0.2 * 0.2 = 0.032 (3 %)
1:2 defeat for chaos inf. after losing the first round: 0.2 * 0.8 * 0.2 = 0.032 (3 %)

Under Sikander's model, unsurprisingly, a unit's chance of winning a 2:1 victory is unaffected by the results of the first round of combat.

If there was a hidden modifier corresponding to hit point ratio, the odds for the second round would change as follows:
If the attacker wins the first round its chance of winning the second is 8:1 or 89 %. (Defender's chance of winning the 2nd round: 11 %)
If the attacker loses the first round its chance of winning the second is 4:2 or 67 %. (Defender's chance of winning the second round: 33 %)

Probabilities assuming "hidden" modifier:
2:1 victory for chaos inf. after winning the first round: 0.8 * 0.11 * 0.8 = 0.0704 (7 %)
2:1 victory for chaos inf. after losing the first round: 0.2 * 0.67 * 0.8 = 0.1072 (11 %)
1:2 defeat for chaos inf. after winning the first round: 0.8 *0.11 * 0.2 = 0.0176 (2 %)
1:2 defeat for chaos inf. after losing the first round: 0.2 * 0.67 * 0.2 = 0.0268 (3 %)

As you can see, if there is a hidden modifier, the probability is different not just for different results but also for different event sequences leading to the same result. This is a key difference to Sikander's model which might provide a good way of testing the "hidden modifier" theory.

The problem is, of course, that such a test would require not just the result of the combat, but also the order in which the hits are inflicted to be recorded.

Regards,
Verrucosus

[Vev]

Now if there was a quick way to change/set all unit's hits, that would be really good.

[Vev]

more saves

Here are saves for 20 vs 20 5:4 for hits ranging from 1 to 8

Found out it was quite easy to do. Set everyone to have 1 hit remaining, set all to peace, except for one other to prevent victory. Then keep waiting until they heal to 8 hits.

The 8 hits save file is a bit wonky and may not be really that useful since I was force to use scenario building mode.

Another nifty function is using the Security Nexus to automate unit of one type and if you space the units out correct, your guys will attack.

What would be nice is to turn off animation completely but I have no idea how to do that.

Just found out how, play as another faction that than those two and they resolve everything in their own turn without having to see all the annoying animations.

May have to include another faction unit, e.g. colony pod, switch to their side to make it work. Great thing is that all 94 rounds are done in under four seconds. Finding out how many won is just as easy with F7 (helps if you set so that the other side is inflitrated). Counting the remaining hitpoints would unfortunately involve your time that increases linearly to the number of units used. Just make sure the sum of units on both sides adds to 94.

The 'End Turn' thing doesn't work for the saves with units less than 4 hits as the opponents would rest until they have 4. For that case, use the automated units to get your own units to attack the opponents.

[Vev]

Just to give you an idea how quickly results can be gathered, 1034 rounds collected under a minute.

5:4 both fission engine
848 to 186

[Verrucosus]

Hmm, this is extremely useful, particularly for tests where only the number of combats won is decisive.

I'd like to return to the test results posted by MariOne which I have read again with interest. I wonder if it is possible to recalculate the chance of each outcome with a variable modifier for the combatants' changing hitpoint ratio without having to calculate the probability of each possible sequence one at a time before adding them up. (For example, there are 8855 possible sequences leading to the 20:4 result that was reached in 20 % of MariOne's 2 vs 1*2 tests.) Is there a formula that can be used to do this calculation faster?

[Vev]

Well, there could be an algorithm for doing this, but my maths is quite rusty. What I would probably end up doing, if I have time, is a state search tracking down the state probabilities.

Just had an idea, if we could set the hits to be staggered. The unit with the better attacking strength has 4-5 health points while the weaker strength has 1 health point. This should reduce the number of states to worry about and check if there is anything strange with repeated wins.