PDA

View Full Version : pK?



kbluck
02 Oct 03, 19:45
I've become a little suspicious of how the pKs are working. Based on a fair amount of controlled playtesting, it seems to me that vehicles are being killed quite a bit more frequently than the relevant pK would seem to indicate.

For example, you know I think small arms pKs against infantry should be quite a bit lower. Well, they are in my database, and yet infantry typically still dies within just a few bursts of MG fire, usually less than 10. And, not all of those bursts are hitting, as evidenced by the lack of suppression. Such a situation should be very unlikely statistically, given that the pKs are generally set in the 2-5% range for most MGs in my database.

When tanks are attacked by AT-5s, even from the front where an M1A2 is listed as only about a 1/3 chance of dying --- they nearly always are killed. It is rare that a tank survives a hit from an AT-5, from any angle. Yet, you'd expect the tank to survive front hits 2 out of 3 times.

Very perplexing indeed.

--- Kevin

kbluck
02 Oct 03, 20:08
Just as a bit of wild speculation: The symptoms I'm seeing are exactly consistent with a random number generator that isn't producing a full range of numbers.

For example, if by chance the RNG for pK is set to produce numbers in the range 1-10 instead of 1-100, you'd get exactly what I'm seeing: things with low pKs like MGs vs. infantry would not get killed every time, but would get killed about 10 times more often than you'd expect, while things with pKs >= 10%, like AT-5 vs. M1A2, would kill every time they hit.

Whatever the cause, I also wonder if the same syndrome isn't afflicting pH as well. Long-range fires seem awfully accurate to me, although without fully understanding the pH I can't really say for sure.

--- Kevin

Pat Proctor
02 Oct 03, 23:43
You will be able to judge this better than I, but could it be lag. If, for instance, a M-16 fires every two seconds, and it is four seconds between the first time a unit is "cycled" in the engine and the next time the engine checks on it, the engine will compensate for the lag by firing the weapon twice. This will only appear as a single burst to you, the viewer, watching the engagement, but, in fact, their were two engagements, and two sets of die rolls (pH and then, if necessary, pK). The results will appear as if the weapon is twice as effective as they actually are.

The only way to test this is to build a scenario with only 4 or 5 vehicles per side and run it, to see if the symptoms persist. And that may be more trouble than you want to go to.

kbluck
03 Oct 03, 13:47
The only way to test this is to build a scenario with only 4 or 5 vehicles per side and run it, to see if the symptoms persist. And that may be more trouble than you want to go to.

No, it's no trouble at all. See attached.

I'm sure its not lag. The clock was running normally. But, just to be extra sure, I've set up a little live-fire range on the lower-left corner of the attached scenario.

When I ran it just before zipping it up, my blue fire team destroyed seven enemy teams within 30 seconds. Looking at the spot reports, all seven were destroyed with either SAW or M16 fire, although there was the occasional AT4 or M203 shot. By my calculations, the team could possibly have generated no more than 30 M16 bursts and 15 SAW bursts in that time frame. Assuming every burst hit (which, judging by the lack of suppression indicator, many did not) that's a kill rate of about 16%. You'll note from the vehicle specs that the very best pK (M249 from the rear) is set to 4%. I think that's a significant statistical anomaly.

Results will vary, of course, but it seems to me that most of the bursts that fail to kill are the result of misses. Suppression indicators flash only very infrequently, which indicates to me that most bursts are missing. I'd guess the true kill rate of hits is in the 30-40% range.

I'd recommend downloading my latest database to be sure we're on the same page with this test:

http://www.netce.com/atf/kbdatamodern.zip

--- Kevin

Pat Proctor
03 Oct 03, 21:31
The random number generator works using the processor clock, rather than the "randomness" in the Windows API. It may turn out that the error is hardware specific (this still makes it an ATF bug, as the game SHOULD work, essentially, the same on every computer).

I encourage as many people as are interested and have the time to also download this scenario and give it a spin, and let me know what you encounter.

I would certainly like to squash this bug if it is re-creatable.

kbluck
04 Oct 03, 17:30
I generated a different "test" scenario whose results are a little easier to track. This one features a T-90 firing on M1A2s. The M1A2s are all facing the T-90, so the front pK should be used, and that front pK is set to 12%. It's attached below.

Note to downloaders: both "Test" scenarios depend on the Bihac map contained in the Scenario Pack released with 1.03, so you'll need that to run them, along with my custom DB.

I ran two tests each on two different computers, a desktop and a laptop, which are about as different hardware-wise as you can be. Both are running Win2000. Here are the results:

Desktop Run#1: 20 shots, 16 hits, 10 kills = 62% actual pK.
Desktop Run#2: 24 shots, 18 hits, 10 kills = 55% actual pK.
Laptop #1: 32 shots, 23 hits, 10 kills = 43% actual pK.
Laptop #2: 24 shots, 18 hits, 10 kills = 55% actual pK.

I counted any shot where a suppression marker didn't appear as a "miss."

So, while there is some random variation, as you'd expect, the two platforms are in the same ballpark of being 4-5 times too high on the expected pK.

--- Kevin

kbluck
02 Feb 04, 12:57
So, did anything ever come of this? I'm still noticing that things get killed a lot more often than they should judging by pK.

Has anybody else been able to reproduce my results?

--- Kevin

CPangracs
02 Feb 04, 14:05
So, did anything ever come of this? I'm still noticing that things get killed a lot more often than they should judging by pK.

Has anybody else been able to reproduce my results?

--- Kevin
Well, I would love to test this for you, kb, but I downloaded your updated database and I get 10 AH-64D's facing an M3A3 Dever's!:nuts:

kbluck
02 Feb 04, 15:38
Well, I would love to test this for you, kb, but I downloaded your updated database and I get 10 AH-64D's facing an M3A3 Dever's!

To be expected, since the earlier test scenario was built with the "old" version.

Here is an updated test, for those who are using the "new" kbDataModern.

I just re-ran it, and got similar results; 27 shots, 15 hits, 10 kills = 66% actual pK. Even if you count shots rather than hits, the kill rate per shot was 37%; much higher than the expected pK of 12%.

--- Kevin

jdeluise
03 Feb 04, 02:32
I ran the test today three different times using the pre-1/02/04 kbluck database and the test.zip from 10/4/03. One thing I noticed was that it seemed that my T-90 hit every time. Every shot causes a suppression marker to appear. Sometimes, the suppression marker would disappear as soon as the splash mark disappeared. However, since a suppression marker DID appear I counted it as a hit.
The other thing I noticed was that according to the spot report log sometimes the ATGM and the main gun would fire at the same target simultaneously. I surmise they are firing at the same target because there was only one fire line drawn on the map but two report entries. Anyhow, here are my results. Like I said, they might be inaccurate because of how I'm counting a hit, maybe kbluck could explain in a little more detail how a "miss" is determined, because I always saw a suppression marker on every shot.

#1: 22 shots, 22 hits, 10 kills = 45%
#2: 25 shots, 25 hits, 10 kills = 40%
#3: 16 shots, 16 hits, 10 kills = 62%

kbluck
03 Feb 04, 13:09
You're right, the gun-launched ATGM does confuse the issue a bit. Here is another test with a T-72M instead, which has no ATGM. You should see nothing but 125mm shots. My test runs still show a kill rate in the 40-60 percent range.

However, it did surface another interesting phenomenon. Sometimes, it seems to get "stuck" on a target. It'll blow through 4 or 5 kills in quick succession, then take a dozen shots or so to kill the next one (which is actually what you'd expect, given the 12% pK) then rip through the rest again. This was being concealed before, since the ATGM frequently followed up a failed kill with the gun.

--- Kevin

kbluck
03 Feb 04, 13:41
Just to poke a bit more, I played with the pK.

Changing the front to 0% prevents the T-72 from firing. So, it appears the game is referring to the correct aspect.

Changing the front pK to 1% slows down the carnage, but it is still surprisingly high. You'd expect a better than 50% chance that the T-72 wouldn't kill *anything* before it runs out of ammo, yet it blew away 4 targets. Again, it seemed "bursty", spending 15 shots on the first target, then mowing down the next two with only 4 shots. Possible, but statistically very, very unlikely. These results are well inside the 5% confidence level (usually considered definitive) that some anomaly is occuring.

A couple of other strange issues I noted with the low pK:

Sometimes, after taking a couple of shots, the target will start moving. I can't see why; they have no SOPs set and the move flag is set for 900+ hours. At any rate, sometimes they complete their move and get killed when they turn away, but one tank will finish, turn north (thus presenting its flank) and the T-72 will continue shooting at it, but seem unable to hit it even though the target is very close. It was being hit while it was moving; only after it stops does the T-72 start failing to hit.

--- Kevin

HoustonAerosFan
03 Feb 04, 14:06
Just to poke a bit more, I played with the pK.

Changing the front to 0% prevents the T-72 from firing. So, it appears the game is referring to the correct aspect.

Changing the front pK to 1% slows down the carnage, but it is still surprisingly high. You'd expect a better than 50% chance that the T-72 wouldn't kill *anything* before it runs out of ammo, yet it blew away 4 targets. Again, it seemed "bursty", spending 15 shots on the first target, then mowing down the next two with only 4 shots. Possible, but statistically very, very unlikely. These results are well inside the 5% confidence level (usually considered definitive) that some anomaly is occuring.

A couple of other strange issues I noted with the low pK:

Sometimes, after taking a couple of shots, the target will start moving. I can't see why; they have no SOPs set and the move flag is set for 900+ hours. At any rate, sometimes they complete their move and get killed when they turn away, but one tank will finish, turn north (thus presenting its flank) and the T-72 will continue shooting at it, but seem unable to hit it even though the target is very close. It was being hit while it was moving; only after it stops does the T-72 start failing to hit.

--- Kevin

Is there a range effect? How far apart are the two sides?

kbluck
03 Feb 04, 15:04
Is there a range effect? How far apart are the two sides?

Well within range of the weapon system.

If you are able, please download and run the test and report your results. Pat Proctor has surmised that it may be a hardware issue with the random generator; it will be useful to test this on the widest possible variety of systems.

--- Kevin

CPangracs
03 Feb 04, 15:38
Well within range of the weapon system.

If you are able, please download and run the test and report your results. Pat Proctor has surmised that it may be a hardware issue with the random generator; it will be useful to test this on the widest possible variety of systems.

--- Kevin
Well, now I get 10 Mortar Teams vs a BTR-D! LMAO!

I may have to just create my own test...:nuts:

jdeluise
03 Feb 04, 16:15
I will try the new test again tonight and post my results. But kbluck, do you see a suppression marker on every shot? I certainly did and counted all of them as hits(though many of them were very short-lived). However, I noticed that on your stats for the test, you had a number of shots with no hits. What's going on??

kbluck
03 Feb 04, 16:39
Curt: (and anybody else experiencing similar issues)

http://www.netce.com/atf/kbDataModern.zip
http://www.netce.com/atf/Test.zip

Download both of these. Unzip them both into your Data folder. If you still get weird results, something is very odd about your configuration. Perhaps your Raging Tiger updated version? But I suspect it will work once you are synched up with the correct file versions.

jdeluise:

Some shots, around 30% or so, on my system do not show a suppression marker at all. I haven't observed the specific syndrome you describe. But it is interesting when things act differently on different systems. Let's see what happens when the ATGM is out of the picture.

--- Kevin

Scully
03 Feb 04, 22:36
Hey KB,
I downloaded both files tonight, but when I went to run the test I got an error. It said:

Atf has caused an error in ATF.EXE

It only occurred with the test scenario, the others, including Traffic Stop, worked fine.

I tried redownloading 3 times with the same result. Any thoughts on what could be wrong?

Thanks,
Brian

Scully
03 Feb 04, 23:24
Ok...I figure out what was wrong and ran the test 3 times. I got some funky results:

Test 1
29 shots
17 hits
10 kills

Test 2
45 shots
18 hits
9 kills (ran out of ammo)

Test 3
45 shots
16 hits
6 kills (ran out of ammo)

A couple of interesting notes:
1. The far right tank moved forward in tests 2 and 3 and didn't get killed until he appeard to turn around for a rear shot.
2. The 1st tank was shot at 15 times during the second test before getting killed.
3. The 7th tank, which was never killed during the 3rd test, actually drove up and turned to be side by side with the T-70 and took 8 shots in the side without getting killed.

I'm wondering if speed of computer has anything to do with results. I have a slow computer P3 733 with 200 and somethin RAM.

I hope this is helpful,
Brian

Ivan Rapkinov
04 Feb 04, 00:13
yep - I think comp has something to do with it

I got normal results on the 800MHz 256MB ram machine, and completely wierd results on the 3GHz 2GB ram machine

Pat Proctor
04 Feb 04, 00:56
The original random number generator seemed not to be acting to random, so we switched to a system that keys off of the system clock.

Every system we tested on, over 1000 data points, appeared to be completely unbiased, providing an even spread over the complete range.

I think it could be one of the following

a) The results are "clumped", i.e. they migrate across the range, providing repetitive results in a much narrower range and slowly drifing across the entire range.
b) Different systems that we did NOT test respond to the randomness in different ways.

If there are any programmers out there that have any links to code that increases the randomness of results, feel free to send them. Otherwise, we will continue to slog through it with the resources at our disposal.

On the separate, pH question. There are a multitude of factors that contribute to whether hits occur. Among these is the range to target.

I guess what I am saying is that, in order to test pH issues, we need a save file that represents exactly the situation you are working with.

kbluck
04 Feb 04, 01:27
If there are any programmers out there that have any links to code that increases the randomness of results, feel free to send them. Otherwise, we will continue to slog through it with the resources at our disposal..

http://www.boost.org

The distribution includes a very complete random number system, offering several different generators and distributions in well-implemented and peer-reviewed C++. For the initial seed, I suggest starting with 32 random bits (good source at: http://www.fourmilab.ch/hotbits ) in a file, then updating the file with new bits every time the game runs by CRC32 hashing the current bits with the current system clock DWORD as derived from clock() ( or GetTickCount(), if you prefer.) Boost also provides a CRC library.

Worst case, why not just use the standard C rand()? It may not be good enough for strong encryption, but legions of game software authors have found it to be more than adequate. If the problem really is the random generator, it certainly can't be worse.





On the separate, pH question. There are a multitude of factors that contribute to whether hits occur. Among these is the range to target.

I guess what I am saying is that, in order to test pH issues, we need a save file that represents exactly the situation you are working with.

This is as controlled a situation as I can make it. All the accounts you read in this thread were with the same test scenario. No variations on terrain or range or facing. Take the test scenario and run it on a few different computers. That should provide all the debugging fodder you need.

--- Kevin

kbluck
04 Feb 04, 12:28
yep - I think comp has something to do with it

I got normal results on the 800MHz 256MB ram machine, and completely wierd results on the 3GHz 2GB ram machine

I'm curious as to what you consider "normal results". Statistically speaking, given a 12% pK, it should take an average of 8 *hits* (not shots) to kill the M1A2 from the front aspect. Given that the T-72 has only 39 shots, you should expect to see around 5 kills on average before running out of ammo, and probably less in reality since you won't hit on every shot.

Is that what you observed on your 800MHz machine?

--- Kevin

Scully
04 Feb 04, 22:43
[QUOTE=Pat Proctor]a) The results are "clumped", i.e. they migrate across the range, providing repetitive results in a much narrower range and slowly drifing across the entire range.[QUOTE=Pat Proctor]

I definitely noticed this. Some took up to 15 plus shots, while some were killed with 1 shot. I don't think there's anything wrong with that though.

Brian

Scully
04 Feb 04, 22:44
I'm curious as to what you consider "normal results". Statistically speaking, given a 12% pK, it should take an average of 8 *hits* (not shots) to kill the M1A2 from the front aspect. Given that the T-72 has only 39 shots, you should expect to see around 5 kills on average before running out of ammo, and probably less in reality since you won't hit on every shot.

Is that what you observed on your 800MHz machine?

--- Kevin

I came fairly close to this on my last test, but it's still a little off. By the way, I had 45 shots with my T-72. A little overstocked I guess. :)

jdeluise
04 Feb 04, 23:58
I ran the test with the new db and the new test.zip, here are my results

Test #1: 33 shots, 32 hits, 10 kills = 31%
Test #2: 40 shots, 34 hits, 6 kills = ~18%

On the first test, as you can see, my tank only missed once. The last enemy unit remaining began driving towards my tank before it got killed.
One the second test, the the fifth enemy unit to get killed drove towards my tank before getting killed. The sixth (and last before ammo ran out) enemy unit drove toward me and parked in front of me. From then on, my tank missed every shot (about 8-10 shots in a row, maybe more) until it ran out of ammo.
In both tests, I noticed the "clumping" effect where the tank fires several times in a row and produces no kills, then kills multiple tanks in a row.
One thing I'd like to point out is that on the second test, I think the pK is a little bit scewed because the tank fired 8-10 shots in a row that MISSED. Obviously, there was something about the landscape or the range or something that was causing the pH to be very low. If the tank had chosen a different target (with a better pH), I believe that the pK would have been higher, but of course that's just speculation.
Just wanted to pass those results on!

jdeluise
05 Feb 04, 00:00
I just wanted to point out I made a clerical error on test #2 results, test #2 should have read 45 shots, not 40. Not that it makes a difference on the pK, but wanted to be accurate:)

Pat Proctor
05 Feb 04, 12:40
I will be incorporating all of this data into v 1.04 patch, which I hope to have complete by the end of March.

kbluck
05 Feb 04, 13:24
OK, techie speculation time.

Pat mentioned that the random generator "keys off the system clock". I'm not sure exactly what that means. If he means that the random generator operates by actually sampling bits out of the system clock every time it generates a number, then that is a serious design error. Given that computer instructions execute in a discrete number of CPU ticks, when you're working in a tight loop you're likely to find "harmonic" effects, where the period of the loop is close to a regular multiple of the timer interval, and you will indeed get "clumping", in much the same way as a moving pendulum can seem motionless if you synchronize a strobe light to match its frequency. The exact harmonic effects would certainly be hardware dependent, since different CPUs and memory configurations would execute instructions at different speeds. This would be true even with the "high-resolution" timers available under Windows. Apparently, Pat had the bad fortune to test only on machines which did not display any obvious harmonic effects.

If this is in fact what was done, these odd results become easy to explain. They can be solved by throwing out all that timing code and instead using plain old std::rand(), which for gaming purposes will produce a very convincing illusion of randomness that will not repeat itself in a lifetime of play. No need to reinvent the wheel on this one.

Of course, not having access to the actual code, I could be completely off base. But that is my best theory.

--- Kevin

Pat Proctor
11 Feb 04, 22:30
For those who were concerned about the randomness of the ATF engine, we HAVE went back, made some tweaks, and will release the changes in version 1.04 (stay tuned for release dates).

I have enclosed an excel spreadsheet which shows a couple sample runs of the generator, the intent being to show that the results are sufficiently random and not "bunched" or tending toward above or below mean (there is almost certainly a better way to depict this in excel, but statistics are not really my strong suit).

Comments are welcome and, of course, one can never predict the "customer's computer factor", but I did want everyone to know that I took this concern very seriously and DO appreciate the work everyone has done in playtesting.

kbluck
12 Feb 04, 13:15
I have enclosed an excel spreadsheet which shows a couple sample runs of the generator, the intent being to show that the results are sufficiently random and not "bunched" or tending toward above or below mean (there is almost certainly a better way to depict this in excel, but statistics are not really my strong suit). Comments are welcome and, of course, one can never predict the "customer's computer factor", but I did want everyone to know that I took this concern very seriously and DO appreciate the work everyone has done in playtesting.

For the entire series, mean and median are around 47 and standard deviation hovers around 28. Skewed a bit low, but within expected variance for a sample of this size, and the distribution appears reasonably uniform based on kurtosis.

But it seemed that way last time you tested it too, right?

There are several PRNG algorithms that are well-tested, commonly implemented, and consistently produce results with good random qualities. There are several others that are not so good, but their quirks are well-understood. None of them are hardware-dependent in any way, being deterministic and reproducible mathematical sequences. If you are using one of them, and seeding it correctly, you should have no further problems on this score. If you are reinventing the wheel with some new, unproven technique of your own, all bets are off.

BTW, using a deterministic algorithm is actually useful. It means that if you give it the same seed, it will produce the same sequence. This is quite useful for debugging, since you can then rerun the same test case with no "random" variation.

If you can give a specific summary of what you're doing to generate numbers, I can give you a technical opinion on its soundness. Otherwise, we'll just have to wait for the patch and see how it plays.

--- Kevin

Pat Proctor
12 Feb 04, 20:21
As a rule, I try not to talk about how code is written, but you have an honest login ;)

I have gone back to rand() but have used a different method to seed the generator which should, hopefully, yield more variable startpoints.

Philip
13 Feb 04, 01:04
I have a look at the data too. I tested to see if the two data sets conformed to a uniform distribution using the non-parametric Mann-Whitney and where found to not significantly different. However it was also found to be not significantly different from a normal distribution as well. This is probably due to sample size which is small (n=139).

However even if the distribution was uniform it still does not prove it was random or not significantly from random. To prove this is very difficult and beyond my humble capabilities this can only really be done by examining the algoritham as far a I know.

As people have pointed out that there does appear to be some discrepancy between computers of different clock speeds. I would guess that the tests done using Kevin's Test file are not significant given the number of tests run so far, which is small. A simple way to see if anything really is wrong is to run the tests that you have been doing on two very different machines but it would be best over about 1000 repetitions repeated at least seven times for each machine if this does not take too long! With this amount of testing you should be able to indeed demonstrate that the distribution is indeed uniform and by examining it as a time series you may or may not find some sort of harmonics.

I am not a statatician therefore anything I or anyone else who is not should be treated with caution. Statistics is a mine field and a little knowledge is extremely dangerous.

kbluck
13 Feb 04, 01:17
rand() should be a big improvement. It will eliminate hardware dependencies, at least. You can probably stop there if you want.

However, quality of the rand() function will vary between compiler vendors, and its hard to know which are good and which are so-so. Boost is a high-quality implementation. It has the advantage of probably being significantly more efficient than vendor rand() as well. Since we're in a sharing mood, I've attached a bit of sample code compliments of yours truly about how to utilize their library for your consideration. It also contains a simple method of seeding with the system clock that will do a better job of spreading around the seed bits than using the tick counter directly, which tends to leave the high-order bits zeroed. A better method would be to save the current seed in a file between runs and hash that in too the next time, so each seed also incorporates the randomness of all the prior runs of the program. That's better than a lot of so-called cryptographic programs do.

Feel free to use it as is or suitably modified. You'll need to obtain the Boost distribution from www.boost.org, as well. Don't worry, their license is very open, even less restrictive than LGPL, and you won't obligate yourself to release source code if you use theirs.

Regards,

--- Kevin

jdeluise
14 Sep 04, 02:50
Old thread, and hopefully rand() has solved the problem...but another high-quality random-number generator unencumbered by licensing at http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html
From the site...


# It is designed with consideration on the flaws of various existing generators.
# The algorithm is coded into a C-source downloadable below.
# Far longer period and far higher order of equidistribution than any other implemented generators. (It is proved that the period is 2^19937-1, and 623-dimensional equidistribution property is assured.)
# Fast generation. (Although it depends on the system, it is reported that MT is sometimes faster than the standard ANSI-C library in a system with pipeline and cache memory.) (Note added in 2004/3: on 1998, usually MT was much faster than rand(), but the algorithm for rand() has been substituted, and now there are no much difference in speed.)
# Efficient use of the memory. (The implemented C-code mt19937.c consumes only 624 words of working area.)

I know of at least one other game using it.

CPangracs
14 Sep 04, 09:19
Which game is using it?

kbluck
15 Sep 04, 15:39
The Mersenne Twister pseudo-random generator algorithm is well known, having been developed in the '90s by a couple of Japanese math professors. Like all such sequences, it has good points and bad points, although it is better than most. I'm sure it is used in a variety of software requiring Monte-Carlo random-number generation, like any number of games.

Pseudo-random sequences often display non-intuitive characteristics. How and when you use the results is often as important as what algorithm generates them. I think Pat tried to invent a home-brew solution that seemed to have good characteristics on his test machines, but the result suffered from harmonic problems on certain hardware with certain timing characteristics.

For a game like ATF that requires relatively small numbers of random results, rand() with a simple processor time seed is in theory quite adequate. It is, however, sometimes implemented in a questionable manner by compiler vendors. The Boost.org library offers a number of peer-reviewed implementations, including MT. I've encouraged Pat to look into it, but I think he has had other more pressing priorities.

--- Kevin

Pat Proctor
16 Sep 04, 11:17
KB et al.

I am currently using the "rand()" solution to random number generation. I am seeding in a manner which is slightly more random than simply using time (using the lo order word of the time, twice).

It seems to have solved all of the problems, though the problem which original spurred my exploration seems to persist: If you have one alternate battleplan, ATF seems to pick it more than half of the time at startup ?!?

jd

Thanks for the link. I will check it out.

kbluck
16 Sep 04, 12:14
I am seeding in a manner which is slightly more random than simply using time (using the lo order word of the time, twice).

It may double the number of bits, but it likely doesn't make them any more random. If you use the same word twice, that is statistically no different than using it once and zeroing the other word. If you are taking the low word at two different instants, you probably still have a hardware timing dependency. Replicating the same 16 bits multiple times still only results in 16 bits of entropy. Similarly, replicating 16 bits with a deterministic relationship between them still only adds 16 bits.

My suggestion for a simple seed generator with high entropy:

Start with an arbitrary set of 32 bits. (Google "hotbits" for a source of true random bits.) CRC32 it with the tick count when you start the game and use that as your seed. Save the seed bits back out, in the registry perhaps. Use the saved hash next time you start and CRC32 it with the tick count again, use that as the seed, and save the result. Ad infinitum.

Now, you are enjoying the randomness not only of the specific time the game starts, but the randomness of every previous time the game started as well.

--- Kevin

Pat Proctor
17 Sep 04, 22:43
I perhaps mispoke.

I cannot remember the number of bits the seeding takes, but rather than using the high and low order word from the time, I am using the low order word twice, in both the high order and low order spot.

One could make the argument that, if the distribution in the rand function is truly "pseudo-random", this should make no difference, but it does have the effect of more widely varying the "start point" from which the ATF Engine jumps into the list of random numbers.

kbluck
18 Sep 04, 15:59
I cannot remember the number of bits the seeding takes...

srand() takes an unsigned int, which in Win32 is 32 bits.


but rather than using the high and low order word from the time, I am using the low order word twice, in both the high order and low order spot. One could make the argument that, if the distribution in the rand function is truly "pseudo-random", this should make no difference, but it does have the effect of more widely varying the "start point" from which the ATF Engine jumps into the list of random numbers.


There is no reason to assume this has any useful effect without testing. In fact, it is highly likely that the opposite is true, that you are in fact inadvertently maximizing bias by this scheme. Let me explain.

Setting the high and low words equal allows no more than 2^16 possible values for your seed. First, this means your set of possible seed values is recycling every 65 seconds, thus vastly increasing the likelihood of reusing the same seed values. Even worse is the fact that you are restricting yourself to such a tiny subset of the possible seed values, about 1/750th of one percent of the possible values in fact, and the seeds are not themselves randomly distributed in sequence. It is thus entirely possible, in fact downright likely given your real-world experience with the results, that this particular subset of arbitrarily chosen values by sheer chance happens to produce a high proportion of pseudorandom sequences with an early skew to the same side, thus causing the battleplan selection bias of which you complain.

If you are unwilling to implement the saved-hash scheme I mentioned, you would still be better off simply using the full 32 bits returned from clock(); at least you'd have the possibility of a larger pool of seeds, depending on how long people leave their computers running. If you want more useful bits and don't mind the platform dependency (probably a non-issue since you already use DirectX) then use the low 32 bits returned by QueryPerformanceCounter(), which provides microsecond resolution rather than millisecond.

--- Kevin

Pat Proctor
18 Sep 04, 18:25
I know I am not describing what I am doing correctly. But it is difficult for me to explain.

The general idea is that, while I am only using 2^16 instead of 2^32 seed numbers, I am using the entire range of possible values, from 0 to 2^32 instead of a small subset of that range.

If you play ATF over a year, you will only cover seed numbers in a range of 2^25 numbers. Over two years, it would only be 2^26.

Since I only seed the random number generator once per time the game is run, the difference should be negligible in terms of perceptible difference to the player.

But, in practice, I have found through testing that the most noticible manifestation of the "non-randomness", battleplan selection, the new method I am using seems to yield more "random" results.

This has to be the most excruciating topic of conversation yet on the ATF/BCT forum for the non-programmer. I think it is just best if we agree to disagree. :)

jdeluise,

Thanks for the link. I will check it out.

davegruenbaum
10 Nov 04, 11:09
Then-1LT Sean Hazlett wrote an article for ARMOR Magazine for May-June 2002, Pages 16-21, "How to Defeat the Motorized Rifle Company at the National Training Center", and included, on page 19, OPFOR and BLUFOR MILES II Kill Probabilities for key vehicles. one should be able to access the Fort Knox website to find the ARMOR Magazine webpage.

kbluck
10 Nov 04, 13:07
Then-1LT Sean Hazlett wrote an article for ARMOR Magazine for May-June 2002, Pages 16-21, "How to Defeat the Motorized Rifle Company at the National Training Center", and included, on page 19, OPFOR and BLUFOR MILES II Kill Probabilities for key vehicles. one should be able to access the Fort Knox website to find the ARMOR Magazine webpage.


To be exact:

http://www.knox.army.mil/center/ocoa/ArmorMag/mj02/3MRC02.pdf


As usual, the full story is surprisingly complicated. ATF is said to be based on MILES, but near as I can tell only used the probabilities for "Catastrophic Kill", which seems to be defined as the entire crew being killed. You might note the surprisingly low odds for many weapons to get a K-Kill on tanks. That is because, contrary to Hollywood portrayals, most tanks knocked out of action are not K-Killed. Even of those that are, a majority can be put back into action with some repair work and a new crew.

MILES also makes allowances for Mobility Kill, Firepower Kill, and Commo Kill. These odds are generally higher, especially for tanks from side and rear. It is possible to accumulate more than one of these states simultaneously.

Since ATF does not track the fate of the crew, it seems to me that a vehicle which has received both an M-Kill and a F-Kill is the practical equivalent of a K-Kill in game terms. (The difference at NTC would be that the crew doesn't need to be "replaced" for an F+M-Kill.) I found that when you sum the probabilities of the K-Kill together with the probability of the vehicle suffering both an F and an M-Kill but not a K-Kill, the effective pKs are significantly higher.

To give an example, an M1A1 being shot from the rear by a 125mm is listed as having a K-Kill chance of 30% in MILES and 40% in ATF. I always thought that 40% was too low for a rear shot, and in fact it is. Here's why: in ATF, the system escapes unscathed if there is no catastrophic kill. In MILES, however, the vehicle also has a 75% chance of a FP kill and a 90% chance of a mobility kill from the same hit.

So, doing the math there is a 68% chance of both FP and mobility kill occurring. Add that on to the 30% catastrophic kill chance, eliminate the double-counting for catastrophic kill (since that state implies F+M-Kill as well) and you have an overall chance of 77% for a "game kill" where the vehicle can neither move nor fire. This is nearly twice the database1 pK value.

If you add on the chance of *either* an F or M-Kill, rather than both at once, then the odds of the tank suffering significant damage that will greatly impair its function is even higher, > 98%. But unfortunately the game doesn't track system damage.

Reviewing the database1 figures, I found that in general, the frontal shots for tanks and the all-around shots for light armor were fairly close to MILES. This is simply because the odds are already generally so low or high, as the case may be, that the additional F+M chances don't move the aggregate number much. However, I did find that tanks in general are greatly overrated against shots from the sides and rear, compared to MILES. I found in those cases the pK was consistently understated by 30-50%. These findings are reflected in my kbDataModern database, where you will find tanks are considerably more vulnerable to side and rear shots than in ATF's database1.

One final consideration: MILES almost certainly overstates the true capability of the OPFOR and understates the BLUFOR. After all, the whole point is to train against the worst-possible-case scenario, and furthermore they don't want to advertise too clearly the true capabilities of US equipment. So, there's a certain amount of guessing involved as to the *real* numbers.

Moral of this story: If you're playing my DB, don't let your tanks get shot in the ass.

--- Kevin