Detection Script

[MBot]

I am currently working on a script that intends to improve the visual detection model of AI ground units. The idea is that you apply a script to a ground unit that will run on top of the AI's hard coded detection model and controls the engagement of script-detected targets via controlling the unit's ROE. Obviously once the script has detected a target and allows to open fire, the AI will engage any target it sees via the hard coded detection model, irrespective if this target has been detected within the script. It's not pretty, but it's the best thing we can currently do with scripts.

 

Factors affecting probability of aircraft detection I intend to implement include:

-line of sight

-distance

-altitude

-angular velocity

-terrain/sky background

-relative bearing from observer (front, side, back)

-time of day

-abstraction of ground clutter masking (since DCS has no true vegetation)

 

Writing the overall script has been fairly easy and quick. All the factors above are already captured by the script. Now I am working on how exactly these will affect the probability and this becomes quite challenging. I was never really good in probability mathematics and am hitting a brick wall here.

 

The way the script works is that each second, the area (radius 10 km) around every designated observer unit is checked for aircraft and helicopter. The above factors determine a probability of detection if the target is truly detected within each search cycle (which repeats every second).

 

Now the base data for the probability of detection I am collecting from various research papers comes in a format like this:

 

 

The aircraft approaches the observer (at a speed specified in the research method, here 400 kts) and the probability that the target is detected at a given range increases as the aircraft is closing. Now how do I transfer this data to a form that is usable by my script? My probabilities are applied every second again, while here a single probability somehow rises with closing distance. The probabilities for my script need somehow need to be a lot smaller, since they are repeated quickly (A 33% probability of detection will on average lead to a certain detection within 3 seconds). I know I can somehow use this real world data, but I am not sure how to apply it. Does anyone have a better understanding of this?

 

 

In case you are interested, here are is some of the research on visual detection I found:

http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=AD0645537

http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=AD0650403

http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=AD0758875

http://www.rand.org/content/dam/rand/pubs/research_memoranda/2006/RM4562.pdf

Edited November 13, 2013 by MBot

[chromium]

OMG I can't wait to test it!!

[VapoR]

This will be very cool if you can pull this off! It sucks that we have to fly against uber-AI or exclude them from mission design in the sake of playability.

[egaRim]

imo you need to reduce the interval when your script checks the probability. The way I see it, your sample chart is only valid for exactly one specific situation. It cannot be applied continously for your mentioned reason. One way out would be to trigger on certain thresholds (for example, every time the target closes a full 1000 yards probability of detection is calculated). Another possibility would be to restrict the interval by simulating the oberserver turning his head at a certain turn rate (say 5 degrees per second). This would result in an "All-Seeing-Eye radar like check" every 360/5=72 seconds, or, if you can pull it off, a direction specific check in a 5° search cone every second.

[MBot]

I do want to keep the interval reasonably high (without drawing too much processing power) to cover pop-up situations. 1 time per second seems to be a good compromise. I don't think is is a problem, if the base probability is in the right magnitude. The question is though, if I can somehow derive this from available real-world data. I think I can come up with numbers that work myself, but I would rather first try if I can directly derive this from available reserach.

 

I would rather like to handly directions with probabilities aswell, instead of making a "physical sweep the search area" type model. I don't think this is a problem either, if the base probability stands for a known search cone size. The probability for any sector size can then be calculated. The graphic above is a cummulation of 45°, 180° and 360° tests. I am still trying to get the orgininal paper with the individual data sets, but I couldn't find it in electronic format yet(Detection of random low-altitude jet aircraft by ground observers, by William Wokoun).

[Joyride]

This is an awesome idea MBot...will be following with much interest.

[Fishbreath]

I think the problem is that the data you have is hard to work with. The probabilities you have would work just about right for a closing target checking once a second, but they don't translate very well to e.g. a crossing target. The graph is probability of detecting a closing target before a given range; you would want something more like probability per unit time vs. range graph.

 

You could maybe cheat it a bit. 400 knots is about 250 yards per second, so the gap between each vertical line on the graph is about four seconds.

 

So, for instance, from 5,000 to 6,000 yards, the midpoint of the curve in there looks to be about 0.08; we'll call that pDetect. There's a corresponding probability that the plane won't be detected: 1 - pDetect = pUndetect = 0.92*.

 

If you have a bunch of independent events, the probability that they'll all come out in a given way is their product. Our pUndetect happens in a four-second window from 5,000 to 6,000 yards, so it's logically the product of the probabilities that the airplane won't be spotted in each of those four seconds:

 

pUndetect = (1 - pDetectPerSecond)^4

 

or, filling in a little and working it out...

 

0.92 = (1 - pDetectPerSecond)^4

4throot(0.92) = 1 - pDetectPerSecond

about 0.98 = 1 - pDetectPerSecond

pDetectPerSecond ~= 0.02

 

So, at 5,000 to 6,000 yards, there's about a 50% chance that an aircraft will be detected after 25 seconds, and it's almost certain that it'll be detected after 50 seconds. That passes the smell test for me.

 

For this specific case (using the probability to detect a 400kt closing target during a certain 1,000 yard window as the probability that such a target will be detected in four seconds if flying head-on in that range window), you can just use this formula to convert to probability per second:

 

pDetectPerSecond = 1 - 4throot(pDetectOver1000Yards)

 

* It's possible to run the math that comes after this the other way around, I think, but it makes more intuitive sense to think about it as the probability of the event failing to happen in this case, and since I'm not a mathematician I can't tell you why**.

** Okay, so I asked a mathematician, and we came up with an answer that has to do with distributions and independence of probability. This is a fun case where the way you ask the question has a significant outcome on how easy it is to find the answer, so that's cool.

 

Pretend that you're rolling a six-sided die, and if you see a six, you detect a target and stop rolling. Now, on any given roll, you have a one-in-six chance to roll a six, but since we don't keep rolling after we detect a target, the probability that we'll roll a six on, say, the third roll depends on the results of the previous two rolls: if we roll a six the first time, there won't be a third roll at all. That's the phrasing that asks about pDetect, with the wrinkle that we have to sum the probabilities for the first, second, and third times. Asking if we haven't rolled a six after the third roll contains the implicit assumption that we get three rolls in, because that's the case we're asking about. The answer to that question just happens to contain the answer to the other phrasing, too.

 

It's actually a little more complicated, because summing the probabilities per second doesn't quite work out (if you assume that for second n, pDetectInThatSecond = pUndetectPerSecond^n * pDetectPerSecond), but I'm given to understand that's because working it out second-by-second is a piecewise approximation of a curve (the one described by a probability density curve).

Edited November 21, 2013 by Fishbreath

[Bushmanni]

I skimmed though the papers and they had multiple different scenarios where the detection probability was estimated from experimental data. I think an analytical solution would be hard to obtain that would consider all the data available. I suggest you design a mathematical model (you have probably done this already) for the detection and then extract lots of datapoints from the experimental data and fit the model to the data. ie. use some kind of algorithm that adjust the model parameters until the error between data and model is minimized.

[MBot]

...

 

 

Thank you for this excellent post. I spend many hours digesting this and thinking intensively about it. While I see where you are going at, I cannot figure out how to apply it in a generalist model. I think I am simply not smart enough for probability mathematics (old memories from school are popping up) :)

 

I decided I will try to go with a less ambitious route and write a simple model that makes common sense in the first place, while not necessary being based on 1-to-1 real world data.

 

I figured that the value we are most interested in is what I call the Mean Time for Detection mtd in seconds at a given range. From mtd, the probability of detection per second p is derived as p = 1/ mtd.

 

As a base value for mtd under optimal conditions for a 360° search area I went with:

 

mtd = 2^(distance in m / 1000) * 4

 

So this gives us:

 

range / mtd

1000 m / 8 s

2000 m / 16 s

3000 m / 32 s

4000 m / 64 s

5000 m / 128 s

6000 m / 256 s

7000 m / 512 s

8000 m / 1024 s

9000 m / 2048 s

10000 m / 4096 s

 

Keep in mind that these are for a 360° search. so times are rather high. The values are adjusted if a group is set to observe a specific direction or if the group has multiple units to search simultaneously.

 

I think this looks already quite promising. I am in the process of debugging the code and will soon prepare a first demo mission. Hopefully we can then tune the values further (especially also how factors such as angular speed, elevation angle etc. affect probability of detection).

[esb77]

I think you may be underestimating how hard this is going to be to do well.

 

My advice would be to start by trying to set this up as a set of independent modules.

 

Meaning that for factors like: line of sight, haze, cloud cover, aspect, ect. get a working script for each of those before you try to build one for everything lumped together. Firstly it'll get you some of the background you need for the final unified script, and secondly if it's well written it should make troubleshooting the final product a lot easier because it'll be broken down into manageable chunks.

 

A far as calculating probabilities goes, a general model is going to be a nightmare (or a least a real pain) because you're not going to have nice easy to work with distributions, and you're probably going to need different distributions for different scenarios if you don't want you math model to run to many pages before you even start trying code it.

 

My overall advice is this: Choose the scenario where you thing the default model does the worst job, and look up real world data for that situation. Then choose a distribution that at least roughly approximates it. Some sort of exponential decay function for P (not spotted) might be a decent starting place (keeping in mind that it can often be much easier to find the probability of something not happening and then subtracting from p=1 to get an answer). Then use your adjusting factors to come up with inputs for the variables and/or coefficients that go into the probability distribution function to get the curve closer to the real world data. If that doesn't get it close enough, you could come up with equations that further adjust it.

 

What makes a probabilistic model so difficult here is that you have multiple independent variables that influence the outcome. Coming up with a really good model is the sort of thing that could make up a thesis for a graduate degree or Ph.D. in applied mathematics (or statistics). I wouldn't be surprised if there is a thesis sitting in a library somewhere that does pretty much contain the solution, finding it could be a bit of a chore though.

 

Multiple piecewise partial solutions aren't as elegant as a good global model, and at a completed state might take up more pages of script. The thing is, you'll have accomplished something that's an improvement a lot sooner than if you try to do a good global model.

[esb77]

O.k. stupid me, I forgot the most basic starting point of engineering and applied mathematics.

 

Come up with a well defined problem statement.

 

In general terms your problem seems to be, "A.I. visual detection of aircraft is too good in some situations."

 

First step is to clearly define what situations, and how much better than it ought to be.

 

Then you can start asking intelligent questions about how it can be fixed.

 

Depending on how serious you are about this one starting step might be to contact Eagle Dynamics and ask what math model they're using for visual detection. That would give you a good starting point for researching what improvements or better models may already exist.

[Bushmanni]

Lets not make this too complicated. Building a model that almost exactly fits the data would be very hard but in this case an error of 5-10% in spotting probability isn't going to be a big problem. I think it's possible to get a pretty good approximation of the real data by hand tuning the parameters until the model gives good results. This problem doesn't need any custom probability distributions but only a function that gives the spotting probability as as function of some parameters (range, background, ...) and then you pick a random number with uniform distribution between 0 and 1 and see if it's larger than the spotting probability.

 

There's no need to make any adjustments for multiple targets as if there are more of them it's natural that you are more likely to spot one of them. This would not be the case if scanning would be simulated in some way but that's not really necessary at least in the initial version.

[MBot]

Just to make it clear, the script is already written. The coding part is actually fairly light, it is only about 150 lines of code. The troubles I had was to determine the p values that is going to be applied, either by deriving it from available experimental data or coing up with a "common sense" model. For now I went with the latter with inspiratin by the former.

 

Hopefully I will be able to prepare a first demo mission later this evening.

[MBot]

Well, here is a first demo mission for you to try out the script. It features a Stinger to the north, a Vulcan to the east, a Chaparral to the west and a group of 3 HMMWVs to the south. Note that the Stinger and the HMMWVs are set to search 360° equally, while the Avenger and the Chaparral are set to concentrate on their forward quarter. Feedback is highly appreciated.

 

The demo mission has debug output enabled which displays all the factors being taken into account. mtd stands for mean time for detection in seconds, which is a variant of probability but easier to work with.

 

 

And here is how it works:

 

The script is applied to groups of ground units you want to use the advanced detection model. Each second, all aircraft and helicopter within 10'000 m and in line of sight of such a group are captured by the script. A variable probability of detection per seconds p (see below how it is determined) is checked against a random number between 0 and 100 each second. If p is bigger than the random number, detection of the aircraft is achieved and the ground group ("observer") is set to open fire. Once an observer detects a target, the detection checks are stopped and the observer remains aware until no target is within 10'000 m and line of sight for at least 20 seconds. Then the observer returns to his search routine and the AI is set back to hold fire again. This emulates the fact that once a target is detected, it will be known and tracked, even over short periods of contact loss, until completely out of physical observation parameters. Please note that the script can only control the detection of the first target. Once the AI is set to open fire, all targets within range can be attacked. There is no way to control who the AI fires on.

 

 

The initial value of the probability of detection per second is dependent on distance to target:

 

p = 1 / 2^(distance in m / 1000) * 4

 

 

If the observer is set to concentrate to his front, he will double his attention to his frontal quarter (p = p*2) at the expense of attention spend on his rear quarter (p = p/2). p for the side quarters remains the same. This corresponds to 45% of the total attention spend on the front, 22% to each side and only 11% to the rear quarter.

 

If set to search all around, the attention is equally distributed around 360° (initial p remains the same).

 

 

Next p is modified according to the target elevation angle in relation to the observers horizon. According to US Army Field manuals, visual target search is to be limited to 20° elevation. Above this elevation, probability to detect targets decreases. In my script therefore if elevation > 20° then:

 

p = p / elevationAngle * 20

 

There is no further justification for exactly this formula, I just took something that generally reduced p above 20°

 

If elevation angle is smaller than 0.5°, then an artificial, abstracted ground clutter terrain masking is applied (p = p/10). The justification is that DCS does not model vegetation to block the line of sight of AI. Therefore a abstracted obstacle of 4m height is assumed at a distance of 500m which reduces chance of detection by 10.

 

 

Next it is checked if the target is seen against a terrain or sky background. Due to decreased target contrast when seen against a terrain backdrop, probability is reduced by p = p/2

 

 

Next comes a check of the angular speed of the target (change of bearing as seen from observer in degrees per second). Targets that appear to be moving are more easily seen. P is modified as

 

p = p + p * angle / 14

 

where angle is the angular speed of °/s. There is no further justification for exactly this formula other than the assumption that a 450kt target flying perpendicular to the observer at 1000 m doubles the chance of being detection (which I pulled straight out of thin air).

 

 

Lastly p is multiplied with the number of units within the observer group. The assumption that twice the pair of eyes searching the air double the chance to find a target.

 

 

I decided to disregard time of day, since that already seems to be covered by the hard coded detection model. So even if the script detects a target at night and allows to fire, the AI will not fire if time of day (or weather) limitations apply.

 

 

 

As you can see, I tried to apply generally known principles but often had to pull the specifics out of thin air (even though a little guidance was found in US Army Field Manuals and various research papers). The above calculations are open for discussion and I hope with a broader input, we can come up with a satisfying result. Currently I tend to think that probability of detection is too low, but I want to make more tests to explore it. One thing that I think is important to remember is that we as observers of the test scenario are biased toward detection. We know that there is a target, where it is and therefore wonder why it is not detected. So while it might be obvious to us, we should remember that if place ourselves in the shoes of the ground unit, scanning 360° of sky, it might not be that obvious after all.

Edited November 17, 2013 by MBot

[MBot]

Work on the script continued. I tuned the formulas so that now the script reproduces the results of the HEL study quite precisely:

 

 

 

It is important to note though that the above graphic itself is a combination of all 360°, 180°, 90° and 45° search sector tests results. I tried to make the script configurable to different search sectors, but that turned out to be extremely difficult. Not only am I unable to find the individual graphs for the above cumulation, but target range plays a very inter winded role in this as well. The more visible the target, the higher the probability you will still see it in your peripheral vision. So even if you have a well defined search sector (lets say 30°), you will still see targets outside this sector if they are close enough. Therefore I decided to not overcomplicated things and tune my script in accordance to the above cumulation of various search sectors.

 

Then I through about how multiple observers will affect p. Observes could either dived the sky into individual search sectors or could all search the same sky simultaneously. One publication mentioned that it doesn't actually make a difference. Well I made test and programmed both variants. And it is true, the probability of detection is almost the same. So I decided to stick with the simpler p = p * n-observer.

 

I did find some data on how sky/terrain background affects the probability of detection. Again this seems to be a fairly complex issue depending on range and rate of closure. But the data indicates that a p = p / 2 reduction for terrain background is at least in the right ballpark, so I figured this is good enough.

 

The effects on high elevation and angular speed could still use some review, but I have the feeling that they too are generally in the right magnitudes.

 

 

I added another new bit to the script. Now target detection can also affect dispersion of ground units. If set to do so, a convoy will get off the street and stop as soon as it has visually detected an aircraft (as compared to as soon as it is being hit in current DCS). I think this is pretty cool and a big improvement for the air-ground environment. And if combined with my Dismounts Script you can set up convoys that stop and deploy MANPADS if they spot hostile aircraft.

 

 

Overall I am really happy how the script turns out. Attached is a new demo mission to check it out against a single Stinger.

DetectionScriptDemo.miz

Edited November 17, 2013 by MBot

[HiJack]

Awesome MBot! Must test this as soon as possibel. Big thanks for your work.

[gunterlund21]

can this be used with your arty script and getting rid of the 10km detection ring with ARty?

[GGTharos]

Is this for simulating one guy sitting around alone spinning his head, or an ADA troops with a well defined (usually 30 deg) search sector, with support from other guys searching their own sectors with 15-min relief intervals and possibly a data-link feed?

[MBot]

The script simulates a group of people (depending on the number of units within a group) actively searching 360° of sky for aircraft. They are aware that hostile aircraft are to be expected (air defense warning red or yellow) but are unaware of the exact time or bearing of an approach (no external target cueing). As such the script replicates real world test results.

 

Multiple units in a group increase the chance of early detection. There is no division into individual search sectors, research data and math show that it has no influence on the probability of detection. The script allows groups to either evenly distribute their attention in all directions or to concentrate their attention to the group`s front at the expense of their rear (good for ADA units to cover specific avenues of approach; or to simulate the forward focus of moving units).

[Dudikoff]

Will have to try this and the dismount script out, thanks for sharing it.. Would it be possible to add preset alertness level that could be added as a parameter to the function which would influence the probability of detection?

Edited November 18, 2013 by Dudikoff

[MBot]

can this be used with your arty script and getting rid of the 10km detection ring with ARty?

 

No, this script only works for the detetcion of aircraft. There is a large bag of issues related to try a similar approach for the ground combat environment. The arty problem could not be circumnavigated anyway.

[MBot]

Would it be possible to add preset alertness level that could be added as a parameter to the function which would influence the probability of detection?

 

I suppose it is possible, it is a good idea actually. Currently the script works on a alertnes level where contact is imediately expected. But a varable could be implemented to set a more lavish search procedure. I will have to think this through thoroughly.

[ENO]

I wondered about that with the suppression script to- whether or not it could be built against the skill level of the AI.

[MBot]

First let me ask a question that hopefully saves me some time on research. Does anyone know how many radar sweeps a search radar needs (for example Dog Ear) to identify returns as genuine targets and build a track? I assume it is more then one, but are two already sufficient?

 

 

So this evening I spend some more time on programming and implemented Early Warning Radars into the script. Observers can be assigned to EWR units to receive cues for targets (one EWR can have multiple recipients but an observer can only access one EWR). EWR detect targets (for now I went with 3 radar sweeps to build a track and recognize a contact as a target, so 6 seconds required for that) and pass down the info to visual observers. The EWR can be set to be datalink capable, in which case the passing down is instant but interpretation of the data by the receiver takes another 4 seconds. If the EWR is set to have no datalink, manual interpretation of the radar screen and passing down the info over voice radio requires another 10 seconds. So time from first radar hit to visual observer reaction is 10 seconds with dlink or 20 seconds without.

 

Observers will have their probability to detect targets visually that have been passed down by EWR modified to match real world data for a 30° scan sector. All targets that were not passed down by EWR (valid radar track required within the last 30 seconds and pass down of data completed) are still searched with the probability for a 360° scan sector.

 

This addition allows for air defenses to be configured to visually search targets either autonomously or with cuing by EWR. Of course the destruction of the respective EWR unit will also remove any target cuing for observers.

 

 

I wont have time in the next days to work on the script. But the next items to work on are the addition of a low alertness state and effects for skill.

[gagipro]

Same problem on Arma3, guys are trying to make exact same scripts because the standard AI inside the game are too much UberAI...

 

Incredible that we encounter this problem on both game. Perhaps you can have a look to some mods about LOS on arma2/arma3 ...