Best way to generate random numbers in lua script

[AKA_Clutter]

Hi,

 

As the to[pic said, I'm looking at the best way to generate random numbers within a lua script without using os.time() (which requires you to deactivate the os sanitation script).

 

I saw a recommendation in a 2016 post to just run math.random() a few times at the start of the script.  THis seems to work OK, but also seems to generate the same number on several different runs.

That for the suggestions in advance.

[Grimes]

Suppose it depends on the range of the number you want to generate. If it is a relatively small range then doing it a lot of times can be one work around. I seem to remember that math.random, at least in DCS, had a tendency to select the first and last possible value less often than any other value. Run it 1000 times and a pattern emerges type of thing. To fix this I made a function for mist that tends to normalize the results. Basically it inserts each possible random value into a table repeatedly to a limit. {1, 2, 3, 4, 1, 2, 3, 4, etc} Then picks a random table index and returns that as the choice.  

 

	function mist.random(firstNum, secondNum) -- no support for decimals
		local lowNum, highNum
		if not secondNum then
			highNum = firstNum
			lowNum = 1
		else
			lowNum = firstNum
			highNum = secondNum
		end
		local total = 1
		if math.abs(highNum - lowNum + 1) < 50 then -- if total values is less than 50
			total = math.modf(50/math.abs(highNum - lowNum + 1)) -- make x copies required to be above 50
		end
		local choices = {}
		for i = 1, total do -- iterate required number of times
			for x = lowNum, highNum do -- iterate between the range
				choices[#choices +1] = x -- add each entry to a table
			end
		end
		local rtnVal = math.random(#choices) -- will now do a math.random of at least 50 choices
		for i = 1, 10 do
			rtnVal = math.random(#choices) -- iterate a few times for giggles
		end
		return choices[rtnVal]
	end

 

[AKA_Clutter]

@Grimes _ Thanks very much for the response.  I did see in Hoggit (I think) that MIST has a function for this.

 

What I want to do is vary the spawn times with in a 10 - 20-minute window., so the numbers aren't huge.  I may want to do a range between 30 and 150 minutes.  Would that be getting too large with this function?.

Also, what is considered a "small range"?

 

TIA

[Grimes]

I think with any range the first and last value are always less likely, it depends on how many options there are and what they mean. If you had 5 groups and wanted to do math.random(5) to pick which one you wanted to spawn, then that would be a small range. The odds of getting options 1 and 5 would be lower than the other 3 and that is where the function shines at. 

With timing it can be less of an issue because you can use a smaller time value. math.random(10, 20) may end up with some pretty similar results, but math.random(600, 1200) would have some variation involved. That function sets a limit for how many table entries it will create to 50, so if you had more than 50 options it would effectively be the same as math.random(n). 

 

Worth noting I haven't tested the variation involved with math.random() and it spitting out decimal values. You might get decent results by doing that, multiply by 10, 100, or whatever, and using math.modf() to get the whole number as the random choice. 

[AKA_Clutter]

Might give that a try..  What I have done is[math.floor{math.random() *1000}/100] to get x.yy

Thanks again.  

 

[trevoC]

The reason you get non-random results is because you cannot use math.random() without seeding the random number generator first.

See below.

math.randomseed(os.time()) -- Seed the random number generator with the current time
local randomNumber = math.random(1, 100) -- Generate a random number between 1 and 100
print(randomNumber) -- Output the random number to the console

 

The math.random function in Lua generates what is known as a pseudo-random number, which means that the sequence of numbers it generates may appear to be random, but it is actually deterministic and can be reproduced if the same seed value is used.

By default, the Lua random number generator is initialized with a fixed seed value, which means that if you call math.random multiple times without changing the seed, you will get the same sequence of numbers every time.

To generate a more "random" sequence of numbers, you can seed the random number generator with a more unpredictable value, such as the current time. In the example I gave earlier, math.randomseed(os.time()) seeds the random number generator with the current time, which should help to produce a more unpredictable sequence of numbers.

However, it's important to note that even with a good seed value, the math.random function is still deterministic and the sequence of numbers it generates can be predicted if you know the seed value and the algorithm used to generate the numbers.

If you need truly random numbers for security or cryptographic purposes, you should use a different source of randomness, such as the operating system's built-in random number generator, or a hardware random number generator but this falls outside the scope of what I'm guessing is more basic needs in DCS for which the above is just fine.

[trevoC]

The math.random function in Lua does not generate weighted results by default. It generates uniformly distributed pseudo-random numbers, which means that each possible value has an equal probability of being selected.

For example, if you use math.random(1, 6) to simulate a six-sided die roll, each of the possible outcomes (1, 2, 3, 4, 5, 6) has an equal chance of being selected, assuming a good seed value.

The problems described above are probably a bad seed value.

[cfrag]

9 hours ago, trevoC said:

The reason you get non-random results is because you cannot use math.random() without seeding the random number generator first.

I believe that is exactly OP's point. The problem for DCS is the seed which would need something OTHER than os.time() (because os would need you to de-sanitize os, which is a no-go for many, me included). The question is: is there a good alternative to os.time() for randomseed()?

 

[AKA_Clutter]

1 hour ago, cfrag said:

I believe that is exactly OP's point. The problem for DCS is the seed which would need something OTHER than os.time() (because os would need you to de-sanitize os, which is a no-go for many, me included). The question is: is there a good alternative to os.time() for randomseed()?

 

Exactly!

[trevoC]

Sorry guys I didn't read far enough back (still haven't so forgive me if even this doesn't help)

You could use any of the following:

math.randomseed(os.clock())
math.randomseed(os.date("%S"))
math.randomseed(os.clock() + collectgarbage("count"))

you could dream up others.

os.clock returns the amoutnn of cpu time used by the program so that will always be different (in seconds i believe so if you call this more than once per second it would be no good)

os.date("%S")) is the same... seconds from date

bottom example just combins values if you need something that changes more often than the system time.

 

Does DCS not allow you to access OS.x? Is that the problem.
 

 

you could seed the number generator with a fixed value that is derived from one of the instruments of the aircraft for example.

math.randomseed(GPSCoordinates) -- seed the random number generator with a fixed value
local random_number = math.random(1, 100)
print(random_number)

What about that....

 

[trevoC]

What values are available to you on the map side that you can call and inject into math.randomseed() instead of an aircraft value if this is a map? I've never built a mission or campaign but I'm guessing there are plenty of variables available that count up or are constantly changing.

My example assumed an aircraft but I get the impression its a map or mission you are working on.

[AKA_Clutter]

It is a mission, and I load the script 2 seconds into the mission.

[trevoC]

As mentioned above, all you need to feed randomseed() is an integer of some type.

couldn't you just create a timer on load or use an existing one if you have one and feed the timer value to the randomseed

math.randomseed(timer)

I'm not familiar with the mission creator or what portion of the API is available, but I'm sure there is some constantly changing integer value you can feed randomseed()

with just a quick search I found timer.getTime()

math.randomseed(timer.getTime()) -- seed the random number generator with a fixed value
local random_number = math.random(1, 100)
print(random_number)

 

[AKA_Clutter]

I'm sure there is a way to do that.  I believe "timer.getTime() gets the mission start time and is static for a mission.  I know that os.time will change over the course of time.  Since I am running this script at 2 sec into the mission, and it sets everything then, I'm not sure that there is much difference for a specific  mission.  I would like there to be a noticeable difference between mission ran using this script.

For that purpose, what Grimes listed above is probably sufficient.  

 

Besides that, it is just a "science" discussion/project, trying to understand what's going on and if there is a better way to do it.

Edited February 22, 2023 by AKA_Clutter

[TJTAS]

what about randomly generating a big number and using that to seed the math.randomseed()

local seed = math.random(1,10000000)

math.randomseed(seed)

[TJTAS]

TBH my own experiments imply to me that somewhere in DCS it is randomizing the seed. I have written scripts to generate a sequence of random numbers. If I run it in a Lua framework without randomseed then i get the same numbers. When I have run it in DCS I do get random sequences of numbers.

[AKA_Clutter]

Just starting to run some test.  I call math.random() 20 times in the script I'm developing.  In three runs, i got .9 or greater 11 times.  11 out of 60 is 18%.  Yes, I only have a small sample set, but that seems a little high.  also bothersome is that it appeared 3 or so times in each run.

And I may give the local seed = math.random(1,10000000) suggestion a try.

More testing to do.

 

[trevoC]

4 minutes ago, AKA_Clutter said:

Just starting to run some test.  I call math.random() 20 times in the script I'm developing.  In three runs, i got .9 or greater 11 times.  11 out of 60 is 18%.  Yes, I only have a small sample set, but that seems a little high.  also bothersome is that it appeared 3 or so times in each run.

And I may give the local seed = math.random(1,10000000) suggestion a try.

More testing to do.

 

why don't you post that code snippet. Not sure you are using it correctly.

[cfrag]

[NOTE: if you are more interested in this, here's a great article about random generator quality in computers]

I've run a couple of tests with the goal to verify the veracity of couple of beliefs I held with regards to DCS's/Lua's random number generator math.random():

• Is Lua's pseudo-random generator math.random() sufficiently initialized at the start of the mission so that the first n (say n=20) random numbers generated are sufficiently unpredictable (i.e.: is the sequence generated run after run sufficiently sufficiently different) between runs, i.e. that every run of the mission starts with a fresh set of random numbers so every mission is different? My belief was that they are not (as discussed in this post) and require an invocation of randomseed() before becoming sufficiently randomized.
• Are the random numbers generated 'random enough', i.e. is the average of the values close enough to "50%" (0.5 on a scale of 0..1)? My expectation was that the average is close enough to 0.5
• Are the numbers sufficiently different between individual random calls, i.e. is there enough variance for numbers 1..100? My expectation was that they are
• As above, but for small numbers 1..10? My expectation was that they aren't - I believed that DCS tends to cluster small random numbers 

Methodology:

I created a new mission "Need for Seed" and added a small script that runs 2 seconds after mission start (with a trigger ONCE, condition Time>2) as follows:

local avg = 0
local numRnd = 30
for i=1, numRnd do 
	local r = math.random(1, 10)
	trigger.action.outText("Random #".. i .. " is <" .. r .. ">", 30)
	avg = avg + r
end

local avg = math.floor(avg * 10/ numRnd) / 10
trigger.action.outText("\nAverage for n = <".. numRnd .. " is <" .. avg .. ">", 30)

avg = 0
numRnd = 50000
for i=1, numRnd do 
	local r = math.random(1, 100)
	avg = avg + r
end

local avg = math.floor(avg * 10/ numRnd) / 10
trigger.action.outText("\nAverage for n = <".. numRnd .. "> is <" .. avg .. ">", 30)

So this script first polls 30 random numbers between 1 and 10, and prints them out, then calculates the average of those 30 generated numbers. It then polls 50'000 random numbers between 1 and 100 and calculates the average value of 50'000 samples.

I then ran the mission a couple of times, and took

Spoiler

screen shots. Although I ran them more then 20 times, I'll not bother you with the details. Here are my findings (or rather, my interpretations of the results):

• Each run starts with with a sufficiently different sequence. I was pleasantly surprised, as it felt differently when I'm debugging missions. We therefore don't require a math.randomseed(os.time()) call at the start of the mission to randomize the sequence. DCS does that for us.
• We see that the average of 50'000 calls for 1..100 consistently is 50.9. This means that the random numbers are slightly skewered up, as the average value should have been 50.5
• For numbers between 1..100, they seem varied enough, although my pattern searching brain does seem to home in on what looks like clusters to me. The problem here is that my brain is not to be trusted, humans are pre-conditioned to seek out patterns and tend to see them where there really are none.
• For numbers between 1..10 I really can't tell. I do see clusters. But they are sufficiently and unpredictably different between runs for me to accept them as random enough

Conclusion

This was much ado about nothing - in other words: a perfect evening  - to me, it seems that Lua's math.random is sufficiently initialized in DCS to not require a call to randomseed, and that the numbers are sufficiently randomized for my non-scientific, gaming-only needs.

Below, please find the miz for your own test, I'm interested to hear your own findings and interpretations, especially if they differ from mine  

 

Cheers,

-ch

 

Need for Seed.miz

Edited February 23, 2023 by cfrag

[PravusJSB]

My 2 pence... 

 

 

[trevoC]

10 hours ago, cfrag said:

[NOTE: if you are more interested in this, here's a great article about random generator quality in computers]

I've run a couple of tests with the goal to verify the veracity of couple of beliefs I held with regards to DCS's/Lua's random number generator math.random():

• Is Lua's pseudo-random generator math.random() sufficiently initialized at the start of the mission so that the first n (say n=20) random numbers generated are sufficiently unpredictable (i.e.: is the sequence generated run after run sufficiently sufficiently different) between runs, i.e. that every run of the mission starts with a fresh set of random numbers so every mission is different? My belief was that they are not (as discussed in this post) and require an invocation of randomseed() before becoming sufficiently randomized.
• Are the random numbers generated 'random enough', i.e. is the average of the values close enough to "50%" (0.5 on a scale of 0..1)? My expectation was that the average is close enough to 0.5
• Are the numbers sufficiently different between individual random calls, i.e. is there enough variance for numbers 1..100? My expectation was that they are
• As above, but for small numbers 1..10? My expectation was that they aren't - I believed that DCS tends to cluster small random numbers 

Methodology:

I created a new mission "Need for Seed" and added a small script that runs 2 seconds after mission start (with a trigger ONCE, condition Time>2) as follows:

local avg = 0
local numRnd = 30
for i=1, numRnd do 
	local r = math.random(1, 10)
	trigger.action.outText("Random #".. i .. " is <" .. r .. ">", 30)
	avg = avg + r
end

local avg = math.floor(avg * 10/ numRnd) / 10
trigger.action.outText("\nAverage for n = <".. numRnd .. " is <" .. avg .. ">", 30)

avg = 0
numRnd = 50000
for i=1, numRnd do 
	local r = math.random(1, 100)
	avg = avg + r
end

local avg = math.floor(avg * 10/ numRnd) / 10
trigger.action.outText("\nAverage for n = <".. numRnd .. "> is <" .. avg .. ">", 30)

So this script first polls 30 random numbers between 1 and 10, and prints them out, then calculates the average of those 30 generated numbers. It then polls 50'000 random numbers between 1 and 100 and calculates the average value of 50'000 samples.

I then ran the mission a couple of times, and took

  Hide contents

screen shots. Although I ran them more then 20 times, I'll not bother you with the details. Here are my findings (or rather, my interpretations of the results):

• Each run starts with with a sufficiently different sequence. I was pleasantly surprised, as it felt differently when I'm debugging missions. We therefore don't require a math.randomseed(os.time()) call at the start of the mission to randomize the sequence. DCS does that for us.
• We see that the average of 50'000 calls for 1..100 consistently is 50.9. This means that the random numbers are slightly skewered up, as the average value should have been 50.5
• For numbers between 1..100, they seem varied enough, although my pattern searching brain does seem to home in on what looks like clusters to me. The problem here is that my brain is not to be trusted, humans are pre-conditioned to seek out patterns and tend to see them where there really are none.
• For numbers between 1..10 I really can't tell. I do see clusters. But they are sufficiently and unpredictably different between runs for me to accept them as random enough

Conclusion

This was much ado about nothing - in other words: a perfect evening  - to me, it seems that Lua's math.random is sufficiently initialized in DCS to not require a call to randomseed, and that the numbers are sufficiently randomized for my non-scientific, gaming-only needs.

Below, please find the miz for your own test, I'm interested to hear your own findings and interpretations, especially if they differ from mine  

 

Cheers,

-ch

 

Need for Seed.miz 9.58 kB · 0 downloads

 

"This was much ado about nothing"

Correct.

Nice to be able to do the work to convince yourself sometimes but the solution is simple and posted above.

the only consideration is feeding a different seed so that you can ensure different results mission to mission if you are concerned that DCS doesn't initialize the seed differently each time (which it may or may not do).

math.random doesn't produce weighted results as discussed above.

[xcandiottix]

If you're really convinced this is a mission critical component then would recommend the following instead of trying to add a 100 lines of code to your mission:

local timestamp = timer.getTime() --returns x.xxx seconds since the mission started regardless of pausing
local str_timestamp = tostring(timestamp) --convert the timestamp to a string
local last_digit = string.sub(str_timestamp, -1) --extract the last character from the string

env.info(last_digit) -- Output: If timestamp is 1234.567 this will return "7"

You could run this every ten minutes and if timestamp is equal to 0 then you have a 10% chance of whatever happening.
If you change it to any number <5 you have a 50% chance of it happening.
<9 is a 90% chance.

You're just trying to randomly make something happen in a video game .. this isn't a banking app so no need to try too hard. And honestly, you could just do math.random(1,10) and then write logic like above (if 1 do x if <5 do y or if <9 do z) and it will be fine.

EDIT: Also, I'm sure someone is going to say if you run it every ten minutes the number will be .000 ... it won't be because scripts ran from the mission editor flow through DCS's coroutine manager and will always vary slightly. Run it for yourself and see. As long as there is actual stuff going on in your mission it will cause sufficient random stuttering in when this actually gets to fire. Appending this to export.do~every~frame would probably not work but I don't think you're trying to be that advanced with this.

Edited February 24, 2023 by xcandiottix

[trevoC]

21 minutes ago, xcandiottix said:

If you're really convinced this is a mission critical component then would recommend the following instead of trying to add a 100 lines of code to your mission:

local timestamp = timer.getTime() --returns x.xxx seconds since the mission started regardless of pausing
local str_timestamp = tostring(timestamp) --convert the timestamp to a string
local last_digit = string.sub(str_timestamp, -1) --extract the last character from the string

env.info(last_digit) -- Output: If timestamp is 1234.567 this will return "7"

You could run this every ten minutes and if timestamp is equal to 0 then you have a 10% chance of whatever happening.
If you change it to any number <5 you have a 50% chance of it happening.
<9 is a 90% chance.

You're just trying to randomly make something happen in a video game .. this isn't a banking app so no need to try too hard. And honestly, you could just do math.random(1,10) and then write logic like above (if 1 do x if <5 do y or if <9 do z) and it will be fine.

EDIT: Also, I'm sure someone is going to say if you run it every ten minutes the number will be .000 ... it won't be because scripts ran from the mission editor flow through DCS's coroutine manager and will always vary slightly. Run it for yourself and see. As long as there is actual stuff going on in your mission it will cause sufficient random stuttering in when this actually gets to fire. Appending this to export.do~every~frame would probably not work but I don't think you're trying to be that advanced with this.

 

you can lead a horse to water......

[cfrag]

This has been an interesting thread so far. I believe that the origin of this thread is this other thread by the same OP, which explains the reference to randomseed.

So I believe OP's question, even if phrased somewhat awkwardly, is the best way to initialize the random number generator with a number that is not predictable. Traditionally, people use os.time() for this, but "os" is walled off in DCS Lua for security reasons and should remain so. After successfully initializing with an unpredictable value, so I infer,  Lua's math.random() is good enough for all random needs in the mission.

The problem to solve was therefore: "how to come up with an unpredictable seed value for randomseed to start math.random's sequence of pseudo-random numbers".

That question has so far not been answered, as it is impossible inside DCS/Lua context to reliably create an unpredictable number that can be used as a random seed. Multiple iterations of random, or using random itself to seed random will not work if random isn't seeded first. Providing alternative code for random/randomseed (as was kindly submitted by @PravusJSB) merely replaces this problem with an alternate version of the same challenge - that code also requires an unpredictable seed, which returns us to square one. 

I ran some tests to see if the initial assumption that for a mission Lua's randomseed() must be invoked prior to invoking random(), for I held that same belief. And - at least to me - it seems (contrary to my belief) that a mission's Lua random generator is properly seeded at mission start by DCS, so this discussion can be resolved: randomseed is invoked outside of our context, and the question of how to properly seed math.random has been rendered moot - it was pushed outside of the mission's context; @AKA_Cluttercan proceed without having to seed random(), and can ignore the recommendation from 2016 to invoke randomseed before using math.random, as it this is now being done outside of the mission by DCS's game engine. 

Edited February 24, 2023 by cfrag

[trevoC]

This is all answered above.

You don't need an "unpredictable seed value" to seed the random number generator. "1234" works fine. Anything actually works.