Jump to content

LUA Multi Threading and DCS


xcom

Recommended Posts

Hey all,

 

We have gotten to a point where the LUA scripts are quite heavy on DCS.

There is processing going on that's quite heavy and in heavy multiplayer missions it is also presented with lag/warping etc..

 

Is there a way anyone can think of that we might be able to run some of the LUA scripts on other threads?

I believe that DCS runs on one single core if I remember correctly, reducing processing power from that core would allow for better connectivity in the multiplayer missions.

 

Thanks in advance.

Link to comment
Share on other sites

I am pretty sure this is not possible without modifications (i.e. completely self-contained in the mission file).

 

If you are willing to install additional things (other than the .miz file) on the server, you have the following options right now:

 

  • Find or write a multithreading library for Lua 5.1 (there seems to be one called "Lua Lanes"), place it somewhere in package.cpath and "require" it. If you manage to make this work, don't use any DCS Lua functions from another thread, it would very likely mess something up because the rest of the DCS code does not expect it.
     
  • Connect to an external application over a TCP connection and pass long-running computation tasks to it. This application can be written in whatever language you like, including Lua. Bonus points: it can run on another computer, too. It could also run inside another DCS process, so you could, for example, use the "land" and "world" objects to make line-of-sight calculations.

 

I doubt that we will ever see multi-threaded Lua support as a core DCS feature, because it would be hard to implement. What might be reasonable to implement would be something like web workers in JavaScript, where you pass a task off to another thread that will only have access to a limited set of API functions which don't depend on current simulation state (such as the terrain stuff), but before that I'd like to see a lot of other features.

 

And before you think about multithreading or multiprocessing, think about optimizing your existing implementation. Can you achieve the same effect with less computation? Do all of those triggers really need to be checked once every second? etc.


Edited by [FSF]Ian
Link to comment
Share on other sites

Seems over what I have time to get into at the moment.

Especially because programming is not my occupation and getting into this would mean a lot of time.

 

Ian;2451902']And before you think about multithreading or multiprocessing' date=' think about optimizing your existing implementation. Can you achieve the same effect with less computation? Do all of those triggers really need to be checked once every second? etc.[/quote']

 

This is after optimizing and because we wanted to think forward as we saw that hard calculations actually create FPS drops on the host machines, in long running missions that would mean de-syncs probably even if the hit is small.

 

Even if the calculations happen once every 15 seconds, when doing large scale calculations it takes a hit on DCS.

I worked around it by creating very specific tables instead of going through a big table, it did get things working better but I'm sure that in the long run, the mission sync would be hit.

Link to comment
Share on other sites

Seems over what I have time to get into at the moment.

Especially because programming is not my occupation and getting into this would mean a lot of time.

 

This is after optimizing and because we wanted to think forward as we saw that hard calculations actually create FPS drops on the host machines, in long running missions that would mean de-syncs probably even if the hit is small.

 

Even if the calculations happen once every 15 seconds, when doing large scale calculations it takes a hit on DCS.

I worked around it by creating very specific tables instead of going through a big table, it did get things working better but I'm sure that in the long run, the mission sync would be hit.

 

Large parts of DCS is built on the assumption of single threading in places. So trying to multi-thread that might cause pretty large problems.

Replacing scripts with actual compiled code would give much better efficiency anyway. Lua does offer some way to do that already I think..

 

By the way, only reliable way to is to force synchronization at some points or periodically.

There are large amounts of reasons why synchronization won't work by expecting each computer to process at same time (they don't).

Generally speaking, PCs are not real-time systems but best-effort times: some additional load or delay and you are entirely out-of-sync after.

 

In networked programs sending datapackets with current information is normal way.

If that is not possible make triggered conditions that force specific status in the mission.


Edited by kazereal

"I would have written a shorter post, but I did not have the time."

Link to comment
Share on other sites

In networked programs sending datapackets with current information is normal way.

If that is not possible make triggered conditions that force specific status in the mission.

 

Can you give an example? or explain how do you mean?

Link to comment
Share on other sites

Even if the calculations happen once every 15 seconds, when doing large scale calculations it takes a hit on DCS.

 

If you have a big job that you want to execute infrequently, you should read up on coroutines. They won't help if the combined work of the simulation and your scripts is too much for one thread, but if your scripts do very little most of the time but then block for a whole second every 15 seconds, you can split the work up into smaller chunks and spread it out over the whole 15 seconds.

 

It's basically what I assume you are already doing now by processing smaller tables instead of one big one, but it's easier to use so you can use it in more places.

 

Here's a quick example which you can try in the DCS Witchcraft Lua console.

 

First execute this snippet:

a = 0
bigjob = coroutine.create(function()
for i=1,10 do
           a = a+1
           coroutine.yield()
   end
end)

 

Let's assume that counting 'a' up from 1 to 10 takes a really long time. We use coroutine.create() to wrap the task of counting up 'a' into a coroutine, which gets stored in the 'bigjob' variable.

 

Now execute this second snippets a few times and watch what happens:

return {coroutine.resume(bigjob), coroutine.status(bigjob), a}

 

Just by inserting coroutine.yield() statements, you can interrupt your big task to give DCS time to take care of other things. The coroutine mechanism will take care to preserve local variables, etc. until coroutine.resume() is called. Of course you have to assume that the game state will change "behind the back of your task" when you call yield(), but that synchronization problem is inherent in any method of concurrent programming.

 

Look at the Lua docs to find out what you can do with coroutines, what the various return values mean and how error handling and reporting works in coroutines.

 

EDIT: Obviously you'd want to use mist.scheduleFunction() or similar to call coroutine.resume() on all your coroutines regularly.


Edited by [FSF]Ian
  • Like 1
Link to comment
Share on other sites

If you have a big job that you want to execute infrequently, you should read up on coroutines. They won't help if the combined work of the simulation and your scripts is too much for one thread, but if your scripts do very little most of the time but then block for a whole second every 15 seconds, you can split the work up into smaller chunks and spread it out over the whole 15 seconds.

 

It's basically what I assume you are already doing now by processing smaller tables instead of one big one, but it's easier to use so you can use it in more places.

 

That's what I already did and it helped considerably.

 

Working with the coroutines and yield could be very efficient.

Link to comment
Share on other sites

Can you give an example? or explain how do you mean?

 

Sorry, I don't know how to do that in LUA. In C++ it would be easy to do and what I've read LUA has some kindof socket-wrapper support also.

I haven't looked into how scripts in missions are executed so it is really upto you to figure out if/how that can be implemented.

 

There are some kind of trigger support in DCS missions that I've seen.

So, for example, when enemy unit reaches some marked point it can trigger some script and so forth.


Edited by kazereal

"I would have written a shorter post, but I did not have the time."

Link to comment
Share on other sites

multi threading

 

Hi I didn't find an other place to post my question about multithreading that the reason why i put it here even it's a more specific question.

 

I'm trying to modify the dcs mission environement during the mission by using a socket. The probleme that i have is that because of a while function dcs freeze.That 's why i thought about multythreading.

Do somebody know how I can use coroutine to avoid dcs to freeze?

 

(sorry for the language mistake I'm french)

Link to comment
Share on other sites

multi threading

 

Hi I didn't find an other place to post my question about multithreading that the reason why i put it here even it's a more specific question.

 

I'm trying to modify the dcs mission environement during the mission by using a socket. The probleme that i have is that because of a while function dcs freeze.That 's why i thought about multythreading.

Do somebody know how I can use coroutine to avoid dcs to freeze?

 

(sorry for the language mistake I'm french)

Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...