Changes to the behaviour of net.dostring_in()

[BIGNEWY]

Changes to the behaviour of net.dostring_in()


This DCS version ( 2.9.18.12722 ) changes the behaviour of net.dostring_in(), an API call that is used to call functions or pass information between the secure zones of DCS (userhook, scripting, gui). 
net.dostring_in() can now return values, which means that one can pass information directly between the different zones like so:
 

a,b,c = net.dostring_in('mission', 'return 1,2,3')

As net.dostring_in() is now even more powerful, security options had to be implemented to ensure that the API is only called between the intended secure zones.
You can define your specific security setup in a file named Saved Games\DCS\Config\autoexec.cfg. This file is not there per default, but can be created easily with any text editor.
The necessary content of the file could look like this:

if not net then net = {} end
net.allow_unsafe_api = {  -- this defines the secure zones where net.dostring_in() can be called from
  "userhooks",
  "scripting",
  "gui",
}
net.allow_dostring_in = { -- and this defines the zones that should be addressed from net.dostring_in()
  "server",
  "mission",
  "scripting",
  "gui",
  "export",
  "config",
} 

If you only want to use a lua file in your Scripts\Hooks directory that accesses some functionality in your mission (which is the most likely use-case), your autoexec.cfg will look like this:

 

 
if not net then net = {} end
net.allow_unsafe_api = {  -- this defines the secure zones where net.dostring_in() should be called from
  "userhooks"
}
net.allow_dostring_in = { -- and this defines the scopes that you want to use from net.dostring_in()
  "mission"
} 

Even if you do not write any LUA scripts, you still might use missions from other developers that do. Or you might use a modification or custom library for DCS from another community member. Any of these scripts might come with net.dostring_in() calls. So if you face any issues while running external scripts or missions with this new version, check the file Saved Games\DCS\Logs\dcs.log for any errors that point to net.dostring_in() and try to create a file like shown above. 

We also warn that by using this API you will allow missions/scripts to do unsafe actions up to arbitrary code execution. This API can be used ONLY WITH TRUSTED SCRIPTS AND MISSIONS!

[Camble]

Is the allow list exhaustive? I know some server-side scripts that use "server" when calling dostring_in.

[Veltro3]

53 minuti fa, Camble ha scritto:

Is the allow list exhaustive? I know some server-side scripts that use "server" when calling dostring_in.

Yes, indeed "server" is required by some server-side mods. For example, Olympus requires it.

[cfrag]

This is atrocious -- I am severely disappointed and aghast. I feel that you (ED and their community managers) could have announced this change more than 24 hours in advance (6 months would have been a better time frame), and perhaps solicited feedback from the community -- because the way that you (ED) chose to implement this sandbox protection scheme is a 1980's design. "autexec.cfg" - are you frigging kidding me? Doesn't anyone at ED understand modern UX and or UI design? Why use such an obsolete and user hostile one-size-fits-all scheme when it would be so much better to build a "sandbox on/off" switch into DCS proper, and then add a simple UI that allow users to add folders to an 'allow unsafe' box that automatically allows unsafe invocations for any mission inside? Users could drag folders/directories and/or missions into the box and - boom! - they allow unsafe invocations. Designing this is (or should be to your talent) near-trivial (I did interface stuff like that in 2 hours -- 2002, on a Mac in XCode!), and a so much better approach for users who grew up with a Mouse. I feel that this new 'design' of yours is unworthy of you, and it feels like such a blatant slap into all of your content creators' face that I am deeply saddened to see how you simply (and callously) implement a far-reaching change like this without ever feeling the need to consult your community and contributors. Are we that worthless to you? That new 'feature' of yours kills any mission that uses net.dostringIn(). You do realise that mission scripters resort to doscript_in because DCS's mission scripting environment requires this to circumvent a bad flaw in MSE's design, do you? This means that any mission today that is sufficiently advanced will stop to work and require that unsafe patch, which will probably result in most DCS installs being unsafe. The exact opposite of security - likely a complete failure of what you set out to do.

Why, why, why did you not try to sound out this change with those who spent lots of time creating missions for your product? 

I would have expected more from you, ED. For the record, can I request that you. or someone from ED, elaborate if you agree that this is merely a coarse stop-gap and that you do have a real, user-accessible and well thought-out future version waiting in the wings? Because this? This is bad design. Will we get a real 'sandbox' security interface for DCS? I don's feel like updating the 50 (?) missions I put on Userfiles, so maybe they run, maybe not. 9 out of the last 10 missions I published there no longer work and now require unsafe invocations. So be it. I cannot be bothered to care if you, dear ED, can't be bothered to engage with your community either. 

What a mess. I hope that you will fix it soon. Will you, ED?

[Veltro3]

8 minuti fa, cfrag ha scritto:

This is atrocious -- I am severely disappointed and aghast. I feel that you (ED and their community managers) could have announced this change more than 24 hours in advance (6 months would have been a better time frame), and perhaps solicited feedback from the community -- because the way that you (ED) chose to implement this sandbox protection scheme is a 1980's design. "autexec.cfg" - are you frigging kidding me? Doesn't anyone at ED understand modern UX and or UI design? Why use such an obsolete and user hostile one-size-fits-all scheme when it would be so much better to build a "sandbox on/off" switch into DCS proper, and then add a simple UI that allow users to add folders to an 'allow unsafe' box that automatically allows unsafe invocations for any mission inside? Users could drag folders/directories and/or missions into the box and - boom! - they allow unsafe invocations. Designing this is (or should be to your talent) near-trivial (I did interface stuff like that in 2 hours -- 2002, on a Mac in XCode!), and a so much better approach for users who grew up with a Mouse. I feel that this new 'design' of yours is unworthy of you, and it feels like such a blatant slap into all of your content creators' face that I am deeply saddened to see how you simply (and callously) implement a far-reaching change like this without ever feeling the need to consult your community and contributors. Are we that worthless to you? That new 'feature' of yours kills any mission that uses net.dostringIn(). You do realise that mission scripters resort to doscript_in because DCS's mission scripting environment requires this to circumvent a bad flaw in MSE's design, do you? This means that any mission today that is sufficiently advanced will stop to work and require that unsafe patch, which will probably result in most DCS installs being unsafe. The exact opposite of security - likely a complete failure of what you set out to do.

Why, why, why did you not try to sound out this change with those who spent lots of time creating missions for your product? 

I would have expected more from you, ED. For the record, can I request that you. or someone from ED, elaborate if you agree that this is merely a coarse stop-gap and that you do have a real, user-accessible and well thought-out future version waiting in the wings? Because this? This is bad design. Will we get a real 'sandbox' security interface for DCS? I don's feel like updating the 50 (?) missions I put on Userfiles, so maybe they run, maybe not. 9 out of the last 10 missions I published there no longer work and now require unsafe invocations. So be it. I cannot be bothered to care if you, dear ED, can't be bothered to engage with your community either. 

What a mess. I hope that you will fix it soon. Will you, ED?

 

I'm sorry but I am not very familiar with "normal" mission scripts limitations. What do you use dostring_in for? It was my understanding that before it was not available in the mission environment.

[cfrag]

10 minutes ago, Veltro3 said:

What do you use dostring_in for? It was my understanding that before it was not available in the mission environment.

It was. It is used to invoke some code in other environments and pass back data. Regard this snippet:

	local command = 't = loadfile("' .. path .. '"); if t then t(); return net.lua2json(towns); else return nil end'
	local json = net.dostring_in("gui", command)
	if json then 
		towns = {}
		traw = net.json2lua(json)
	end

Above (plus correct path info of course) reads all towns with their location from any map, and allows scripts to determine where something is, provide better locations, and even determine if a location if worth an attack (it is required because "towns" isn't otherwise accessible in MSE. And it simply worked. Same for radio frequencies, getting the name of the currently running mission (for easy restarts because the trigger version saves a string which buggers versioning). And of course, the venerable SSB (simple slot block) used it to pass slot blocking info. All gone to this 'improvement'

Edited Wednesday at 04:58 PM by cfrag

[Veltro3]

15 minuti fa, cfrag ha scritto:

It was. It is used to invoke some code in other environments and pass back data. Regard this snippet:

	local command = 't = loadfile("' .. path .. '"); if t then t(); return net.lua2json(towns); else return nil end'
	local json = net.dostring_in("gui", command)
	if json then 
		towns = {}
		traw = net.json2lua(json)
	end

Above (plus correct path info of course) reads all towns with their location from any map, and allows scripts to determine where something is, provide better locations, and even determine if a location if worth an attack (it is required because "towns" isn't otherwise accessible in MSE. And it simply worked. Same for radio frequencies, getting the name of the currently running mission (for easy restarts because the trigger version saves a string which buggers versioning). And of course, the venerable SSB (simple slot block) used it to pass slot blocking info. All gone to this 'improvement'

Understood, thank you. I have only used dostring_in in the hook environment.

[Benom8]

It's good to have more thought put into security around scripts, but I'm guessing (because of the way lua works) it's only possible to allow/deny on an environment basis, not for particular scripts. 

 

So, if users use any scripts that use dostring_in, they would need to disable protections globally anyway.

[cfrag]

6 minutes ago, Benom8 said:

but I'm guessing (because of the way lua works) it's only possible to allow/deny on an environment basis, not for particular scripts. 

It's really not hard (actually, it's trivial) to patch net.doStringIn() so that a security layer inspects the context and allows or disallows it depending on the mission file that is running, from ission to mission, in the 'mission' environment. The same goes for any other environment. I respectfully contradict your kind message and assert: anyone with access to the interpreter level and a modicum of Lua knowledge can make net.dostringIn() safe depending on the mission that is running. If we have a set of missions that we want to allow unsafe invocations, all we need is a list of their file names. And users could manage that easily, heck even (if you can't be bothered to give that to your customers) have them put in a folder '/unsafe' inside your '/Missions' folder. It's exceedingly easy to manage this, and it looks like no thought has gone into this at all. That's why I'm so disappointed.

[Camble]

1 minute ago, cfrag said:

It's really not hard (actually, it's trivial) to patch net.doStringIn() so that a security layer inspects the context and allows or disallows it depending on the mission file that is running, from ission to mission, in the 'mission' environment. The same goes for any other environment. I respectfully contradict your kind message and assert: anyone with access to the interpreter level and a modicum of Lua knowledge can make net.dostringIn() safe depending on the mission that is running. If we have a set of missions that we want to allow unsafe invocations, all we need is a list of their file names. And users could manage that easily, heck even (if you can't be bothered to give that to your customers) have them put in a folder '/unsafe' inside your '/Missions' folder. It's exceedingly easy to manage this, and it looks like no thought has gone into this at all. That's why I'm so disappointed.

Valid point. Since people will have to disable for an entire environment, the protection is effectively lost since they can accidentally run a malicious mission if they allowed for another mission. Same goes for IC, something gets added to IC and the only workaround is to disable it completely.

[okopanja]

So which additional functions will be added to this scheme in future?

[Benom8]

14 minutes ago, cfrag said:

It's really not hard (actually, it's trivial) to patch net.doStringIn() so that a security layer inspects the context and allows or disallows it depending on the mission file that is running, from ission to mission, in the 'mission' environment. The same goes for any other environment. I respectfully contradict your kind message and assert: anyone with access to the interpreter level and a modicum of Lua knowledge can make net.dostringIn() safe depending on the mission that is running. If we have a set of missions that we want to allow unsafe invocations, all we need is a list of their file names. And users could manage that easily, heck even (if you can't be bothered to give that to your customers) have them put in a folder '/unsafe' inside your '/Missions' folder. It's exceedingly easy to manage this, and it looks like no thought has gone into this at all. That's why I'm so disappointed.

I would like to think I have a fair amount of Lua knowledge from the C and script side , but I have not seen a reflective approach to checking which file is being executed. I will have a think about that. (Not to derail the conversation into Lua technicalities, just wanted to say thanks for pointing out this possibility!).

Edited Wednesday at 05:45 PM by Benom8
Typos

[eekz]

Where is the "server" env which I used to use? Seems don't work anymore. Have to fix it on the go during event....

PS: seems it's now "scripting"

Edited Wednesday at 07:30 PM by eekz

[eekz]

by the way, i don't know why dostring_in needed a return values implementation if it raises such security issues.. there are plenty of other ways to transfer data from scripting to gui and vise versa. Can someone explain the benefit?

personally i would love to have the ablity to use net.spawn_player(clientId) on a client directly from the server. that would really add additional functionality and possibilities.

Edited Wednesday at 07:51 PM by eekz

[Karlos]

2 ore fa, cfrag ha scritto:

And of course, the venerable SSB (simple slot block) used it to pass slot blocking info. All gone to this 'improvement'

Will all servers using SSB no longer work after this update?

DML included?

 

[cfrag]

11 hours ago, Karlos said:

Will all servers using SSB no longer work after this update?

DML has completely gone South, and today I do not feel charitable. Any mission using a recent version of DML, Olympus, SSB, or slotblock (i.e. anything Foothold or similar) will no longer work without you changing your DCS install. Expansion, most of the missions I released to UserFiles in the last year, and many interesting missions created by others in the last 24 months are affected.

They continue to run, but you must install that godawful "autexec.cfg" <shudder> in the correct location(s). And yeah, that includes your servers. Good luck figuring out which context name corresponds to "scripting" - you may have to rewrite your server scripts. If you can. Fun.

If this continues, I foresee that - out of simplicity/convenience - 90% of all DCS installs are set to execute insecure code, completely ruining security - the exact opposite of what the kind people at ED set out to do. Maybe they should have asked someone...

Edited Thursday at 07:40 AM by cfrag

[Karlos]

This is crazy. My server and almost all the servers I've visited use SSB and DML. So, the work of people who dedicate secure and free scripting to the ED community is not good? Are we crazy? I'll shut down my servers. For ED, are we just numbers? Probably.

 

[jmossr]

We already had at least one return value. I used to use this call with a single return value as follows:

local ret, success = net.dostring_in(env, lua)
if success then
...

Just tested  both a failing call and a successful call and both still work as expected with a single return value. I haven't tried with more return values yet. And yes, I had to add the autoexec.cfg as specified. The only thing I had to change in my own code was replace "server" by "scripting" as mentioned by eekz.

I'm not sure I understand the reasoning behind these changes.

[Asphelite]

5 hours ago, Veltro3 said:

Yes, indeed "server" is required by some server-side mods. For example, Olympus requires it.

Hello

Can confirm whilst fixing HBCW, "server" is indeed a valid field and does need to be allowed in the dostring_in table in the autoexec.cfg

If possible, Id like to see the example list (posted by BN) to be updated Cheers!

Edited Wednesday at 09:44 PM by Asphelite

[Special K]

vor 2 Stunden schrieb Asphelite:

Hello

Can confirm whilst fixing HBCW, "server" is indeed a valid field and does need to be allowed in the dostring_in table in the autoexec.cfg

If possible, Id like to see the example list (posted by BN) to be updated Cheers!

Done.

vor 4 Stunden schrieb Karlos:

This is crazy. My server and almost all the servers I've visited use SSB and DML. So, the work of people who dedicate secure and free scripting to the ED community is not good? Are we crazy? I'll shut down my servers. For ED, are we just numbers? Probably.

 

This feature was a long asked question from the community. Mod makers are licking their fingers after it. But it comes with a cost. It opens up options that can be critical for the users of DCS World, so ED had to do something.

Sure, there are better ways of doing it. But the enhanced security was indeed needed. It is 2 lines you add to your autoexec.cfg, so I am pretty sure one can survive that. And no, I am not going into detail what potential this call has, for reasons.

Edited Thursday at 12:21 AM by Special K

[MarcosR]

Can someone confirm is working? I can't make it work in rotorheads..

Can execute, but cant receive any data.

 

Edited Thursday at 01:28 AM by MarcosR

[392198523]

2小时前，Special K说：

It is 2 lines you add to your autoexec.cfg

Yep, that's a 2 lines change which totally ruins our newly introduced enhanced security.

 

6小时前，Karlos说：

This is crazy. My server and almost all the servers I've visited use SSB and DML. So, the work of people who dedicate secure and free scripting to the ED community is not good? Are we crazy? I'll shut down my servers. For ED, are we just numbers? Probably.

I have finally decided to shut down my server since the dispute between 15E. Many things have totally dampened my enthusiasm at DCS, and my trust to ED.

[Buta]

How can I know what those "server", "mission", "scripting", "gui", "export", "config", "zone" mean and what they can do? I hope the answer won't be "guessing", "trying" and "testing". Is there a doc to explain this api?

[Mr.Jazzy]

If mod makers are licking fingers after this feature, what kind of mods are we talking about?

[cfrag]

6 hours ago, Special K said:

enhanced security was indeed needed.

No discussion here. DCS needs good, scalable and easy to maintain security. In the past, I have recommended a couple of approaches in these forums how, for example, DCS could easily sandbox (better: sidestep) the lfs 'sanitize' issue by allowing a controlled, sandboxed 'writeString()' method that saves a string to a mission without having to 'de-sanitize' DCS (and compromise security). It was never followed up on. Pity. Instead we got the 'save mission' system that simply doesn't work, solves no problem at all, lfs still needs to be de-sanitized using a really, really bad 'unlock this for everyone' approach, just like this approach for dostring_in(). And to truly underscore how bad things are, the two approaches to resolving related issues (security) aren't harmonized. To me this is indicative of deeper troubling issues at ED than merely a bad IT security design team.

6 hours ago, Special K said:

Sure, there are better ways of doing it. But the enhanced security was indeed needed.

The problem is that this new 'enhanced security' is, as implemented now, utter garbage; an inept, heavy-handed approach that will lead to LESS security. Why? Because soon (if they haven't already) videos and article will appear that tell you how to 'fix DCS' when all these "attempt to call field dostring_in" errors pop up in-mission. Invariably they will tell you to 'simply place this autoexec.cfg here, and you are done'. Because of the hare-brained one-size-fits-all approach, now the entirety of DCS is open, for every mission (not to mention providing a new attack vector through a malicious autoexec.cfg that is provided by the attacker in the video). Anyone with merely passing working experience in security can tell you: this approach is bad, much worse than not doing anything.

So my opinion: this 'security' design is utterly inept, and it results in REDUCING security because everyone will bypass this 'security' for their comfort. Its like putting a fantastically complicated lock on your front door. Because it inconveniences everyone, they just keep it unlocked, opening the door for anyone. Is that improved security?

A better design would allow per-mission sandboxing. There a billions of possible approaches: put the mission in a folder /insecure, and if it is in there, use the settings screen in DCS to regulate what is allowed (and yeah, lfs and io should be in there too). Better yet: have a GUI for that allows players to drag missions/directories into. Any mission in there is allowed risky stuff (configured by the GUI). Simple, and it does not take a lot of talent to implement. I'm not saying that above is good security design. Far from it. I'm merely saying that it's better by orders of magnitude than the current, inept design. What deeply troubles me is the fact that something this badly designed has made it past the design stage. What angers me is that it made made it into production. ED - you have a serious quality issue. Cheap, badly thought out code like this should never make it to the customer.

This is really, really bad, ED. Please fix it. And please, if you don't have the talent, I'm sure there is ample talent in the community to help you out. Just ask.

Edited Thursday at 06:32 AM by cfrag