Jump to content

LUA diff in secure way?


Recommended Posts

Hi all. I am new to this platform and so I am soliciting ideas/experience.

 

I was trying to diff two mission files to see what had gone wrong, breaking the product. My first approach was this:

 

  • expand miz files
  • convert each LUA table to a canonical form, meaning all fields are sorted and pretty printed the same way

 

That looked something like this:

 

#!/bin/bash

function canonicalize {
       lua - $1 <<EOF
local serpent = require("serpent")
local text = assert(loadfile(arg[1]), "failed to read "..arg[1])
local value = string.gsub(arg[1], '.*/', '')
text()
local formatted = assert(serpent.block(_G[value], {comment = false}))
print("rewriting "..arg[1].." in canonical form")
io.output(arg[1])
io.write(value.." = ")
io.write(formatted)
EOF
}

for MISSION in *.miz ; do
       BASE=${MISSION%.miz}
       mkdir -p "$BASE"
       pushd "$BASE"
       unzip -o "../$MISSION"
       for FILE in `find . -type f -not -name "*.*"` ; do
               canonicalize $FILE ;
       done
       popd
done

 

Then I was able to recursively diff. Now here is my problem: normal LUA pretty printers (I used the LUA rock 'serpent') require the table you want to pretty print be in a table variable. So that means my script reads the tables from the files. This is equivalent to executing LUA on my machine that I didn't write myself. Note the invocation of

text()

to get the table file contents into a variable.

 

Anyone could put anything they want in their mission file and I would end up running it on my machine. You see, in this scenario, the LUA is executed in an unrestricted LUA interpreter, not some embedded one in the game.

 

How do people deal with this? Is there a non-LUA parser/formatter for LUA tables that doesn't require executing the source? I realize this is a direct result of using a script LANGUAGE to store DATA, and I am taking short cuts by just running it. But there must have been other projects that handled this. I have seen some LUA code that attempts to read a table without executing anything, is that what you use?

 

Cheers and thanks for reading this long-@$$ post ;)

Link to comment
Share on other sites

ok I got a pointer from somewhere else, so here is a new proposed script (simply removing ALL LUA functions from the calling environment):

 

requirements:

  • LUA 5.1
  • luarocks
  • lua rock "serpent"
  • bash
  • unzip

 

#!/bin/bash

function canonicalize {
       lua - $1 <<EOF
local serpent = require("serpent")
local text = assert(loadfile(arg[1]), "failed to read "..arg[1])
local value = string.gsub(arg[1], '.*/', '')
setfenv(text, {})
text()
local formatted = assert(serpent.block(_G[value], {comment = false}))
print("rewriting "..arg[1].." in canonical form")
io.output(arg[1])
io.write(value.." = ")
io.write(formatted)
EOF
}

for MISSION in *.miz ; do
       BASE=${MISSION%.miz}
       mkdir -p "$BASE"
       pushd "$BASE"
       unzip -o "../$MISSION"
       for FILE in `find . -type f -not -name "*.*"` ; do
               canonicalize $FILE ;
       done
       popd
done

 

Do you think this is safe? It worked fine in a simple test but I am no LUA expert.

Link to comment
Share on other sites

I am not sure I got your point exactly, but when I needed to parse and then alter "mission" files LUA table, I pushed them into an ordered Python dictionary, without running any piece of LUA code. Then changes can easily be made programmatically, and re-injected into the mission file, or spawn any number of new tweaked *.MIZ as necessary. But you could do pretty much whatever you want with the resulting data, very easily.

 

I chose Python because of it being widely available on almost every machine/server.

 

If you're interested, I already made the parser, along with all the *.MIZ file handling. You'd just need to tailor it down to your specific needs.

Link to comment
Share on other sites

Daribouca:

 

Let me try to re-state my point then:

 

I was trying to canonicalize the LUA tables, so that tables from two different miz files can be compared, regardless of the order of fields and escaping/quoting/formatting used originally.

 

Sucking the whole thing into python dictionaries and then formatting back out in canonical form would presumably accomplish this. But you are then relying on your own implementation of any escaping that LUA might use.

 

So yeah, you could do that. But I was asking about a more direct way in LUA, and whether the answer I gave for that (see the LUA code embedded in the bash script above?) is sane from a security perspective.

Link to comment
Share on other sites

Anyone could put anything they want in their mission file and I would end up running it on my machine. You see, in this scenario, the LUA is executed in an unrestricted LUA interpreter, not some embedded one in the game.

 

How do people deal with this? Is there a non-LUA parser/formatter for LUA tables that doesn't require executing the source? I realize this is a direct result of using a script LANGUAGE to store DATA, and I am taking short cuts by just running it. But there must have been other projects that handled this. I have seen some LUA code that attempts to read a table without executing anything, is that what you use?

 

If you need to dostring on untrusted code, you make a sandboxed environment for that code, and run it in there.

 

do
local untrustedLua = <some string data from where ever, that you don't necessarily trust>	

local env = {}
--[[ now, either add to env all the functions you think are safe, or copy 
everything from _G into env except those functions/libs you think are unsafe.

It is probably better to manually add references to safe functions/libs to env rather than copy
over _G and then remove unsafe refereneces.  That way, you are not dependent on what else _G might
or might not contain.
]]

--[[in this example, I copy a few SAFE funcitons/libraries from _G into env that are safe.
Do not consider this a complete set of safe functions!  Google "Lua Sandboxing" for a more
complete list!]]

env.print = print
env.string = string
env.table = table
env.math = math
env.loadstring = loadstring
-- etc.


local f = loadstring(untrustedLua)  --now, compile the untrusted code
if f then  -- the untrusted Lua compiled successfully.
	setfenv(f, env) -- Switches the environment the compiled, untrusted code will run in from _G to our new, safe environment.
	pcall(f) -- now, run the unsafe code in our safe environment, where it cannot do any damage.
                          -- Also note the use of pcall.  pcall prevents runtime errors in the untrusted Lua code from causing a 
                          -- Lua error.
end

end


Edited by Speed

Intelligent discourse can only begin with the honest admission of your own fallibility.

Member of the Virtual Tactical Air Group: http://vtacticalairgroup.com/

Lua scripts and mods:

MIssion Scripting Tools (Mist): http://forums.eagle.ru/showthread.php?t=98616

Slmod version 7.0 for DCS: World: http://forums.eagle.ru/showthread.php?t=80979

Now includes remote server administration tools for kicking, banning, loading missions, etc.

Link to comment
Share on other sites

[edit] sorry Speed, I just realized that you replied to my first post, which didn't have the sandbox. I believe that we are on the same page.[/edit]

 

I already do that in the code I presented above. I didn't pcall because I WANT to abort if anything errors in the untrusted code, so I don't overwrite the file in that case.


Edited by derammo
Link to comment
Share on other sites

[edit] sorry Speed, I just realized that you replied to my first post, which didn't have the sandbox. I believe that we are on the same page.[/edit]

 

I already do that in the code I presented above. I didn't pcall because I WANT to abort if anything errors in the untrusted code, so I don't overwrite the file in that case.

 

Ummm... is that pseudocode you posted?

 

Anyway, without pcall, the Lua calling the untrusted code will crash. I donno why you would ever want to do that. It's better if you just test whether or not the first value returned by pcall was true or false. pcall returns true (plus any values returned by the function) if the function it calls executes without error. If there is an error, pcall returns false, plus an error message string that tells what the error was.


Edited by Speed

Intelligent discourse can only begin with the honest admission of your own fallibility.

Member of the Virtual Tactical Air Group: http://vtacticalairgroup.com/

Lua scripts and mods:

MIssion Scripting Tools (Mist): http://forums.eagle.ru/showthread.php?t=98616

Slmod version 7.0 for DCS: World: http://forums.eagle.ru/showthread.php?t=80979

Now includes remote server administration tools for kicking, banning, loading missions, etc.

Link to comment
Share on other sites

I see our difficulty at communicating now :). This is definitely not pseudo code, it runs fine.

 

I work on Linux in bash shell, and this is a bash script that contains and runs a LUA script.

 

The bash script iterates all the .miz files and creates a directory for each MIZ into which it is unzipped. For each such directory, it iterates all the files that have no file extension (these are the tables, rather than the LUA scripts.)

 

Then it calls a bash function for each such file. This function is declared in the same file. It simply calls the command line lua interpreter, feeding it a LUA script that is also embedded in the script. If anything goes wrong in reading the table, lua exits with an error and it does not get to rewrite the file. If it correctly formats pretty printed and sorted table output, then it overwrites the original table file with that.

 

When its all done you have a directory for each .miz file in the current directory, containing canonical form tables, which can therefore be compared since all the fields are in the same order.

 

Here's the code again with markup to follow along [edited to correct LUA script, previous version was not doing anything]:

 

#!/bin/bash
# this comment means linux automatically runs this script in bash if executed

# a bash shell function
function canonicalize {
       # run LUA command  interpreter, taking script from standard input (-) and passing name of table
       # << EOF means replace standard input with contents of this file from here until
       # you see EOF again
       lua - $1 <<EOF
-- beginning of LUA code
-- load "serpent" LUA rock, which is a pretty printer
local serpent = require("serpent")
-- load the table or crash, returning to iteration of files in bash
local text = assert(loadfile(arg[1]), "failed to read "..arg[1])
-- remove path to get name of table variable we expect in the file
local value = string.gsub(arg[1], '.*/', '')
-- disable ALL LUA functions
setfenv(text, {})
-- invoke table code, which will just define that one table variable
text()
-- retrieve expected variable from sandbox environment
local loaded = getfenv(text)[value]
-- make sure we loaded something, so we don't serialize nil
assert(loaded)
-- serialize canonical LUA
local formatted = assert(serpent.block(loaded, {comment = false}))
-- comment
print("rewriting "..arg[1].." in canonical form")
-- since we didn't crash, replace the original file
io.output(arg[1])
-- add the assignment back since this table file is supposed to create this table var
io.write(value.." = ")
-- write the pretty table
io.write(formatted)
-- end of code piped into LUA
EOF

# and now we are back in bash
}

# iterate all *.miz files
for MISSION in *.miz ; do
       # get the name of the miz file without the extension
       BASE=${MISSION%.miz}
       # use this as a directory name
       mkdir -p "$BASE"
       # change to that directory, remembering where we come from
       pushd "$BASE"
       # unzip the MIZ
       unzip -o "../$MISSION"
       # for all files without a file extension
       for FILE in `find . -type f -not -name "*.*"` ; do
               # call canonicalize function to rewrite it, assuming it is a LUA table
               canonicalize $FILE ;
       done
       # return to directory we came from
       popd
done


Edited by derammo
code was incorrect
Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...