fixed Mission 04- "can not open debriefing file"

[baltic_dragon]

I know there is the same issue with some of the @Reflected missions as well, which looks like a more general problem.. and so far I haven't found a way to fix this on my end, but will keep you posted. 

[Reflected]

5 minutes ago, baltic_dragon said:

I know there is the same issue with some of the @Reflected missions as well, which looks like a more general problem.. and so far I haven't found a way to fix this on my end, but will keep you posted. 

I sent my mission to @[ED]Obi to test, fortunately it's a very short one.

[BuzzLine]

Short answer : Don't name a flag "break" or any other lua-reserved keywords.

 

debug_lua_parse.miz

Edited September 5, 2022 by BuzzLine
added a trivial repro case

[baltic_dragon]

Guys, does any of you who encountered this problem has a dsc.log and debrief.log files from straight after the issue appeared? It would be super helpful in trying to troubleshoot this issue. Thanks in advance!

[BuzzLine]

@baltic_dragon : expanding on my answer above, I believe the issue is caused by an orbit task given to Raven 22 (weed) that is stopped when a flag called "break" is set. Rename that flag to something else and that should fix the issue. (see screenshot below)

 

My guess on the root cause :

The debrief.log file is written on disk formatted as a lua file. It's not code, just objects, but it's convenient. It lists many things including all the events and the states of the flags and when they were set. The code that fills the debriefing window (when you end mission) is mostly written in lua and for convenience, it just gobbles up the debrief.log as if it were some lua code :

-- load debriefing data
function load(path)   
    local f, err = base.loadfile(path)

(taken from me_debriefing.lua somewhere in ED's code)

But loading debrief.log as if it were lua code assumes that debrief.log is a properly formatted lua file. HOWEVER, in our case, the debrief.log contains the following in the flags section : 

(...)
['50'] = 
{
	time	=	706.028,
	value	=	1,
}, -- end of ['50']
radio = 
{
  time	=	157.855,
  value	=	1,
}, -- end of radio
break = 
{
  time    =    433.901,
  value    =    1,
}, -- end of break
(...)

(in this snippet I show the flag numbered 50, and the flags named "radio" and "break") The problem is that "break" is a reserved keyword in lua (and many other programming languages) so the above snippet is not valid lua and that confuses the parser when it tries to loadfile it.

Anyhow. I don't work for ED, I don't have full access to their code, this is just guesswork, and maybe I'm all wrong. But to me it looks like a decent explanation for the issue. And it's a fairly benign fix that I'm sure someone would be willing to test, if you were to provide an updated mission file.

 

HTH, Buzz.

 

Edited September 6, 2022 by BuzzLine
typo

[Pyrocumulous]

Here is a dcs.log and debrief.log from last night (DCS 2.7.17.29140 Open Beta). This is following completion of a @Reflectedmission (Bf 109 K-4 Jagdflieger: Mission 2) in Steam VR, land, shutdown aircraft and then quit. I am greeted by this error when I should be at the debriefing screen: "can not open debriefing file." Click okay, and the next prompt is "Your progress will be lost. Are you sure you want to quit?" If I click No, it just leaves me in the hanger that should have the debriefing screen in it, but with no way to exit or do anything. If I end the mission early (shot down, crash, or otherwise), it debriefs fine.

The end of the dcs.log says:

2022-09-06 10:23:48.278 ERROR   Gui: ERROR loading debrief.log: constant table overflow
stack traceback:
    [C]: ?
    [C]: in function 'loadfile'
    [string "./MissionEditor/modules/me_debriefing.lua"]:437: in function 'load'
    [string "./MissionEditor/modules/me_debriefing.lua"]:402: in function 'show'
    [string "./MissionEditor/MissionEditor.lua"]:699: in function 'openReturnScreen'
    [string "./MissionEditor/MissionEditor.lua"]:717: in function <[string "./MissionEditor/MissionEditor.lua"]:711>

The debrief.log is 35 megabytes (!), which is way bigger than allowed to attach so here is a Google Drive link: https://drive.google.com/drive/folders/17LISQ9XJWb3KnixPQ_YCwKCVO-YIYhx6?usp=sharing

Hardware: Ryzen 5800x3D in Asus Pro WS X570-ACE motherboard, 64gb ECC DDR4 @ 2800mt/s, Radeon 6800XT, game on PCIe4 NVME, HP Reverb G2 in Steam VR.

If time allows, I'll try on DCS 2.7.17.29493 and an older version as well this evening.

Edited September 7, 2022 by Pyrocumulous
Corrected file name

[Reflected]

On 9/6/2022 at 1:36 AM, BuzzLine said:

Short answer : Don't name a flag "break" or any other lua-reserved keywords.

 

debug_lua_parse.miz 9.02 kB · 1 download

 

What other names should we avoid?

[BuzzLine]

48 minutes ago, Reflected said:

What other names should we avoid?

Assuming my hypothesis is correct and unless/until ED fixes the issue, avoid the following identifiers : https://www.lua.org/manual/5.1/manual.html

The following keywords are reserved and cannot be used as names:

     and       break     do        else      elseif
     end       false     for       function  if
     in        local     nil       not       or
     repeat    return    then      true      until     while

(case sensitive so Break and BREAK would be fine but I'd still avoid this).

[Reflected]

Thanks!

[Pyrocumulous]

2 hours ago, Pyrocumulous said:

If time allows, I'll try on DCS 2.7.17.29493 and an older version as well this evening.

Sample size of only one, but I was able to complete Jagdflieger: Mission 2 on DCS 2.7.17.29493 Open Beta and made it to the debrief screen. So, maybe whatever was broken is fixed in the latest DCS build? I was unable to find keyword "break" in this particular MIZ.

Debrief.log was 22 megabytes this time.

Edited September 7, 2022 by Pyrocumulous
add keyword break

[SrSosio]

Quick question. If I skipped the mission, and due to my OCD, want it to look as Success once the issue is fixed, can I do it without overwriting my progress in the further missions?

[baltic_dragon]

@BuzzLinethe "break" flag is the answer, thank you so much! 

[markom]

A someone who spent weeks last year dealing with the fallout of "processing logs as code" nonsense... loading a logfile as code is a profoundly dangerous practice and a huge security risk. ED should really not be doing that... 

[BuzzLine]

52 minutes ago, markom said:

A someone who spent weeks last year dealing with the fallout of "processing logs as code" nonsense... loading a logfile as code is a profoundly dangerous practice and a huge security risk. ED should really not be doing that... 

I thought as much but I looked and the call is wrapped in a properly set up setfenv jailbox. That's as secure as it gets, at least for consumer-grade applications. 

[markom]

27 minutes ago, BuzzLine said:

I thought as much but I looked and the call is wrapped in a properly set up setfenv jailbox. That's as secure as it gets, at least for consumer-grade applications. 

Unless "log" file is modified outside the calling function such that it executes bad things. This is, essentially, XSS in a different form. Anyway, not the place to discuss this :-).

[BuzzLine]

3 hours ago, markom said:

Unless "log" file is modified outside the calling function such that it executes bad things. This is, essentially, XSS in a different form. Anyway, not the place to discuss this :-).

Yes but no. I tried.

You can inject code through the flag name but at execution time it will have no access to any function, none at all, that's the whole point of the setfenv sandbox in Lua. ( see http://lua-users.org/wiki/SandBoxes ) I prefer of course when inputs are sanitized but here the problem is well contained and the solution is to avoid Lua-reserved keywords. No other vulnerability detected. Case closed.

[BuzzLine]

Maybe we should point this issue out to the devs ? They should either add a check that flag names are not a reserved Lua keyword or (better) properly escape the keys when they serialize data on disk ? Or move this thread to the Bugs section ? Let's ask @NineLine . For convenience, as this thread is getting long, below is my detailed investigation: 

 

[BIGNEWY]

1 hour ago, BuzzLine said:

Maybe we should point this issue out to the devs ? They should either add a check that flag names are not a reserved Lua keyword or (better) properly escape the keys when they serialize data on disk ? Or move this thread to the Bugs section ? Let's ask @NineLine . For convenience, as this thread is getting long, below is my detailed investigation: 

 

I believe it has been reported, but I am checking with the team.  

Edit: Yes this has been reported and a fix is being tested. 

thank you

[baltic_dragon]

I've also sent an updated version of the mission which fixes the issue on my part. 

[SrSosio]

32 minutes ago, baltic_dragon said:

I've also sent an updated version of the mission which fixes the issue on my part. 

How do those updates work? do we need to delete and re-download the campaign, or does it update like a game update?

Also, I skipped this mission, how can I fly it and be flagged as Success without overwriting the following missions?

Edited September 14, 2022 by Assamita

[Alpha_Mo66]

My guess is the update will only come into effect with the next DCS patch.. Best we get a confirmation from Baltic or ED in this case

[baltic_dragon]

It should be fixed with yesterday’s update guys.  

[SrSosio]

6 minutes ago, baltic_dragon said:

It should be fixed with yesterday’s update guys.  

Thank you BD. What about the other question, about re-running the mission without overwriting the campaign? I'll try to fly it as a single mission, see what it does

[SrSosio]

I just played the mission again (making sure the game was up to date), and I still have the same error message

[Alpha_Mo66]

It worked for me without getting the error message.. Thanks for the fix @baltic_dragon