1.2.6 Update 3 Server Crashes

[NineLine]

Thanks, just know that we know there is server instabilities. I am just letting you know that we are trying to find steps to reproduce the crash(s). Nothing is random, something is causing them, you dont need access to anything but your current release to find that, if you look through this thread people give suggestions of what might be causing them and we submit those to the team to try during test sessions. If you have nothing more to add, that is fine... I am just saying I cant report to the devs that your server crashed as they will ask for the steps to reproduce it, with out those its a shot in the dark.

 

Anyways, lets keep this thread on topic, its been fruitful so far.

 

If I was able to identify the problem I would, however thats not my role nor in my spec without access to trunk.

 

I report the bug and supplied the logs as requested and a copy of the latest mission (which is open to anyone who joins the server)

 

Nothing more I can do for you at the this point other then report that it still is not fixed.

 

Pman

[Pman]

Thanks, just know that we know there is server instabilities. I am just letting you know that we are trying to find steps to reproduce the crash(s). Nothing is random, something is causing them, you dont need access to anything but your current release to find that, if you look through this thread people give suggestions of what might be causing them and we submit those to the team to try during test sessions. If you have nothing more to add, that is fine... I am just saying I cant report to the devs that your server crashed as they will ask for the steps to reproduce it, with out those its a shot in the dark.

 

Anyways, lets keep this thread on topic, its been fruitful so far.

 

I agree lets keep it on topic, Crashes in 1.2.6 Update 3.

 

I have provided you with everything that you need to report this as is the point of this section of the forum.

 

If you want me to spend the time trying to figure out why the sim crashes why ask for crash logs and dumps?

 

Although I appreciate that nothing in computer coding is truly random there is some degree of back end knowledge required that unless you work directly with the code members of the public will not have.

 

If you need more detailed information then I'd suggest that something is added to Mantis asking for more detailed error reports to be exported from DCS upon a crash.

 

I acknowledge that there has been some great work done by Eno, Frostie and the other server admins regarding combat problems which are more easily replicated (errors with Su25 weaponary for example) however in a free flight server its not as easy to reproduce and should actually indicate a more basic flaw in the code somewhere as the complications of weapons and AI units is removed from the scenario.

 

If you need any more information from my server then let me know

 

Pman

[NineLine]

If you need any more information from my server then let me know

 

Pman

 

Nope.

[Tango]

I've been flying on the VA server when the server has taken a dump, and no-one was doing anything in particular at the times it crashed.

 

For anyone to do more in-depth examination would require a debug build of the server, and/or more detailed crash logs.

 

It is a problem that basically requires a lot of dev time running the server and client in a debugger, and just flying until the thing crashes, then taking a look to see what happened.

 

If you want to see the problem in action, just fly on the VA server - it will fail eventually.

 

Best regards,

Tango.

[ENO]

I'm kind of hoping when 1.2.6 goes into this beta testing version they'd talked about before that some enhanced functions will be available.

 

I'm wondering if tacview files would be of any value... mine crapped out a couple updates ago but I'm thinking it may be worth putting back in service.

[NineLine]

Which is what we do during our MP test sessions.

 

But as well, I have to ask if their is any steps to reproduce, it could be as simple as comms with ATC, or something else you might be doing without even thinking about it that could be linked to the crash.

 

I am not saying finding the steps to reproduce is easy, but all I ask is if you were on during a crash, and you were performing any sort of action, try repeating it when the server comes back up... you never know when you might stumble on to the little missing bit of info to help find some of these instabilities.

 

I've been flying on the VA server when the server has taken a dump, and no-one was doing anything in particular at the times it crashed.

 

For anyone to do more in-depth examination would require a debug build of the server, and/or more detailed crash logs.

 

It is a problem that basically requires a lot of dev time running the server and client in a debugger, and just flying until the thing crashes, then taking a look to see what happened.

 

If you want to see the problem in action, just fly on the VA server - it will fail eventually.

 

Best regards,

Tango.

[NineLine]

I'm kind of hoping when 1.2.6 goes into this beta testing version they'd talked about before that some enhanced functions will be available.

 

I'm wondering if tacview files would be of any value... mine crapped out a couple updates ago but I'm thinking it may be worth putting back in service.

 

I am hoping this still happens as well, I would love to see large scale stress testing with a debugger running.

[NineLine]

So far we havent had much luck reproducing this, the test on Thursday yielded no results. That said it could be that the issue related to it is fixed, but that could just be hopeful wishes... I think the plan is to try it again the next test.

 

Okay- a couple steps here for Killbox since I've never had any trouble at all blowing it up at the very start:

 

What I'm assuming is that it has something to do with the mission scripting engine being unable to sync the "group to random zone" orders for the variety of convoys... The crashes don't happen until a couple of people are on the server. The crashes are always server side and clients generally ping-timeout but evidence of the crash is obvious much sooner. It happens as soon as a couple of blue aircraft enter the zone. Quite often if you're not on the server side you'll see vehicles start to float or just start driving through everything- off roads. It's important to note that all the convoys have "on road" instructions (the tanks up near Temryk too)- so when they start off roading you'll know that the server has crashed. ALWAYS Transport.dll.

 

As for the Wartweasel- again... server side crashes in MP setting only- The buses are being activated right at the beginning of the mission now and will basically hold for a few moments and then go about their business. I don't want to structure blowing them up wholesale too much in the event that some relation / causation things happen and blind us to the fact that something else is going wrong. You may want to set up an invisible air start slot close to the waypoint- most of the time I think guys are getting these errors it's while using gun or mav on those buses.

 

 

A couple other things happen around that time-

 

a) a boat offshore is late activated and goes about its route.

b) some speedboats are timed in eventually so that they make the beach just after the buses do.

c) AI from many of the vehicles is turned off (to simulate a retreat)

 

When I've had it happen before, we'd blown up a couple of buses before the game unhitched. Again- we noticed it originally when our weapons stopped killing vehicles. F2 views of other aircraft showed them blown up or flying through the earth... just a basic de-sync... while the server will have already crashed.

[ENO]

It sounded like something they knew about and had lots of info so I'm sure hopeful. Between these two errors and that be64 stack hash if they get those fixed that's basically it for crashes we are having.

[NineLine]

It sounded like something they knew about and had lots of info so I'm sure hopeful. Between these two errors and that be64 stack hash if they get those fixed that's basically it for crashes we are having.

 

 

I hope so, like to see you guys rewarded for your hard work in supplying all this data.

[ENO]

Different mission this time... during the fly in. Logs to follow. Wasn't there but I'll forward any notes if there are any.

 

Application Version: 1.2.6.19532

Application Timestamp: 5260f1fc

Fault Module Name: Effects.dll

Fault Module Version: 1.2.6.19532

Fault Module Timestamp: 52580634

Exception Offset: 000000000009ead8

Exception Code: c0000005

Exception Data: 0000000000000008

OS Version: 6.1.7601.2.1.0.768.3

Locale ID: 1033

Additional Information 1: b3b7

Additional Information 2: b3b7c28941ecf48f5561a1ed55d9909c

Additional Information 3: 1738

Additional Information 4: 1738e963fc8f22fe2e5d701c8cabc14b

crash 11.15.13.zip

Edited November 16, 2013 by ENO

[Azrayen]

For client crashes it would be better to start your own thread, unless it also brings down the server with it.

 

Rgr that, will do if able to reproduce.

[ENO]

Got a theory on effects.dll- I'm starting to wonder if its an AI unit that is either AI off or visible before start (late activated) getting destroyed.

 

Sith I may have had you guys shooting at the wrong things in your test with wartweasle.

 

 

Edit:

 

 

I did go in afterwards to see if I could get something to crash based on my hunch- I tried both late activated units as well as AI off units. I was a bit surprised to see that units that are late activated are invisible to the AI-

 

AI off units were fair game, however- and my AI forces blew them up with gusto.

 

With that being said, I haven't been able to try this in an MP format (because of the fly in)- but maybe someone else could give it a shot or I'll try at a later time. Basically it just needs to be a couple of people... some AI off / Late activated units (humans have to kill the LA units)... and go to town.

Edited November 16, 2013 by ENO

[SkateZilla]

Crashed / Lockup During the Sim HQ Flyin,

 

Was Taxi to runway,

-Enemy launched Parachute Flares over Target Area,

-Someone Destroyed the SA-9, Other stuff going on

 

(will fill in when I get that info)

 

DCS Appeared to Lock Up, only the Mouse Crosshairs were active, no view changes, nada.

 

CTRL+ALT+DEL, To End Task, and DCS Was using All 14 GB of Available Free Memory. (16GB Total, some being used by Windows/Applications.)

 

dcs - Copy.txt

[ENO]

Not my favourite DCS night last night by a long shot.

 

The one client crash was a crash but there was no log. This is the same one that happened when Skate got kicked. Well- we both basically just froze up wholesale. I had to shut down my computer to get back on the horse again.

 

 

The server didn't crash- but it was the firehouse... which is why these are in here. I have plenty of server crashes to add, however... and am in the process of doing so. Will be up shortly.

[ENO]

Server side logs- including the dcs.old that should have been recording that client crash included by skate and I. There was also a kick where we lost half a dozen guys... but a few of us stayed in. Pretty random stuff.

crash 11.17.13.zip

dcs.log.zip

Edited November 17, 2013 by ENO

[SkateZilla]

Forgot to mention Windows had control of my page file which was at the MAX Setting of 25GB as Well

[NineLine]

I've submitted loads of error logs and dump files- but in the event there is ANY value whatsoever I'm going to put all of them from October 5th or so on in one place.

 

Any hints to any actions you might believe are associated?

[ENO]

Sorry Sith- maybe these aren't any use at all... I just figured having all of them together from all the past submissions I've made-

 

All appear to be related to the transport.dll or effects.dll. A majority of them are from those two missions I mentioned before (Killbox and Wartweasel). A few of those BE64 Stackhash ones as well/

[NineLine]

ok, I dont think there has been much luck on the effects.dll front as far as reproducing it, not that that is bad news, it could mean its fixed in test versions... I think Grimes already commented on the trasnport.dll one...

 

Sorry Sith- maybe these aren't any use at all... I just figured having all of them together from all the past submissions I've made-

 

All appear to be related to the transport.dll or effects.dll. A majority of them are from those two missions I mentioned before (Killbox and Wartweasel). A few of those BE64 Stackhash ones as well/

[ENO]

Yeah, you're right... probably just frustrated. I've deleted.

[NineLine]

Yeah, you're right... probably just frustrated. I've deleted.

 

 

No worries, I understand, lets focus our attention at possible causes and less on logs and such for now. Lets see if we can find more concrete steps to reproduce...

[xcom]

I'd like to help anyway I can.

For example last night we had a mission with around 20 guys... Mission crashes twice after around 30 minutes on different servers, what would you need to help assist debugging?

 

BTW - is there a symbols file we can use to read the mini DMP file that DCS throws?

[NineLine]

I'd like to help anyway I can.

For example last night we had a mission with around 20 guys... Mission crashes twice after around 30 minutes on different servers, what would you need to help assist debugging?

 

BTW - is there a symbols file we can use to read the mini DMP file that DCS throws?

 

I dont have anything on how to read the DMP files, as far as helping, any chances of reproducing the crash are huge, so if you can either determine the steps to reproduce OR reproduce yourself, that goes a long way. For example tracks leading up to the crash could reproduce the crash.

[ENO]

Problem signature:

Problem Event Name: APPCRASH

Application Name: dcs.exe

Application Version: 1.2.6.20768

Application Timestamp: 528babc0

Fault Module Name: ZweiBlau.dll

Fault Module Version: 1.2.6.20768

Fault Module Timestamp: 528ba287

Exception Code: c0000005

Exception Offset: 0000000000030005

OS Version: 6.1.7601.2.1.0.768.3

Locale ID: 1033

Additional Information 1: 813f

Additional Information 2: 813fe86798e48d494a05ba73c1bc8fb7

Additional Information 3: 4910

Additional Information 4: 4910510aef5fd77434232307f6796398

 

 

strikes again. Logs to follow. interesting though- no crash file or dmp file...

do have the performance monitor in there too though.

crash 12.01.13.zip

Edited December 2, 2013 by ENO