[REPORTED]Latest OB massive desync issues on large and small missions.

[robgraham]

So since the last open beta, the TGW servers and I'm going to guess others by the other thread here has been experiencing massive amounts of Desync between Clients and the server. Bignewy asked that I post the server specs etc here so I am, along with the last set of logs though right now this sets not going to show much as it's from well off peak times. But I'll point out that we are seeing this not only on our main mission which has been running with out issues on the previous builds no desync nothing for months now but also on small missions with little to no scripting and it is happening across many different clients and providers.

When this happens if you have the new information displayed you can see that it's not due to either high Ping Rates or Packet Error to the Client as the RTT which again i'm going to presume is Return Trip Time can be sub 50ms and the packet loss can be sub 0.09% all of which should mean that the link between client and server is rock steady.

 

The server is also running fine with out any issues when these are happening.

 

This is happening I'll point out on a server that is more than capable of running the 2 server instances we have running an I7 7700k with 32GB Ram DCS on a SSD, sitting on gigabit fibre at the OVH datacenter in Sydney, with a 250mb/s upload pipe and a 1gb down pipe, and can have this happen with as little as 1 or 2 people on the server, a server again at times we've had 32 people on with out issues and this exact same mission etc running fine, traceroutes from clients to the server at time of desync's show exactly what DCS shows same RTT in ping and 0% packet loss.

 

I've included the log as you asked newy but as shown in the thread here: https://forums.eagle.ru/showthread.php?t=267030 we aren't the only ones having these issues, 2.5.5.X was rock solid no issues, 2.5.6 first release was fine except for crashes caused by the transport.dll's etc, the last patch before this one had some ground unit desync start to happen and this patch just has things going insane.

DCS is running in dedicated server mode.

 

Yes we have Tacview running, Yes we have SLMOD running and yes we have lotatc running, but again we've always had these running and while I'd kill them until DCS offers Admin tools SLMOD is required and lotatc was working fine this time last week on the server and the server was running fine this time last week as well.

 

Server CPU utilisation even on our main server is at the highest 15-20% for each DCS instance and the main server has only ever reached 6gb of active ram useage, network utilization has NEVER been saturated and monitoring the Network while desync happens the send rate is no where near being hammered with it occurring at times with a out send of as low as 1.5mb/s.

 

we have a planned mission tonight that were hoping to run so i'll also try and get the log from that as well and have it running in full debug mode but this is the main server log but again while this mission is 'complex' we have had the desync occur on missions that have the minimum amount of scripting in them.

 

Its getting really frustrating and has a lot of our users wondering if the Supercarrier is even going to be usable when it drops because of the increasing amount of desync and other issues rather then improvements.

tgwmainserver.rar

[BIGNEWY]

Thank you for the data and report we will take a look.

[BIGNEWY]

Can we ask you to remove the carriers and see if you still see the same issue.

 

Thank you

[robgraham]

Yeah I'll remove the carriers and put the server back up and let you know what gets reported to me, even though Carrier ops are a chunk of our mission server

[Kid18120]

Can we ask you to remove the carriers and see if you still see the same issue.

 

Thank you

Getting rid of the carriers seems to actually solve the issue.

Some new code slipped in by accident? :music_whistling:

[BIGNEWY]

Getting rid of the carriers seems to actually solve the issue.

Some new code slipped in by accident? :music_whistling:

 

Maybe, we do have this reported internally, but thanks for the feedback, it has been very helpful. :thumbup:

[TAW_Blaze]

We have awful warping after ~ 3 hours in our server. I heard from others it's the same in the other bigger servers aswell.

[flyingsonofagun]

Yet another server admin here who has MAJOR issues in latest build with desync, rubberbanding, and rope slinging with choppers causing crash to desktops.

[robgraham]

Yeah we've still got a little but moment the CV group was removed we saw a massive increase both in terms of performance and in the Rubber Banding Newy.. Sorry it's taken so long, things have been a little .. busy here.

[BIGNEWY]

Yeah we've still got a little but moment the CV group was removed we saw a massive increase both in terms of performance and in the Rubber Banding Newy.. Sorry it's taken so long, things have been a little .. busy here.

 

No problem, real life always comes first, thank you for your feedback. :thumbup:

[flyingsonofagun]

OP, have you noticed any correlation with units being placed in groups vs individually? I too suspect the CV stuff is a CPU whore. I have spent literally days worth of time performance-hunting and I've gone mad over this.

[BIGNEWY]

If anyone is willing to send me a server track and a client track showing desync with the carrier present please PM me.

 

The track only needs to be long enough to show the desync

 

I want to get data from a busy server if possible.

 

Thank you

[TAW_Blaze]

The problem is even in busy servers generally the warping starts after 2-3 hours. I can ask around if someone has a track, I actually disabled my track recording in the hopes of improving performance.

[Mustang]

Remember in the old days when ED devs would host a large MP session to expose bugs? Why not do this again?

[BIGNEWY]

I will mention it to the team, they were helpful tests in the past. Things have been very busy internally as you can imagine so finding the time is the biggest issue.

[Mustang]

Op Desert Eagle V2 trackfile from server side (from same session as USMC_Trev posted above)

 

https://drive.google.com/open?id=183akjvhZMV25nmI-fqoc1OYdSJi7akc7

[BIGNEWY]

Thank you we will take a look.

[EpicNinjaCheese]

We have noticed the same issue on our servers. Running a moderately large mission (with no carriers present as it happened to be)

 

I was buddy lasing a target for another player, they were trying to hit a moving convoy on the ground. It appeared that the players aircraft was about 200m off position for the convoy. They released their GBU-12 and it seemed to travel about 200m to the east of where the target was. When the bomb hit the ground, it exploded. But so did the truck being lased 200m away from impact point. We also had issues with GBU-38's hitting about 100-200m off target in a seperate mission on the same build. I was giving precise coordinates for the ground vehicles, but the bombs would always miss by 50-100m (these targets were stationary however). It appears that ground vehicles are indeed desynchronised between clients

 

Hopefully this helps out a bit. I can provide more info if needed

 

Server specs: I7, 32gb ram, plenty of up/down bandwidth

[Maverick87Shaka]

Unplayable at all with more than 20 players, even without the carrier.

Worst experience ever :-(

Edited April 14, 2020 by Maverick87Shaka