DCS Server randomly freezing - need suggestions for troubleshooting

[Kid18120]

Hi all,

we're experiencing a bit of a weird issue with DCS Server (modular installer version) on a fresh Windows Server 2022 installation on a dedicated server box. 
To cut the text short, we have 3 instances of DCS server running which work properly but they do tend to randomly freeze (task manager shows them as "not responding" and not necessarily all 3 of them at the same time). 

Server logs were a bit of a mess so we cleaned up the machine and started the instances once again, so i will provide some logs as soon as the issue rehappens. 
It seems like the issue happens after about 10-12hours of running time, though sometimes it happens on instance1, some other times on the instance3, or other times on 2 of the 3 instances.
In  the meantime here are some more server related details:
 

• Server specs are: CPU Ryzen 5 7600X, RAM 64GB ECC, 1TB NVME SSD, running on a Windows Server 2022, no dedicated GPU
• DCS was installed via the dedicated server modular installer
• the user running on the server is the actual "Administrator" user with full access rights
• instances are all working on their dedicated saved games folders (instance1, instance2 and instance3 respectively)
• each instance has its own autoexec.cfg (see below for its content) where the only difference between the instances is the webgui port
• we have SRS, Olympus, LotATC and Tacview installed but currently only DCS, SRS and Olympus are being used
• running missions use DSMC and CTLD
• the 3 instances are running 3 different mission files, featuring  light, medium and havy loads in unit counts

 

While waiting for the servers to freeze up to get proper logs, which i will post here as soon as we have them, i'm eager to get your opinions on what to try or check in order to troubleshoot the issue.

 

Things we have tried so far:

• Cleanup/Reinstall software used (SRS, Olympus, etc.) 
• We were previously running on a local administrator user that was not "Administrator" so we switched everything back to use the actual "Administrator" account
• Change DCS, webguid, SRS and LotATC ports
• Running DCS_Server.exe as admin (Right click - properties - compatibility - run this program as administrator) ---> this is the latest change we made so we don't know if it might have fixed the issue, we'll see

 

Autoexec.cfg file content:

if not net then net = {} end
net.download_speed = 131072000
net.upload_speed = 131072000
net.use_ipv6 = false
net.use_upnp = false
options.graphics.render3D = false
webgui_port = 1234
disable_write_track = true
no_device_hotplug = true
crash_report_mode = "silent"

 

 

 

Thanks to anyone who will chime in.

Edited July 29, 2024 by Kid18120
Added some more info

[HC_Official]

we removed LotATC from our server a while ago as it was causing freezing/crashes

[Kid18120]

22 hours ago, HC_Official said:

we removed LotATC from our server a while ago as it was causing freezing/crashes

We thought about doing so as well but we do actively use it, plus on the old box it seems not to cause issues. Same software version for everything except Windows 10 on the old box and Server 2022 on the new one, so i think it's more likely a problem with either the new environment or some config/settings on DCS-side

Edited July 30, 2024 by Kid18120
fixed typos

[MK84]

22 hours ago, HC_Official said:

we removed LotATC from our server a while ago as it was causing freezing/crashes

This. Had to remove LotATC since it's causing frequent freezing and crashes with the July patch.

[Kid18120]

We're now running off a new fresh system install with all the bells and whistles except LotATC, seems to be stable. Sucks but hey, it works

[MsKatze]

I also encountered this problem, but I observed from Olympus that every time this situation occurs, the server-side FPS will be displayed as 0. At this time, I only need to pause the server and delete the redundant AI units (using Olympus) to restore its FPS to temporarily solve this problem until the next AI number increases.

Since the last update, the server usage has obviously increased, and I had to cut some game content to make the server stable.

Please forgive my poor English ability, I hope you can understand it

Edited August 2, 2024 by MsKatze

[Kid18120]

18 hours ago, MsKatze said:

I also encountered this problem, but I observed from Olympus that every time this situation occurs, the server-side FPS will be displayed as 0. At this time, I only need to pause the server and delete the redundant AI units (using Olympus) to restore its FPS to temporarily solve this problem until the next AI number increases.

Since the last update, the server usage has obviously increased, and I had to cut some game content to make the server stable.

Please forgive my poor English ability, I hope you can understand it

 

Thanks for the reply and yes, it's clear what you mean.
However, our server is set to that it goes into pause when no clients are present and the freezes always happen while noone is on the server. 
Olympus won't respond as the server is completely frozen and windows' task manager reports the DCS_server.exe as "not responding" and won't move from there until we kill it.