reported Forced Termination (Login Expired) when Re-establishing Connection to Master Server

[Actium]

TL;DR: Not a crash, but a totally unnecessary forced termination: After having lost the connection to the master server temporarily, both DCS client and dedicated server will forcefully terminate with a Login session has expired popup message only when re-connecting to the master server.

IMHO it is very illogical to forcibly terminate the client only when the connection to the master server is re-established. Users' network connections may be intermittent. Additionally, the DCS master server is a global single point of failure with no redundancy (api.digitalcombatsimulator.com has a single A record). Outages have happened before. Either scheduled or repeatedly unscheduled (1, 2, 3). The latter have caused the described forced termination, dissatisfying customers.

This is a gigantic footgun simultaneously aiming at all customers' feet (a foot cluster bomb? a footnuke?). A short 20~30 minute master server outage will boot everyone from their game. Almost guaranteed to disgruntle those affected.

@ED: If you decide to fix this, please also consider enabling multiplayer access for sessions with saved authorizations. This would enable us the still play DCS in multiplayer (connect via IP), even if your master server goes down for more than just a few minutes, which can totally happen for lack of redundancy.

More Details

When DCS starts, it will authenticate with the master server. Afterwards, the client will periodically poll the master server in ~10 minute intervals. If the connection to the master server is lost, nothing happens, apart from more frequent attempts to reconnect. AFAICT, longer periods (tested for 1~2 hours) of a continuously unreachable master server have no direct consequences: DCS will stay connected to a multiplayer server, connect to new servers via IP, run the mission editor, etc. Obviously, the multiplayer server list will not populate.
However, as soon as the client re-establishes connection to the master server, DCS quits with the aforementioned popup message:

Steps to reproduce

1. Start DCS (in Online Mode), open the Mission Editor. Wait a minute.
2. Disconnect DCS from the master server api.digitalcombatsimulator.com (185.195.197.4). Either by disconnecting from the Internet, by using a firewall rule for the aforementioned IP, or by setting a non-existing network route via a command prompt with administrator privileges (route add 185.195.197.4/32 0.0.0.0). Alternatively, you can also just suspend your computer.
3. Keep DCS disconnect from the master server for 30+ minutes (less may suffice, too, but AFAICT it needs to miss more than just one periodic check-in).
4. Re-enable the connection to the master server. If you used the route method, delete the bogus route: route delete 185.195.197.4/32 0.0.0.0
5. Wait for DCS to quit within the next minute.

I've attached an exemplary dcs.log with DEBUG messages enabled. The connection was disabled ~1 minute after starting the game and re-enabled after an hour at exactly 09:59:13Z. DCS made the next reconnection attempt shortly after and then immediately exited with above popup.

Edited March 29 by Actium

[BIGNEWY]

Hi, 

we have made a report for the team to check. 

I have moved out of crash and into core wish list for now.

thank you 

[Actium]

Thanks. I really do consider this a crash-like situation, as the game decides to intentionally terminate itself, for no logical reason whatsoever. While IMHO still entirely unreasonable(!) from a usability and stability perspective, it would have made more sense to terminate when losing the master server connection. However, terminating when regaining the master server connection appears so extremely illogical that it can only be a bug*. Hope the Devs agree

*) Even for an always online title, which DCS -- fortunately -- is not, termination after successful reconnection and reauthentification would not make sense, unless its intent were purely punitive in nature.

[Actium]

Master server was down unannounced this morning for ~29 minutes from 00:02Z to 00:31Z. See attached dcs.log excerpt and associated tcpdump.log. As described above, my dedicated server decided to die only after the master server came back up:

00:33:57 hostname dcs-watchdog.py[794795]: No response from server: [Errno 111] Connection refused
00:34:12 hostname dcs-watchdog.py[794795]: No response from server: [Errno 111] Connection refused
00:34:27 hostname dcs-watchdog.py[794795]: No response from server: [Errno 111] Connection refused
00:34:27 hostname dcs-watchdog.py[794795]: Terminating after 3 missed responses.
00:34:27 hostname systemd[1395]: Stopping dcs-server@server1.service - DCS Dedicated Server...
00:34:28 hostname sh[804672]: Close message sent to top-level windows of process with PID 19873904.
00:34:30 hostname systemd[1395]: Stopped dcs-server@server1.service - DCS Dedicated Server.
00:34:30 hostname systemd[1395]: Started dcs-server@server1.service - DCS Dedicated Server.
00:34:30 hostname systemd[1395]: Started dcs-watchdog@server1.service - DCS Dedicated Server Watchdog.

Same happened to all dedicated servers on all discords I checked (4YA, TDCS, DFA/Grayflag) as well as to presumably all clients [1, 2, 3, 4].

@BIGNEWY Bringing this to your attention again. This issue takes down all clients and servers absolutely unnecessarily. This is not a nice-to-have wishlist item, but an undeniably serious bug that affects all players and server admins. It is exacerbated on dedicated servers, by requiring manual acknowledgment of above Login session has expired error popup (also see here). This prevents most automatic crash recovery scripts from restarting the servers, because the server process is blocked on the popup message and does not crash/exit. The process terminates after acknowledging the popup.

Edited March 29 by Actium

[Flappie]

BIGNEWY won't be here next week. I relayed your message on the internal report.

[Special K]

Am 25.2.2025 um 11:08 schrieb Actium:

TL;DR: Not a crash, but a totally unnecessary forced termination: After having lost the connection to the master server temporarily, both DCS client and dedicated server will forcefully terminate with a Login session has expired popup message only when re-connecting to the master server.

IMHO it is very illogical to forcibly terminate the client only when the connection to the master server is re-established. Users' network connections may be intermittent. Additionally, the DCS master server is a global single point of failure with no redundancy (api.digitalcombatsimulator.com has a single A record). Outages have happened before. Either scheduled or repeatedly unscheduled (1, 2, 3). The latter have caused the described forced termination, dissatisfying customers.

This is a gigantic footgun simultaneously aiming at all customers' feet (a foot cluster bomb? a footnuke?). A short 20~30 minute master server outage will boot everyone from their game. Almost guaranteed to disgruntle those affected.

@ED: If you decide to fix this, please also consider enabling multiplayer access for sessions with saved authorizations. This would enable us the still play DCS in multiplayer (connect via IP), even if your master server goes down for more than just a few minutes, which can totally happen for lack of redundancy.

More Details

When DCS starts, it will authenticate with the master server. Afterwards, the client will periodically poll the master server in ~10 minute intervals. If the connection to the master server is lost, nothing happens, apart from more frequent attempts to reconnect. AFAICT, longer periods (tested for 1~2 hours) of a continuously unreachable master server have no direct consequences: DCS will stay connected to a multiplayer server, connect to new servers via IP, run the mission editor, etc. Obviously, the multiplayer server list will not populate.
However, as soon as the client re-establishes connection to the master server, DCS quits with the aforementioned popup message:

Steps to reproduce

1. Start DCS (in Online Mode), open the Mission Editor. Wait a minute.
2. Disconnect DCS from the master server api.digitalcombatsimulator.com (185.195.197.4). Either by disconnecting from the Internet, by using a firewall rule for the aforementioned IP, or by setting a non-existing network route via a command prompt with administrator privileges (route add 185.195.197.4/32 0.0.0.0). Alternatively, you can also just suspend your computer.
3. Keep DCS disconnect from the master server for 30+ minutes (less may suffice, too, but AFAICT it needs to miss more than just one periodic check-in).
4. Re-enable the connection to the master server. If you used the route method, delete the bogus route: route delete 185.195.197.4/32 0.0.0.0
5. Wait for DCS to quit within the next minute.

I've attached an exemplary dcs.log with DEBUG messages enabled. The connection was disabled ~1 minute after starting the game and re-enabled after an hour at exactly 09:59:13Z. DCS made the next reconnection attempt shortly after and then immediately exited with above popup.

 

There is a difference if your server went down or if the master server went down.

If your server / network connection goes down, the session is being invalidated, if you do not send heartbeats. This happens after a specific while and is quite normal (session termination). In such a case, when your client (or in this case the server that is a client) re-connects, your session token is invalid, causing the error.

This leads to a situation, that you try to talk to the server with an incorrect session token, which would not work.

I agree, that if the ED server goes down and there is in fact no session anymore, the error should be different. And I was in contact already with the guys, to look at the reconnect to recover that. What can not happen is that ED has to keep all sessions of people where it is not to be detected if your server has crashed, your network has vanished or you just decided to go on vacation. So the reconnect has to be a temporary solution to either cover singular heartbeat misses or login server outages.

[Actium]

Universal, straightforward solution: If the master server responds 401 – for whatever reason, e.g., an expired token – the client (or dedicated server) should simply try to log in again in the background (no interaction required), instead of terminating itself. The regular 3 day grace period should start automatically (i.e., no popups that need manual acknowledgment). Only after it expires without ever reaching the master server again, the client (or dedicated server) may terminate itself. The grace period should reset after successful (re-)login with the master server.

The current situation where when the network goes down on either end, all affected clients (or dedicated servers) will terminate themselves, results in a terrible user experience, which should IMHO be of utmost concern. Good user experience, leads to customers willing to spend money on the product, which is presumably the primary revenue source. An expired session token, which could simply be renewed by logging in automatically in the background, as suggested above, is no justification to unceremoniously terminate the game.

On 4/2/2025 at 5:20 PM, Special K said:

What can not happen is that ED has to keep all sessions of people where it is not to be detected if your server has crashed, your network has vanished or you just decided to go on vacation.

Irrelevant wrt. above suggested solution, but just for the sake of argument: Why not? A session token should take no more database space than a properly salted and hashed password. I see no technical difference between keeping session tokens for up to 30 minutes (should be in non-volatile memory, anyway, for the sake of master server crash resilience) or retaining the most recent session token for each account, indefinitely.

[Special K]

vor 12 Stunden schrieb Actium:

Universal, straightforward solution: If the master server responds 401 – for whatever reason, e.g., an expired token – the client (or dedicated server) should simply try to log in again in the background (no interaction required), instead of terminating itself. The regular 3 day grace period should start automatically (i.e., no popups that need manual acknowledgment). Only after it expires without ever reaching the master server again, the client (or dedicated server) may terminate itself. The grace period should reset after successful (re-)login with the master server.

The current situation where when the network goes down on either end, all affected clients (or dedicated servers) will terminate themselves, results in a terrible user experience, which should IMHO be of utmost concern. Good user experience, leads to customers willing to spend money on the product, which is presumably the primary revenue source. An expired session token, which could simply be renewed by logging in automatically in the background, as suggested above, is no justification to unceremoniously terminate the game.

Irrelevant wrt. above suggested solution, but just for the sake of argument: Why not? A session token should take no more database space than a properly salted and hashed password. I see no technical difference between keeping session tokens for up to 30 minutes (should be in non-volatile memory, anyway, for the sake of master server crash resilience) or retaining the most recent session token for each account, indefinitely.

What you ask for is a change in the game. I even wonder where you see all these session failures you are reporting here? I am admin in 2 of the largest servers (GS and HBCW) and we do not see that at least. I get occasional messages from the other server admins but not on a daily basis or in any means as frequent as you describe it here.

And the session token is kept for more than 30 mins already. Each account has multiple session tokens btw. You can run more than one session at a time (one client and up to 5 server sessions on each account).

Edited April 4 by Special K

[Special K]

Just checked, we had a total of 6 of these events in the past half year on GS which probably corresponds with the auth server outages that were there.

[Actium]

10 hours ago, Special K said:

What you ask for is a change in the game.

Exactly. I'd consider it a no-brainer to fix an issue that'll boot all clients and brick all servers if the master server takes a break.

11 hours ago, Special K said:

You can run more than one session at a time (one client and up to 5 server sessions on each account).

Good point. But still shouldn't be a problem to track these.

11 hours ago, Special K said:

Just checked, we had a total of 6 of these events in the past half year on GS which probably corresponds with the auth server outages that were there.

So that's once a month on average. I'd call that quite often. Too often to forcibly terminate all currently logged in sessions, IMHO.

[Special K]

Am 4.4.2025 um 23:48 schrieb Actium:

So that's once a month on average. I'd call that quite often. Too often to forcibly terminate all currently logged in sessions, IMHO.

Well, we're talking a game here and not a high available banking application. I think an outage once a month is quite a great availability. You do not even get that SLA from any of your internet providers, so you can not expect that from ED also. Anyway, your point is noted, BN already said that the team will look into it. So lets see what comes out of it.