Why is ED suddenly recommending MP players use BETA over Stable?

[Dangerzone]

On the page DCS World (digitalcombatsimulator.com) ED has now added "Best choice for Multiplayer mode" against their Open Beta download (but still recommends stable for SP???). From what I can tell, this recommendation has only just been added.

I'm wondering what's changed to now recommend Beta for players? (I know every person has their individual preferences as to what to use- but this question is not about what people prefer to use, but rather wanting to understand the logic behind ED's decision to now recommend Open Beta for multiplayer server).

I would have thought that Open Beta is best for those willing to test new features - before it's officially released as stable, not for an actual production environment. ED even state on the same page:

Quote

The Open Beta is an optional version of DCS World that we use to first test new content and technologies before moving them to our release version.

Shouldn't Stable be recommended for a better experience, leaving BETA testing to the enthusiasts and testers that are happy to test and identify bugs instead of recommending the MP community migrate to BETA? To me it seems recommending the general community perpetually run a BETA install than stable is only be inviting problems when bugs are introduced and not given time for community testing.

This leaves me with the question - why is ED recommending server operators and players actually run Open Beta over Stable Release, what am I missing, and why should I be choosing to host on OB instead of Stable? 

Edited October 5, 2022 by Dangerzone

[Exorcet]

A lot of players ignore the distinction between Beta and Stable and use Beta in the first place. If they're doing that already then it seems they care less about new bugs so they might as well be testers I guess.

[Dangerzone]

20 minutes ago, Exorcet said:

A lot of players ignore the distinction between Beta and Stable and use Beta in the first place. If they're doing that already then it seems they care less about new bugs so they might as well be testers I guess.

I get a lot of players ignore this - but that's their choice, and I have no issues with that. It seems a bit weird to me on the other hand that ED would promote this though as the best path for MP. What's best for people, and what they choose are often two different things. (Otherwise my doctor may as well say the best drink for me is beer, even if it isn't because hey - that's what people are doing). 

So, I'm more interested in ED's reasoning behind advocating that path - not so much what users are doing. 

[chroma_aus]

Because nearly all the multiplayer servers run Open Beta, and always have. ED's probably just adding that now to cut down on the daily masses of people asking why they can't see anything on the connection list.

There's also no reason not to run Open Beta, since Stable is just an older version of OB with no changes made. Both versions are essentially as buggy as one another, with the exception that OB gets fixes first. 

[draconus]

1 hour ago, Dangerzone said:

So, I'm more interested in ED's reasoning behind advocating that path - not so much what users are doing. 

1. It's in ED's own interest to have more testers, esp. in MP.

2. Most servers already use beta so it's a no-brainer this version is recommended by everyone including ED.

[Tom Kazansky]

and since ED established their closed beta testers team, the quality of open betas they put out seems to be quite decent, compared to earlier days.

[QuiGon]

1 hour ago, chroma_aus said:

Because nearly all the multiplayer servers run Open Beta, and always have. ED's probably just adding that now to cut down on the daily masses of people asking why they can't see anything on the connection list.

Probably this

Edited October 5, 2022 by QuiGon

[Rudel_chw]

3 hours ago, Dangerzone said:

I'm wondering what's changed to now recommend Beta for players?

Probably because the network protocol was changed on the last OB update, having everyone on OB would ensure all players used the same protocol:

 

"DCS Open Beta 2.7.18.30765

https://www.digitalcombatsimulator.com/en/news/changelog/openbeta/2.7.18.30765/
DCS World

• Network protocol version changed"

[martinistripes]

I think most people just can't wait for the latest features and updates. I've always wanted to stay with stable. Stable makes even more sense in a competitive environment. I was perplexed when I couldn't find the multiplayer servers I wanted to join. Then discovered it was because they nearly all ran open beta. I don't have the disk space for two installs, so I've been dragged onto the open beta cart like everyone else.

[BIGNEWY]

Hi all, 

we have added this to the download page as most servers are using open beta ( a choice the community has made ) and often new users will ask why there are not many servers on the stable version. It can be frustrating for new users who have downloaded the stable version only to find the community is using open beta for multiplayer and have to download open beta after the fact. 

We totally understand peoples need to see new features and modules as soon as they hit a public version of DCS (even if it is a public test version), but also want new users who want to play multiplayer to hit the ground running and have access to as many servers as possible. 

thanks 

Bignewy

 

[QuiGon]

3 hours ago, Rudel_chw said:

Probably because the network protocol was changed on the last OB update, having everyone on OB would ensure all players used the same protocol:

 

"DCS Open Beta 2.7.18.30765

https://www.digitalcombatsimulator.com/en/news/changelog/openbeta/2.7.18.30765/
DCS World

• Network protocol version changed"

That changes with most updates though.

[Rudel_chw]

1 hour ago, QuiGon said:

That changes with most updates though.

 

Didn't knew that, it's the first time that I have seen this being mentioned on a changelog.

[BIGNEWY]

protocol change happens so different versions dont see each others servers, that is all. For some reason the team put it in the change log this time. 

thanks

[QuiGon]

Just now, Rudel_chw said:

Didn't knew that, it's the first time that I have seen this being mentioned on a changelog.

They often just don't mention it in the change notes, but it changes with most of the updates to avoid desync issues that might happen if two clients with different versions play on the same server.

On occaison it is mentioned though:

https://www.digitalcombatsimulator.com/en/news/changelog/openbeta/2.7.8.16140/

[Dangerzone]

11 hours ago, BIGNEWY said:

Hi all, 

we have added this to the download page as most servers are using open beta ( a choice the community has made ) and often new users will ask why there are not many servers on the stable version. It can be frustrating for new users who have downloaded the stable version only to find the community is using open beta for multiplayer and have to download open beta after the fact. 

We totally understand peoples need to see new features and modules as soon as they hit a public version of DCS (even if it is a public test version), but also want new users who want to play multiplayer to hit the ground running and have access to as many servers as possible. 

thanks 

Bignewy

 

Thanks Bignewy for the reply. So to clarify, the recommendation has nothing to do with a better experience as far as stability, better for server hosting, or any reason other than that the majority of the online community have chosen to host and play Beta versions to get earlier access rather than to have a more stable platform. And if so (and I and the team I play with don't care about numbers) - we're still OK (and probably even better off) sticking with Stable Release?

[BIGNEWY]

11 minutes ago, Dangerzone said:

Thanks Bignewy for the reply. So to clarify, the recommendation has nothing to do with a better experience as far as stability, better for server hosting, or any reason other than that the majority of the online community have chosen to host and play Beta versions to get earlier access rather than to have a more stable platform. And if so (and I and the team I play with don't care about numbers) - we're still OK (and probably even better off) sticking with Stable Release?

Correct, open is and will always be a public test build, for a more stable experience stable version should be used. The notice is purely to let people know the majority of online community servers are on open beta.

thanks

[ApoNOOB]

Most of the community playing MP is on open beta. So if people want to get the most out of MP experience, they will need to use Beta version. Traditionally the MP scene is running beta because new modules/toys.

[Mike_Romeo]

Also stable isnt as stable as the name suggests

[BIGNEWY]

On 10/9/2022 at 12:16 PM, Mike_Romeo said:

Also stable isnt as stable as the name suggests

It is a build that has the lowest potential for a crash, which is why we have the public open beta as a public test version. There will always be bugs and the team works on them, you only have to look at the change logs we post to see that. I think you know that. 

[Mike_Romeo]

1 hour ago, BIGNEWY said:

It is a build that has the lowest potential for a crash, which is why we have the public open beta as a public test version. There will always be bugs and the team works on them, you only have to look at the change logs we post to see that. I think you know that. 

I agree with you that there will be always bugs and I see the progress you guys do but what I dont get is how the stable version is supposed to be more stable when its on the excat same version number as the open beta one as right now. The stable version is most of the time just 1-2 patches behind the open beta and gets too often updated to the open beta standard and so it appears to me more as a outdated open beta version instead of a stable branch.

I would suggest to leave the stable version always at least 2 patches behind open beta but never on the same version number as the open beta one. This way you can say that the latest patches of the open beta are still in testing and so you cant put them into stable but when you regulary update the stable to open beta standard, it really doesnt appear more stable and I really dont see the benefit in choosing the stable over the beta.

Here a overview about how long stable and open beta had been on the same version since 2021

Stable and open beta where on the same version number for 137 days this year alone and this year only had 285 days so far. Its fair to say that both are half of the year the same.

Edited October 14, 2022 by Mike_Romeo

[BIGNEWY]

1 hour ago, Mike_Romeo said:

I agree with you that there will be always bugs and I see the progress you guys do but what I dont get is how the stable version is supposed to be more stable when its on the excat same version number as the open beta one as right now. The stable version is most of the time just 1-2 patches behind the open beta and gets too often updated to the open beta standard and so it appears to me more as a outdated open beta version instead of a stable branch.

I would suggest to leave the stable version always at least 2 patches behind open beta but never on the same version number as the open beta one. This way you can say that the latest patches of the open beta are still in testing and so you cant put them into stable but when you regulary update the stable to open beta standard, it really doesnt appear more stable and I really dont see the benefit in choosing the stable over the beta.

Here a overview about how long stable and open beta had been on the same version since 2021

Stable and open beta where on the same version number for 137 days this year alone and this year only had 285 days so far. Its fair to say that both are half of the year the same.

 

We test public open beta to ensure there are no show stopping issues that would prevent the stable version from being as stable as it can be. 
Once we can see open beta is not producing crashes and no show stopping issues remain it gets pushed to the stable version. There is no mystery here, when we have a open beta running well as it can be with new features and content it is pushed to the stable version. Stable users also want to have new features and content as quickly as possible but dont want the increased potential for disruption. Agree or not this system works for us and we have no plans to change it. 

thank you

[Dangerzone]

On 10/15/2022 at 3:58 AM, Mike_Romeo said:

I would suggest to leave the stable version always at least 2 patches behind open beta but never on the same version number as the open beta one. This way you can say that the latest patches of the open beta are still in testing and so you cant put them into stable but when you regulary update the stable to open beta standard, it really doesnt appear more stable and I really dont see the benefit in choosing the stable over the beta.

 

Hey Mike_Romeo,

As a SR user - I find your proposal rather problematic, and if you'll bear with me - I'd like to try and politely express why. I've come up with an example using the release dates you've shown above. 

(However, this is an example only, not saying the below is what happened with these builds but just to try and paint a picture of a generic release experience)...

2.7.5.10869 - SR & OB on the same version (our starting point)

2.7.18.12852 -OB - New features added. Problems found with CTD (#A), new features need further refinement (#B)

2.7.6.13133 - OB - CTD Still occurring (#A)- new features refined

2.7.6.13436 - OB - CTD Fixed - new features working well

2.7.6.13436 - SR - We're happy with this version - release to stable 

2.7.7.14727 - OB - AI new features added, but feedback shows some AI won't engage (#B) and causes problems with some campaigns (#C)

2.7.7.15038 - OB - Refined AI new features. #B and #C resolved. OB Testing shows that it's working well.

2.7.7.15038 - SR - Releasing updated features to Stable

2.7.8.16140 - OB - Adding new cloud features - CTD Problems (#D) & Performance issues (#E)

2.7.9.17830 - OB - Managed to fix CTD issues - but users still reporting performance issues with clouds at certain times (#E)

2.7.9.18080 - OB - Fixed cloud performance issues - users having a reasonable experience, although still some performance issue (#E)

2.7.9.18080 - SR - Release to Stable - performance issues considered 'acceptable' for now.

2.7.10.18996 - OB - New features added, new netcode done to reduce latency and rubber banding. Users report problems with new FM (#F).

2.7.10.19402 - OB - Tweaked new features. Applied fix for FM. Users report still having problems with FM & provided tracks to replicate.

2.7.10.19473 - OB - Further tweaked new features, and FM. User response seems to be at an acceptable level

2.7.10.19473 - SR - Release to Stable Release.

 

From an OB experience, you get to experience all the major problems (show stopppers) of #A, #B, #C, #D, #E and #F

From a SR experience, we get the last bit of #E only - as all the other issues were identified and resolved whilst in Open Beta and between Stable Releases. It's a smoother experience for us this way.

In contrast - using your example (if I understand you correctly)- if SR was to just trail by 2 weeks, but we'd still get hit with every release meaning that we would also suffer through #A, #B, #C, #D, the worst parts of #E, and #F. It also means we're updating our servers and clients more often, and having to check scripts more. Our benefit is gone, we get no gain, additional work, additional bugs, yet still have to wait longer for the new features. I'm not sure what benefit you were proposing for this type of release.

I do get your point in that SR still definitely get some/other bugs that get through too which can still be frustrating at times (but it is agile development after all, so that will always be the case). But I'm hoping as you can see in the example above - the idea isn't for it to be bugless - it's that SR misses most of the show-stopper bugs (that may be CTD, cause AI problems in campaigns or multiplayer servers, or a number of different this)... at the cost of having to wait a little longer for ED to identify the 'good' builds. (I say 'most' as from time to time an SR release gets released before I think it's ready for Stable - but that is obviously an objective opinion and YMMV as will ED's). 

Also keep in mind that OB isn't the only builds released. There are closed beta testers as well, and then higher up internal dev builds behind that. CB wouldn't see all the builds Dev do, and OB wouldn't see all the builds that CB tests, just as SR doesn't see all the builds OB does. The extra builds inbetween are weaning out all the problems so each 'step' gets progressively more stable. The idea I believe is to get the builds to the most stable part before 'moving on' to the next big changes.

So, from a SR users perspective - I believe ED has this right. Stable releases may not be released "as stable" as I or you may like them to be - but there's a balance between releasing features as quickly as possible vs trying to give stability and dodge 'show stoppers'. OB is where the people happy to be Guinea pigs get to experience all the problems, at the benefit of having new features earlier, and it seems most people are happier to put up with the extra problems to have newer features earlier. (I get that, I admit, there are rare times I get impatient & have switched over to OB because I just have to be able to try things 'now' - aka when the AH64 was released )

Now the fact that OB and SR have been on the same build for a number of days most likely simply means that the next build that the Dev's or CB team have is not considered 'good enough' for OB release yet, and to save you additional/extra pain, they only release to Open Beta when they believe it's now at an 'Open Beta' standard. To me (at the risk of speaking out on OB users behalf) this is a good thing because considering the problems I see already reported - this would be compounded even more if they released too early. 

If I'm misunderstanding and have misrepresented your proposal I apologies in advance, and more than willing to be corrected.

What originally confused me was that ED would actually recommend OB over Stable to anyone (MP or SP)- except those specifically wanting to be Open Beta testers - because those users will experience more of the show stopper bugs (just as BN expresses above). Especially as someone who has hosted servers and done scripting this seems counter productive- because it's hard enough to keep up with changes (and deal with the bugs that slip through - I don't have the desire to add to that. (Maybe because I do the same thing with my day job too)  )  

However with his response to this thread - I see now that ED have identified most people have a bigger problem with 'not as many servers around' or 'want the new features now' as higher issues than having a more stable experience, and as such are now recommending multi players go to OB. Personally - I'm not convinced I would have worded it that way, but think it may have been better to advise that more MP servers are currently available on OB, as 'recommending' a BETA product seems to invite user complaints about quality- but that's ED's choice and I respect that they most likely know their user base better than I. Thankfully they have also given us a choice as well between two release types, which I'm very grateful for.

I hope this helps to clear up things from a Stable Release users perspective. 

Edited October 16, 2022 by Dangerzone