Quality

[FlightControl]

Don't know where else to put this. Please read, let it sink in a bit...

 

In the light of 1.5.7.

 

Would it be an idea to actively involve a selected bunch of community members before such an important update is done. Patches, ok... But updates should be better Q&Ad involving the community.

 

The tools are there, the process can be improved. Beta is an important asset.

But who can do a test run every 2 weeks, and still survive their marriage?

To do a complete test I need at least 4 hours. 2 for single player, and two for multi player in a client/server setup. The devil is always in the details.

Guys, I am not talking modules here, but scripting engine. And try to keep motivated that bugs may be introduced, and are likely not to be solved for months, if not years.

 

People have missions that cover 99% of the DCS api,

and friends and squadrons can help to test the stability of the modules and do Q&A of the functionality and graphical consistency.

 

Don't want to reinvent the wheel here or make people upset.

Just suggesting there are people willing and able to help with Q&A.

 

Communication can be improved, expectations can be better aligned, activities can be synced.

Where are the lines in the release announcements from Chiz and Rik, that indicate something about scripting functions being added? And that the menu and user interface was adapted? Maybe I missed it ... There were lots of core changes done.

I am sure in this release a lot of improvements were made, and it is a pity that the hard work from the development team from ED and the testing team has resulted in this state.

 

Example, today we are discovering markpoints... A great addition, the api has been adopted and if it wasn't Grimes who takes the time to document all that, we would not know about it.

 

Pls don't shoot the messenger.

 

Sven

[Drexx]

The main mission I work on 'dynamic caucasus' will test almost every asset in the game (allow WWII assets to be seen by people who don't own those assets and I will make it use them all, my mission places units by randomization templates), we work on this stuff everyday (what can I say, we love the game), we want to make it better, make it more amazing than 1 off missions, let us help you, give us some sort of reliable 2 way communication channel! (we will keep it professional, I promise, most of us are professional software developers anyway)

 

-afinegan aka Drex

[WinterH]

Did we not see the very same thread a year ago, or was it a couple years? My memory might be playing tricks but, I seem to remember it was your thread again was it not? :)

 

I don't think involving "the community" aside from ED's own beta testing team is that great of an idea. "Involving the masses" isn't as great as it sounds with many things in my opinion :).

 

That said, for specifically the script engine, I can wholeheartedly agree.

 

Mission scripting can get the DCS out of its conundrum of sterile feeling "missions". But it seems to get broken so often, that can hardly become a mainstream thing. So if the scripting engine can be made to shine with aid of people working on it continously, I'd say do it.

 

If the scripting engine was more robust / stable between the DCS updates, I'm sure we'd be seeing many payware dynamic campaigns, or at least missions with some dynamism in them. I wanted to do one for helicopters myself, but things worked in one version was getting broken in other and eventually gave up, as I was having a busy time with IRL stuff too. MBot does excellent stuff, but his campaigns seem to get broken by updates often too. Same with scripts like CTLD, which add functions core sim lack and make utility helos enjoyable in their intented role, but even that doesn't seem to be immune to getting broken with updates.

 

Then there's the multiplayer side of things, if complex scripts could work there in a stable fashion, and more people could try that easily, the majority of DCS online wouldn't be airquake / teamdeathmatch with planes anymore.

 

So yeah, making the scripting engine great, accessible, and stable, is a noble cause indeed.

 

But I'm reeeeaaaaaly not sure of the whole QA process going through "community representatives", see it already sounds like we are talking politics :P...

[FlightControl]

History repeats itself. No politics involved, why waste time on that? My god! No.

We wanna get things working and help ED. Seriously.

Do you know how much time we are spending in creating assets for the dcs community?

[probad]

Do you know how much time we are spending in creating assets for the dcs community?

that's your own voluntary choice, not something to use as leverage to advance your interests.

 

you want to be in the loop and get compensated for your time, go apply for a position.

[FlightControl]

that's your own voluntary choice, not something to use as leverage to advance your interests.

 

you want to be in the loop and get compensated for your time, go apply for a position.

 

Please don't pull things out of context.

[Pikey]

Since the ED team are ultimately 'closed door' with their strategy, I cannot verify much of this but it's peiced together over time. As I understand, builds undergo testing from the wider community "ED tester team" who are unpaid volunteers with fringe benefits, and the builds are available to that team with perhaps an unknown (but limited?) level of explanation and change logs. That process happens before release, it seems to be an ongoing and fluid cadence. There's a possibility they are asked to test things in particular when items arise of importance or key changes, but I couldnt say for sure. Would seem sensible.

 

The Beta builds are usually brought around a week beforehand and beta seems to only be published if there are significant changes which seem to add a layer of additional crowd checking to the masses. There's definitely not a beta every release build.

 

In terms of adding to the tester team i've only ever known two people invited since I first joined the forums and it's kept small. ED has a robust policy surrounding what access people have and I believe there's an intimidating NDA legal doc and closed door 'invite only' access.

 

Quite why anyone would want to commit to testing DCS: World in any formal sense is completely beyond my understanding. To view the application in an even less stable state would sap the remaining enjoyment from the most ardent fan. I've no doubt you could provide a lot of feedback directly to developers, but simply put we would never see you again, it wouldn't change DCS direction, you wouldnt be able to discuss issues any sooner, you would have divided loyalties and no freedom to act. It's not something I would ever wish for and is a burden reserved for the unpaid team who I am sure try hard for our sake.

 

As far as the "hidden change log" goes, I couldn't agree more. I don't think there would be any harm in spelling out the list of detailed changes under the hood for the community to prepare for, rather than to guess our way blindly with Black Box troubleshooting. But that's how they do it, and until provisioning for servers really gets any attention it's the community that pays the price for things going wrong that no one expected. Sure, we get tidbits thrown in from folks like Grimes. They don't substitute a proper change control that we can react to or focus testing on.

 

Why? Because they want it this way. I long ago stopped any fantasy breeding in my mind about changing Eagle Dynamics, the clue is the course of history that has gone on over the long years we've been a part of the community. The pattern is clear. Nothing else to say.

 

Do not wish for this. Change the things you can change, leave the fights you cannot win.

[David OC]

Projects like MOOSE is a real key in many future campaign missions or standard scripted missions if you just want to make the mission less sterile with very little work.

 

These API's could lead to Dynamic Campaign one day perhaps?

 

I can understand why the guy's that put in the time to move this forward would like a little heads up in the direction and potential changes with all the work that's going on in trying to help improve the user experience in DCS.

 

 

.

[FlightControl]

The issue is clear:

 

1. A beta test stream exists. End users can contribute, not replace quality assurance activities.

2. The scripting engine makes or fails DCS depending on what works or not. The devil is in the details.

3. A subset of end users, called mission designers can verify the scripting engine.

4. It takes a lot of effort to verify a beta and scripting engine. With a large update more time is needed.

5. Clearly there was no thorough test on the last update 1.5.7 by those who are responsible, end users in beta and mission designers in beta. (I am also in fault).

6. Communication between ED and the community could be better. The better the change log, the better the Q&A.

 

I get the feeling that this thread is a waste of time. Wrong audience maybe.

Each update can make or fail DCS. It is impossible for the ED team to test all.

 

So the suggestions are:

 

1. ED tries to add info on what has changed, also in core and engine. Not just modules and graphical visible things.

2. Beta is important. We use it.

3. On big updates, this is told to the community. So we are aware.

4. Community checks and provides feedback to ED team. Go/nogo. The last one is important for ED.

 

 

 

Sven

[Home Fries]

This may be a cross-post (cross-quote?), but I feel that what I originally said is better illustrated and expanded upon here (emphasis added).

 

<...>some principles of schedule and continuity can be adopted once 2.5 is live and major bugs squashed.

 

What I mean is that we love getting improvements<...>. However, the health of the DCS community relies heavily on external mission builders, and even modders to a significant degree. Right now, incremental updates in DCS, while welcome, tend to break earlier missions to the point that are most prolific mission designers are playing it safe with lua code so that they won't have to redo the mission in a short timeframe. Additionally, some of the best missions over the past few years break if you save them with the latest version of DCS (what was once a time-honored practice of ensuring compatibilty). As a modder, I have put all of my mods on hold because I don't have the time to test compatibility with every new build.

 

Once 2.5 is stable, what I recommend for ED is in line with the Agile process: Have a plan for incremental updates, post those plans, and post what datafiles/systems will be modified. For example, if I know that some nations are being added to the ME in a month, I can plan to update and test my Nations mod with respect to the new lua files. Likewise, if lighting is to change in 2 months, I know that I may need to update specular textures around that timeframe. Similarly, if features are added/modified in the ME, advance notice gives mission builders an idea of what to expect.

 

A solution like this allows innovation to thrive while encouraging and supporting the community in making DCS better than the sum of its parts.

[Pikey]

I get the feeling that this thread is a waste of time. Wrong audience maybe.

Each update can make or fail DCS. It is impossible for the ED team to test all.

 

So the suggestions are:

...

They do read these posts. They just don't reply to posts where they have to defend their position on things.

They provide a beta stream, did provide it to the EU to test, they provide Beta when big changes occur but not all the time. The non public builds are provided to the community testers prior to public release. Other than that, the suggustion on more detailed patch notes is very valid.

If you are saying there should be more than that, ask away.

[Buzzles]

The scripting engine breaking again sounds like there's a lack of unit tests on it, or if there's any then they're not comprehensive enough. It's also not something the community can do and shouldn't be something they need to test specifically for.

 

That's fixable, just a bunch of time and effort.

 

I've personally always found it a hard sell within businesses where the feature is king and clients want the next thing. ED are their own client however, so the only issue will probably be resources.

 

That said, major changes to the scripting engine should be documented for the mission builders to know about.

[David OC]

I get the frustration here, for both sides! I bet ED's team is going through a lot of coding and changes leading up to 2.5, I'm really hoping things can settle down to some extent and the focus and direction will be lock down somewhat for all moving forward here.

 

Perhaps finish up here in the public forum guy's by contacting SiThSpAwN by PM about the DCS API concerns and some of the strategies you have moving forward, so SiTh can discuss this with Wags.

 

 

 

I get the feeling that this thread is a waste of time. Wrong audience maybe.

Each update can make or fail DCS. It is impossible for the ED team to test all.

 

So the suggestions are:

 

1. ED tries to add info on what has changed, also in core and engine. Not just modules and graphical visible things.

2. Beta is important. We use it.

3. On big updates, this is told to the community. So we are aware.

4. Community checks and provides feedback to ED team. Go/nogo. The last one is important for ED.

 

Sven

 

.

Edited July 19, 2017 by David OC

[tintifaxl]

Automated testing is what DCS needs. Unit and integration tests that run after every build. Sadly, as always, creating and maintaining such a testing environment is costly.