Bug Reports / Testing Process / Updates?

[Ranger79]

Questions for ED:

 

Is it possible for the community to get updates on bugs reports being worked like a bug tracker with a possible ECD? What is the Quality Control process for a release candidate (update) to the community? Who decides which bugs are priority and which are not? How does the process involving the ED Test team work? Do the programmers and leaders of the community understand how crippling some of the bugs introduced in the latest patches have effected the community and custom missions?

 

IMO custom missions have been the life blood of the game thus far, without new content we will be in for some serious problems. Most of the community give their time and free content to enhance the game.

 

Explaining to the community how the process works and giving us information would definitely help, we deserve that much as paying customers. I speak for many who are afraid to post due to fear of retribution, founded or unfounded.

 

Change isn't always easy, but something needs to change. I understand everything is subject to change, but we do not expect things that were working fine to become broken and remain broken for months.

[nickexists]

+1

[Flagrum]

As far as I understand it, ED has a bug tracking system that the testers use to file issues.

 

While I can understand that a company might not want everyone to see how much errors their products have, how long it takes them to be fixed, etc., I really support the idea of opening up the bug tracker database to, well, "the public"? "a broader audience"? I mean, that issues exists is no secret, we all know it already.

 

And a bit of "Glasnost" would help to put expectations in a more realistic context. Would people still bitch about bugs that are not yet fixed? I bet. Maybe their complains are reasonable ("minor bug" vs. "game breaking"?). Maybe they are not. But that is no different to what we have now - only that everyone involved would have the same base information.

 

I would not necessarily open the bug tracker to everyone to actually file new issues. That should still be limited to experinced testers. But read access would help a lot and would restore some confidence in the community. Today we can't even be really sure if ED actually knows of all bugs (see my sig ... lol). Yes, we are always told, the devs monitor the bug sub-forums ... but then, why can't issues just be acknowledged then? So everyone could move on?

 

I don't need to know who is working on what bug for how long and when it is due. But it would help if I knew, yes, ED knows about it and as it is considered a "minor" issue, and therefore I can expect it to be fixed in a couple of months - based on the experience we all will have gained some day with this new way of dealing with bugs.

Edited May 25, 2014 by Flagrum

[cichlidfan]

Considering how many times it has been brought up, and received a negative response (not from an ED employee, that I can recall). I sincerely doubt that any sort of access, even read only, to the list of existing issues is going to happen.

 

However, I would love to be proven incorrect.

Edited May 25, 2014 by cichlidfan

[Sabre-TLA]

They don't have to show all open bugs but at a minimum communicate the bugs that are currently being worked on and expected to be fixed in the next release.

 

That would at least give the community some acknowledgement that critical issues are being addressed (or not).

[cichlidfan]

If memory serves, one of the reasons given for not doing something like Sabre suggests, is to prevent the 'discussion' of why this bug and not that bug, in terms of what is/isn't being giving the proper level of attention.

 

I can see the validity behind that point of view but I am not sure that the bad outweighs the good in this case.

[Flagrum]

If memory serves, one of the reasons given for not doing something like Sabre suggests, is to prevent the 'discussion' of why this bug and not that bug, in terms of what is/isn't being giving the proper level of attention.

 

I can see the validity behind that point of view but I am not sure that the bad outweighs the good in this case.

I don't see where it is valid ...

The issues behind that still exists - just not talking about it does not solve anything.

 

Yes, perhaps such discussions could be "heated", but dealing (moderating) with that would be the better way imo - better than hoping everyone forgets about it ...

[Silver_Dragon]

I think some similar was the old "disappear" FC3 Bug-tracker:

http://forums.eagle.ru/showpost.php?p=1700249&postcount=1035

Or the old disappear Friday Wags updates:

http://forums.eagle.ru/showthread.php?t=110202

[Flagrum]

In a way, we already have something similar with the beta test sub-forums. The thing is just, that it is not maintained consequently. Which is partly understandable - it is difficult to keep track of all the postings and discussions going on there at times, trying to keep the real bug tracker in sync and all that by only a few unpaid volunteers.

 

Opening the real bug tracker would allow to get rid of such kind of administrative work, I would guess ...

[PFunk1606688187]

Yes, perhaps such discussions could be "heated", but dealing (moderating) with that would be the better way imo - better than hoping everyone forgets about it ...

 

Its not as if these forums are particularly under moderated as it is.

[Rider1]

+1 Seems like every patch usually screws up something that was previously working fine is my biggest frustration.

 

ED has had what, 6 years so since the release of the very first A10C beta to get the Bug/QA/Testing process to where it needs to be. I have a feeling that this is as good as it is going to get.

 

To me it feels like DCS is a side project to EDs commercial and military projects, to be worked on as man power and time permits.

 

Ranger, sorry about your missions. I can only imagine how much time and effort it took to create them. :( I enjoyed playing the Suse missions and serving them to our group. Most of the multiplayer community missions that I served up have been broken as well.

[Flagrum]

+1 Seems like every patch usually screws up something that was previously working fine is my biggest frustration.

There can always break something unexpectedly. But if that happens and is pointed out during the open beta, I would at least expect that those things are fixed first. It is easier to locate and probably fix things that broke yesterday - as the people involved are familar with the code they just worked on - than fixing things in the code that someone else had changed half a year ago ...

 

Yes, there are other factors that play a role in there. Maybe fixing it is pointless because the code will be obsolete with the next patch. That was probably the case with the issue in 1.2.7 where no A-10C controller settings could be saved if Thrustmaster MFDs where connected ... that code has probably completely changed since then for the new .diff.lua config files.

 

This is just annoying and simply telling us (or letting us have a peek into the bug database) would probably have saved some of us quite some nerves. This is the point where the "0 info" policy is really getting annoying ...

Edited May 26, 2014 by Flagrum

[159th_Viper]

This is just annoying and simply telling us (or letting us have a peek into the bug database) would probably have saved some of us quite some nerves. This is the point where the "0 info" policy is really getting annoying ...

 

Probably best just to wait it out at this stage of proceedings.

 

EDGE is being integrated into the entire code-base and until that is complete and there exists a single, unified platform with which to move forward with any attempt at contemplating any type of 'public' bug-reporting system is not even worth contemplating.

 

In any event, sitting on this side of the fence (tester) I am firmly of the opinion that any type of public reporting system will be a disaster. Forum reports, as currently in use, are just about the best for this (ED's) particular set of circumstances.

[winz]

There can always break something unexpectedly. But if that happens and is pointed out during the open beta, I would at least expect that those things are fixed first. It is easier to locate and probably fix things that broke yesterday - as the people involved are familar with the code they just worked on - than fixing things in the code that someone else had changed half a year ago ...

You assume that code needed to fix is the same that was modified during implementation of new things. It's usually the other way arround, the things that break by new code are most of the time totally different parts of code, where no-one ever thought that the new functionality would affect them. ;)

[Flagrum]

You assume that code needed to fix is the same that was modified during implementation of new things. It's usually the other way arround, the things that break by new code are most of the time totally different parts of code, where no-one ever thought that the new functionality would affect them. ;)

I know. But chances are high that some side effect of the last change had something to do with it. Better to dig from there deeper than to start all over some weeks later.

[sorcer3r]

EDGE is being integrated into the entire code-base and until that is complete and there exists a single, unified platform with which to move forward with any attempt ....

 

 

I read something similar before DCS world... Anyway perhaps this time it's different. ;)

[159th_Viper]

I read something similar before DCS world.....

 

That's the inherent danger of basing personal assumptions on insufficient information ;)

[Winfield_Gold]

In any event, sitting on this side of the fence (tester) I am firmly of the opinion that any type of public reporting system will be a disaster. Forum reports, as currently in use, are just about the best for this (ED's) particular set of circumstances.

 

I am curious, as a "tester" does this mean you guys only get to test what is not yet released to the public or do you actually have a hand in the coding?

[159th_Viper]

I am curious, as a "tester" does this mean you guys only get to test what is not yet released to the public or do you actually have a hand in the coding?

 

Yes we test what is not released and generally no, we are not hands-on with coding. There are however, as always, exeptions.

[Rider1]

For example, how do things like the RWR lights being broken on the SU25 a few patches ago get past testing and QA? If anyone in QA would have jumped in the Rook and taken a test flight it would be immediately obvious that they were not working.. Having no idea where the threats that were locking you up was a pretty big deal for us that fly it exclusively.. Why was it released in this broken state and why did it take 2 patches before it was fixed?

 

The ECM lights and IR jammer lights on the 25 T have been broken since the 6dof cockpit release was well. You would think that a free jet to entice customers into the DCS World would be a top priority..

[Snoopy]

For example, how do things like the RWR lights being broken on the SU25 a few patches ago get past testing and QA? If anyone in QA would have jumped in the Rook and taken a test flight it would be immediately obvious that they were not working.. Having no idea where the threats that were locking you up was a pretty big deal for us that fly it exclusively.. Why was it released in this broken state and why did it take 2 patches before it was fixed?

 

The ECM lights and IR jammer lights on the 25 T have been broken since the 6dof cockpit release was well. You would think that a free jet to entice customers into the DCS World would be a top priority..

 

Just because it doesn't get fixed doesn't mean testers didn't identify the issue. Prior to leaving the testers team I identified many issues that have not been fixed yet.

[Rider1]

Redacted

Edited May 26, 2014 by Rider1

[NineLine]

For example, how do things like the RWR lights being broken on the SU25 a few patches ago get past testing and QA? If anyone in QA would have jumped in the Rook and taken a test flight it would be immediately obvious that they were not working.. Having no idea where the threats that were locking you up was a pretty big deal for us that fly it exclusively.. Why was it released in this broken state and why did it take 2 patches before it was fixed?

 

The ECM lights and IR jammer lights on the 25 T have been broken since the 6dof cockpit release was well. You would think that a free jet to entice customers into the DCS World would be a top priority..

 

Just to add to what Paul said, there are a number of factors in play here as well, their are available resources to hunt down and find what caused the bug, as well as what to do to actually fix it. Its not always going to be something easy, it could be a deeper rooted change that you cant just revert to fix. This is all balanced in with what they are currently working on. It will end up being like triage, trying to hit the bugs that are more damaging first and the less damaging bugs might fall farther back on the priority list.

 

If the person that would fix the ECM light is buried in other projects right now, a fix might have to wait... we have to remember that ED is a smaller team than some of the giant companies, so turn around time on some bugs may be longer than we want.

[NineLine]

Thanks Paul. It must be extremely frustrating for the testers who test the crap out of this stuff and not have ED worry whether things get released broken or not. :(

 

This is an un-fair statement, I cant speak for every tester, but at no time do I feel that ED doesnt worry about bugs that come up with new updates. I think they do the very best they can with the resources they are afforded. If you dont think the devs take pride in what they are doing you are very wrong.

[chardly38]

For example, how do things like the RWR lights being broken on the SU25 a few patches ago get past testing and QA? If anyone in QA would have jumped in the Rook and taken a test flight it would be immediately obvious that they were not working.. Having no idea where the threats that were locking you up was a pretty big deal for us that fly it exclusively.. Why was it released in this broken state and why did it take 2 patches before it was fixed?

 

The ECM lights and IR jammer lights on the 25 T have been broken since the 6dof cockpit release was well. You would think that a free jet to entice customers into the DCS World would be a top priority..

 

Look at ATC. They have sent it out to us not working correctly. That is that is the way they do it. What would be nice is something that would say ( This.......is not working 100% because we are........) But lets hope that this limited resourced company can deal with having two sim engines to work on.