How pro's report performance issues...

[Yeti42]

WARNING - This is not a paper on performance testing, to find this useful you need to spend a lot of time setting up stuff and some time basic scripting. There are multiple tools you can use, I mention some, Google is your friend ;)

 

Caveat. I DON'T work for ED, Do as I say not as I do ;) I haven't got the time to setup a proper performance and capacity test... or I can't be bothered spending 4 hours doing this after I come back from work. For those who do have the time and can be bothered, this is typically how you quantify a performance problem and help dev fix them:

 

DON'T just say "my performance is worse now than it was", this means nothing without data and a baseline

DON'T say "I tweaked some stuff, upgraded some drivers, did a clean install... and put a finger in the air whilst standing on my left foot and performance is fine on my system now" ;)

 

Do debug the issue if you want to. Make sure you can demonstrate what you changed (removing AI, triggers etc.) and also run that track/mission against the release you are using for the baseline.

DO understand that very high FPS on their own mean very little, It's smoothness and acceptable frame times that mean more. 60 FPS with small 1% low variations and an even frame time history means no stutters and smooth gameplay...

 

Basics:

1. When measuring performance, you must have a baseline to compare it with. This could be a previous release or the stable version of DCS

 

2. You must be comparing apples with apples - This means you must use a mission or track replay with pre-defined graphics settings including screen resolution that demonstrate the problem

 

3. It helps if you can run DCS with the track or mission automatically and create the data (perf logs etc) automatically. - There are ways to do this... you can create a track file and basic scripts (batch or any other scripting language) that start DCS with a track or mission run it for a set duration and export performance info to a log (riva tuner RTSS etc.) You may need to use some sort of GUI tool to click on "start", I've used AutoHotkey in the past.

https://www.autohotkey.com/

 

4. Know what to measure and log - Generally for hardware: RAM usage, Individual CPU core usage (NOT the generic all cores % as this really is useless in this sim as it's not truly multi threaded), Disc usage, GPU usage.

 

For client in-game performance: Frametime, framerate, 1% low, average fps.

 

FPS and average FPS can be misleading, in the case of stutters for instance, a graphical representation of frame time history will visualize any stuttering you have, also 1% low should uncover stutters.

 

5. Reporting - If you graph and overlay your test run metrics in excel for instance, it's very apparent where the problems are from release to release, this with a timeline that can be viewed against a track file is very useful for development.

 

If you do all this, you'll have a framework that you can performance test all releases with and send valuable feedback to ed, also you can send me the scripts so I don't have to.... ;)

Edited April 20, 2020 by Yeti42