Benchmarking DCS

[djacd]

So I created a track file and benchmark 'procedure' to yield results that can be compared against each other.

I also created a YouTube video to demonstrate how I use it.

https://youtu.be/6y5n0uewpsE

The Track file and my bench mark results are all captured in 4K. The settings I used for all the benchmark runs are;

This is a list of benchmarks run using the above setting and the track file in case anyone wants to compare results. Going to be curious when Vulkan arrives.

 

Benchmark Run.trk

[derneuemann]

Can this also be done for VR?

Edited March 10, 2023 by derneuemann

[djacd]

The programs, CapFrameX and RTTS do both work in VR - I don't see the overlay in VR but the benchmark data is collected.

Also track files do NOT control your view in VR, so you need to fly it yourself.

[jawzzy]

Thank you for the track and benchmarking procedure.  I tried your track but the plane crashed as soon as you selected the first weapons.It also didn't use your views in replay. It just sat inn the cockpit view.

 Maybe we needs a track made with the latest version of the game.

I ran it anyway up to that point, butt I used 1440p resolution as it's my native resolution and the monitor can't do 4k.

The faster one is with the multithreading exe.

 

[djacd]

Yes, the track file is broken with this patch. (sigh)

Two issues with I see;

#1 after launching the Kh-58U's at the SA-6 units they fail to loft and/ or track and just fly into the ground. - they are broken as well

#2 the AI fly's the Su-25T into the ground right after launching the Kh-58U.. 

I've re flown the mission with 4 Kh-25MPU and use them for the SEAD on the SA-6's but haven't uploaded the track yet. (need to record the views part) I'm curious to see the how it handles the MASSIVE carpet bombing on Nalchik. In ST mode pre patch the FPS dropped to 14 I believe - HEAVY GPU and CPU load.

Will upload it shortly, I will also put my monitor in 1440p and run it as well to compare. 

[jawzzy]

I tried to record a different track, with the new version, with Mirage F1 in a pretty heavy CAP+ Strike mission. That one was messed up too. In replay it all goes wrong and the plane just crashes very early.

 I think .trk files are not capturing enough info to ensure repeatability, in replay. In Il-2 we used .NTRK files and those save more info and are more reliable.

 I wish DCS team would make a benchmark mission, that everyone can try.

 Something like that "Black death" in the original Il-2.

Edited March 13, 2023 by jawzzy

[Thinder]

8 minutes ago, jawzzy said:

I wish DCS team would make a benchmark mission, that everyone can try.

 Something like that "Black death" in the original Il-2.

100%, we need something that would actually work for us and dispense with the extra apps...

[Art-J]

^ Same applies to DCS - multiplayer tracks contain more data and are kinda-sorta more reliable than singleplayer ones, which are meant to be compatible only with game version number they were recorded on, but often fail even at that.

Edited March 13, 2023 by Art-J

[M-Flux]

Is there a way to run and replay a .trk file from the command line (and maybe even install DCS World from a CLI?) ? If so then it could be useful to add DCS World to the Phoronix Test Suite (https://github.com/phoronix-test-suite/phoronix-test-suite), which would then show up on https://openbenchmarking.org/. Would be way easier to compare results with the latter, which provides nice graphs, than trying to compare forum posts here. Would also be very interesting and unbiased to check FPS gains (or losses) with the new MT support through PTS, for instance.

(And would be free advertisment for DCS, if it showed up on PTS + Openbenchmarking.org.)

[djacd]

So I re did the track files and saved them into 3 formats.

One for 1080p, one for 1440p and one for 2160p (4k)

as well another video about them 

https://youtu.be/OC8WS8eiHHM

Ran all three on different computers so should work with these settings.

 

Benchmark Run 1080p.trk Benchmark Run 1440p.trk Benchmark Run 2160p.trk

[veggy432]

hi everyone,

sorry for the noob question, I couldn't find the controls to record track files during gameplay, I would like to make my own trackfile for benchmarking in VR and I want to do in one of the campaigns I'm playing (as there are plenty of units and stuff that loads up) to get a more realistic result; does anyone know how to start and stop a track file? I know that the game starts recording at mission start, but I'd like to skip all the initial intro and taxing phase and concentrate on a single part of the mission to record and use for benchmarking after (I'm using msi afterburner with rivatuner, it works very nicely)

thank you!

[Paganus]

DCS track files may not be the best thing for benchmarking anything other than how DCS track files run.

 

[Bucic]

Is there a way to add MT and non-MT DCS.exe separately in CapFrameX?

[SharpeXB]

Using a track for a benchmark might seem like a good idea but that feature has been broken for years. So since you won’t get reliable replays, there’s no point in doing this. 

[Bucic]

7 hours ago, SharpeXB said:

Using a track for a benchmark might seem like a good idea but that feature has been broken for years. So since you won’t get reliable replays, there’s no point in doing this. 

That's not what I meant. Right now there is no way for me to tell which benchmarks in the CPX capture list were ran using ST and which ones were run using MT version of DCS.

Even manually viewing the JSON files I can tell they don't contain the executable file path.

[SharpeXB]

36 minutes ago, Bucic said:

That's not what I meant. Right now there is no way for me to tell which benchmarks in the CPX capture list were ran using ST and which ones were run using MT version of DCS.

Even manually viewing the JSON files I can tell they don't contain the executable file path.

Well but even if you could, the tracks won’t repeat accurately. 

Edited July 12, 2023 by SharpeXB

[Bucic]

13 minutes ago, SharpeXB said:

Well but even if you could, the tracks won’t repeat accurately. 

 

For my purposes, they don't have to.

[SharpeXB]

2 minutes ago, Bucic said:

For my purposes, they don't have to.

I would assume if you want accurate benchmarks all the replays of the track need to be identical. In DCS that’s pretty much a lost cause. 

[Bucic]

29 minutes ago, SharpeXB said:

I would assume if you want accurate benchmarks all the replays of the track need to be identical. In DCS that’s pretty much a lost cause. 

I can extract meaningful information from a 60 second part of a single graph in MSI Afterburner. Rest assured I can do a lot more with what CapFrameX offers. To put it simply, no, two benchmark sessions do not have to be identical to be meaningful, especially with statistical metrics.

As for CapFrameX  I have to rely on comments which is really cumbersome.