VR Benchmarks (initial tests)

[Qcumber]

Interesting video on benchmarking in DCS

 

 

Edited April 10 by Qcumber

[Qcumber]

I have followed the methods used in the above video and have compared several tracks in different situations.

Edit: This has been updated so that I can share more data. See below. 

 

 

 

 

 

 

 

 

 

 

Edited March 19 by Qcumber

[Qcumber]

I have two tracks of a 109 flying over the desert for about 1 min. 

I have compared a range of settings to see what impact this will have on performance and GPU latencies. 

This is using a Quest Pro with QVFR unless otherwise stated. 72Hz 

The base resolution coming from quest link is set to x1 (1808x1856 per eye). This is then upscaled using QVFR (or OTT when this is disabled). 

Equivalent full resolutions across the whole frame.
Base 2.2 1.6 1 0.8 0.5 Hor 1808 3978 2893 1808 1446 904 Vert 1856 4083 2970 1856 1485 928

The foveated region is set at x2.2 and is 0.2x0.2 in size

The data show the frequency at which the latencies fall into 0.1 ms bins expressed as percentage of a total number of events. The vertical red dotted line shows the 13.89 ms level which equates to 72 FPS. If a latency goes above this then you will see stutters and missed frames.  

Comparison of no AA, MSAA and DLSS

All settings are foveated 2.2 at 0.2x0.2 and the periphery at 0.8 

Interestingly MSSA has less impact on the latency than DLSS quality. I think this is because it is only having to work hard on the foveated region (!?). The DLAA latencies were more variable for some reason. 

This compares the tract with no QVFR and with QVFR with a foveated region of 2.2 with different settings for the peripheral resolution. As expected there is an increase in latency when the peripheral resolution is increased. 

I have also done some analysis of a more demanding track flying low over Cairo and can post the results if anyone is interested. 

 

 

 

Edited March 17 by Qcumber

[sleighzy]

Unrelated to your bench marks but if you drop your Preload Radius from 150000 (which is really high) to about 60000 you'll save yourself a bunch of RAM and give you more headroom for both RAM and VRAM (stops paging to disk when you start hitting limits). Will also speed up load times as you're not loading a whole bunch of potentially redundant stuff into memory.

[Qcumber]

Thanks. I'll test that and see if it makes a difference. 

 

[Qcumber]

F-16 over Syria.trk

[Qcumber]

There should have been some text to support this. I'm not sure why it is not here. 

The results suggest that, whilst a 150k preload increases VRAM use by about 1Gb, There is a 1 ms improvement in latencies in less demanding scenarios. 

The track I used was the instant action F-16 free flight over Syria. 

F-16 over Syria.trk

The first part is mid altitude over rural terrain ( the lower latency part of the chart 7-9 ms). The second part is low level over Tripoli (the longer latency part, 11-13ms). 

With the 150k preload, the VRAM increased by about 1Gb, but the low latency part of the chart improved by about 1 ms. I am not sure if this has any meaningful effects in game terms. I have been testing other DCS parameters which I hope to post soon. 

[sleighzy]

4 hours ago, Qcumber said:

There should have been some text to support this. I'm not sure why it is not here. 

The results suggest that, whilst a 150k preload increases VRAM use by about 1Gb, There is a 1 ms improvement in latencies in less demanding scenarios. 

The track I used was the instant action F-16 free flight over Syria. 

F-16 over Syria.trk 1.12 MB · 0 downloads

The first part is mid altitude over rural terrain ( the lower latency part of the chart 7-9 ms). The second part is low level over Tripoli (the longer latency part, 11-13ms). 

With the 150k preload, the VRAM increased by about 1Gb, but the low latency part of the chart improved by about 1 ms. I am not sure if this has any meaningful effects in game terms. I have been testing other DCS parameters which I hope to post soon. 

Yeah that 1Gb saving is nice. For smaller spec'd machines we even advise users to drop it to 30000, which would mean even greater savings. For folk on 8GB cards (or even more) that's a decent saving and hopefully performance increase with less need to page in and out of RAM. I agree that the 1ms is likely not an issue, and folk would be probably be vastly happier to get back 1+Gbs of VRAM vs. a 1ms saving somewhere (doubtful if noticed).

Thanks heaps for the testing and posting of your results @Qcumber , was good to see some actual physical numbers around this and less feels.

Edited March 19 by sleighzy

[Qcumber]

Some more results.

This is using the F-16 track from above. I started off comparing three setups in DCS: low, medium and high (screenshots below). 

QVFR

smoothen_focus_view_edges=0.15
sharpen_focus_view=0.9
horizontal_focus_section=0.25
vertical_focus_section=0.25
peripheral_multiplier=0.8
focus_multiplier=2.2
 

QP at 72Hz, link cable 960mbps. 

Its not a surprise that the high settings really push my setup. The medium settings are close to what I would normally run.

What interested me about this is that the medium and high settings produced a biphasic chart, corresponding to the flight over rural terrain (low latency) followed by the flight low over urban terrain (longer latency). The long latency component was not there for the low settings. So I spent some time testing various combinations of settings and found that the main cause of this longer latency part were visibility range. These are for the next post. 

 

 

 

Edited March 19 by Qcumber

[Qcumber]

 

The only difference in these groups is the visibility range. "Medium" versus "Ultra". You can see that the low latency part of the chart has gone, so I think the Visibility Range is the major factor when flying over dense urban areas. 

Edited March 19 by Qcumber
Text was lost

[Qcumber]

"Low" terrain textures looks quite bad so I compared this with "High" This is the only difference between these two groups so have opted to use "High".

There is very little difference in terms of latency but a 1.2 Gb increase in VRAM.

Edited March 19 by Qcumber

[Qcumber]

In the next run I tried maxing out all the details sliders. T8 has these sliders set mid range. T9 are all maxed out. 

You can see that the "Flight over Tripoli" part of the track is causing an increase in the latency which is as expected but overall its a small amount and can probably be controlled with backing off on some of the details sliders and not others. I am not sure why the VRAM is showing as lower. I think this is some error in the recording. I will try to repeat it.  

[Qcumber]

The final results and my current settings. Very similar to the "Low" settings and better than my old settings

 

 

Comparing the F-16 flight over Syria with F-4 take off from Syria and Marianas

Some dropped frames on Marianas but overall a playable experience. The F-4 take off in Syria has a biphasic component which relates to longer latencies on the runway and much lower latencies once airborne. 

Edited March 19 by Qcumber

[Rolln]

And you're not getting any ghosting on other aircraft when using DLSS?

[Qcumber]

6 hours ago, Rolln said:

And you're not getting any ghosting on other aircraft when using DLSS?

Ghosting is much better than it used to be with DLSS 4 and preset K but it is still there. It's noticeable against the ground and in some situations against clouds. I find it is very usable compared to how it was. I'm still on the fence about wether I should use DLSS or MSAA. 

I noticed it was better going from a 4070 to a 5070ti. I also think that increasing the resolution as high as possible helps too. I'm speculating here but I do think that maintaining consistent GPU frametimes helps too. 

How does it compare to MSAA? Well, MSAA is still a sharper image and no ghosting but comes with "jaggies" and "shimmering". DLAA/DLSS is much better in this regard.

If you look at one of my comparisons above DLSS does come with a significant performance hit (about 1.5 ms Vs MSAAx4). It's even worse with DLAA (+2.5 ms).

I need to do some more benchmarking on this, particularly in challenging missions. I am planning to use "xrframetools" so that I can capture every frame. I think this is important in more intensive GPU loads so that there is a more accurate reporting of missed frames etc. I don't think OXRTK does this very well as it only reports and average of about 10 frames. 

[Flyingfish]

Apologies for the slightly off topic question but I see you've upgraded to a 5080 and you're using Meta link. Are you having any problems using Meta link and the new GPU? The 50 series GPUs aren't officially supported yet by Meta so I wasn't sure if they'd actually work 

[Qcumber]

11 minutes ago, Flyingfish said:

Apologies for the slightly off topic question but I see you've upgraded to a 5080 and you're using Meta link. Are you having any problems using Meta link and the new GPU? The 50 series GPUs aren't officially supported yet by Meta so I wasn't sure if they'd actually work 

Meta link has worked fine for me with the 5070ti and the 5080 with the latest NVIDIA drivers. The meta link desktop app does not recognize the cards but just gives the message "your system does not meet the requirements". It still runs though. 

[Qcumber]

I have made a new post with my methods and spreadsheet.

 

[scommander2]

Hi @Qcumber, if it is possible that can we try F-4 in Afgh map from the instant action?  That action has lots of ground objects.  Thanks!!

[Qcumber]

1 hour ago, scommander2 said:

Hi @Qcumber, if it is possible that can we try F-4 in Afgh map from the instant action?  That action has lots of ground objects.  Thanks!!

I'll give it a go. 

[Qcumber]

OK. The F-4 take off is painful. Its worse than a take off on the Marianas map! What the hell is going on. It does not seem to be very well optimised in terms of objects etc.

Green is F-4 free flight. Blue is F-4 take off (Afghanistan map instant actions).

Not all of the GPU render times are being captured so there is something odd here. On the ground both CPU and GPU render times are all over the place. No consistency at all. Once airborne and high enough FPS stabilises.

The free flight is a dream performance. Solid FPS and low render times. 

 

 

Edited April 21 by Qcumber

[Qcumber]

On the suggestion of a Discord member (Orion_42) I have tried this test again without using QVFR and by changing some of the settings. 

The charts below are  3245x3302 pixels set using OTT. The blue trace (T2) is with visibility range set to Ultra and all the scenery slider set to max. The red trace (T3) with visibility range set to Ultra and all the scenery slider set to lower settings as per the screenshots below. 

There is only a marginal difference and there is much less stutter and random frametimes than I saw in the previous post using QVFR. 

To be sure, I retested QVFR using the same settings as above. 

smoothen_focus_view_edges=0.1
sharpen_focus_view=0.9
horizontal_focus_section=0.4
vertical_focus_section=0.4
peripheral_multiplier=0.5
focus_multiplier=1.1

The dark purple line is with max visibility settings. The pale purple line is minimum visibility settings. The black dotted line is the "blue" curve from above for reference. 
 

Overall these results are much better than yesterday and more stable. Interestingly QVFR with these settings has very little benefit in terms of GPU render time (all three situations were around 13 ms. CPU render times were 8-9 ms. Some stutter but still comfortable. 

 

Edited April 21 by Qcumber

[Mr_sukebe]

Some interesting results.

One thought about the stats is that you’ve not included GPU and CPU usage.

Whilst reviewing the performance over Germany what I’ve found is that:

- RAM usage is high, easily over 40GB

- GPU usage with similar settings to yours have been around 70-80% with 72fps most of the time

- VRAM seems to be mostly aircraft dependent.  The Apache was pushing nearly 12GB, the FC3 Mig29 just 5GB.  That’s promising as your testing of preload radius implies that maxing it out will only increase it by 1GB, which I have spare, and may have marginal gains elsewhere 

- the killer has been CPU usage.  I tried adding SAM sites at 31 of the idendified sites and that resulted in at least one CPU core being maxed out.  That was enough to drop my fps down to the mid 30s.  I removed probably 30% of the SAM units such that the CPU was not completely maxed out and sure enough, back to 72fps, lovely and smooth.

My learning point was that if I’m targeting 72fps, ensure that I’m not fully maxing out any of the CPU cores, the GPU, VRAM or RAM.  Maxing out any of them results in a bottleneck.

Edited April 21 by Mr_sukebe

[Qcumber]

1 minute ago, Mr_sukebe said:

Some interesting results.

One thought about the stats is that you’ve not included GPU and CPU usage.

Whilst reviewing the performance over Germany what I’ve found is that:

- RAM usage is high, easily over 40GB

- GPU usage with similar settings to yours have been around 70-80% with 72fps most of the time

- the killer has been CPU usage.  I tried adding SAM sites at 31 of the idendified sites and that resulted in at least one CPU core being maxed out.  That was enough to drop my fps down to the mid 30s.  I removed probably 30% of the SAM units such that the CPU was not completely maxed out and sure enough, back to 72fps, lovely and smooth.

My learning point was that if I’m targeting 72fps, ensure that I’m not fully maxing out any of the CPU cores, the GPU, VRAM or RAM.  Maxing out any of them results in a bottleneck.

That's a good point regarding CPU and GPU usage but I'm not sure how I can capture this alongside the other data. Any suggestions about software I could use? It would need to allow data export to a CSV file. 

It would be interesting to correlate usage with render times. My observation is that if GPU is maxed out then the render time exceeds 13.88ms and fps drops; which is exactly what you would expect. CPU usage would've harder to correlate due to multiple votes. 

[Mr_sukebe]

Sorry, I don’t know enough about data exports to understand how you’d match up their precise timings with your render time info.

Regardless of that, it was very easy to see that maxing out any of those resources results in a significant degrade to fps.  The interesting bit is how to turn this understanding into a rough guideline on recommended settings for people with different rigs.