Foveated Rendering Support for DCS

[nikoel]

1 hour ago, Scavenger said:

Hi folks. Apologies if this is off-topic, but I'm looking to dip my toes in the VR space and this foveated rendering sounds quite promising - I was looking at reviews of the Quest Pro in DCS and a few users were saying it's a great headset but that the price shows it's not really 'meant for DCS' (effectively that DCS doesn't take full advantage of it). Would you say the implementation of foveated rendering here negates those comments? 

In addition, do you think the performance gains here are enough for a 4070 (non-Ti), 5800X3D and 32 GB of DDR4 to make use of the headset, or should I be going for the Quest 2 instead? Sorry again if off-topic! 

What hippo said

The one thing he did not mention is that the QP is now easier to run with 37% higher pixel density than the quest 2 because of DFR

the reason for this is becuase our eyes are only sharp within an extremely small range. Like a G2 but our sweet spot moves - for instance I am typing this on my iPhone and I have to chose between looking a the keyboard or what I am typing even though they are a tiny distance from each other. So DFR is amazing because it renders like a around your periphery all the while being  sharp in your focus area

DFR can’t be implemented on the Quest 2 because of the lack of eye tracking

Quest 2 utilises only a part of the the stated resolution because it uses one long screen. QP uses two = fewer “wasted pixels”

Quest Pro has - thus the pixels you render all around you are not going to ferens lens waste 

With this said, we all chose QP because we felt it was the best headset for our budget and feature-set balance. Ask the same question over at the Pimax thread and you will get the same recommendations from them for their headset (even though I personally can not stand them as a company; they have to release something universally acclaimed for me to even consider them)

Edited June 23, 2023 by nikoel

[nikoel]

Great Elevator Music 

Also ~35-40% uplift according to FPS counter (30ish to 50ish FPS)

[dburne]

23 minutes ago, nikoel said:

Also ~35-40% uplift according to FPS counter (30ish to 50ish FPS)

I would say that is about what we saw with the Aero's implementation as well. Absolutely incredible.

[Dogmanbird]

Thank you Mbucchia!. Got it working here too and is a nice improvement. I thought the quest pro eye tracking would have been faster than it is and I can see the high res area catching up to my eye's movement. I's still good thing though.

@dburne - does the dfr on the aero trail behind your eye movement?   

@ everyone - does it take a long time to load / get into the aircraft for you?

 

Thanks

Edited June 23, 2023 by Dogmanbird

[Hippo]

Please could someone who is getting an uplift >= 30% post their Oculus, DCS and Meta-Foveated settings, and system spec?  Thx.

[Dogmanbird]

Hi Hippo. I first tested with a vanilla setup (no OTT or Debug Tool), default settings installed by the metafov program and just set the link resolution to max at 90hz. I think with this alone, the dfr gave a 20% fps improvement and distant clarity, maybe more.

I've been experimenting with other settings in OTT etc. Unfortunately I'm still unable to get it to a point where I'd use it over the reverbg2 for distant detail but it's getting better. 

 

 

Edited June 23, 2023 by Dogmanbird

[nikoel]

39 minutes ago, Dogmanbird said:

Thank you Mbucchia!. Got it working here too and is a nice improvement. I thought the quest pro eye tracking would have been faster than it is and I can see the high res area catching up to my eye's movement. I's still good thing though.

@dburne - does the dfr on the aero trail behind your eye movement?   

@ everyone - does it take a long time to load / get into the aircraft for you?

 

Thanks

 

It’s fast. I reckon you’re 45FPS AWS - which is by definition twice as slow 

[Dogmanbird]

Hmm. The eye tracking in the game is the same speed as the test available after calibrating it. Maybe I've missed something in the quest setup? Are there eye tracking speed options? 

[dburne]

46 minutes ago, Dogmanbird said:

Thank you Mbucchia!. Got it working here too and is a nice improvement. I thought the quest pro eye tracking would have been faster than it is and I can see the high res area catching up to my eye's movement. I's still good thing though.

@dburne - does the dfr on the aero trail behind your eye movement?   

 

Not at all. I hardly even notice it. I do believe the Aero eye tracking is faster though than the Quest Pro's, pretty sure I read that somewhere. I don't think the Quest Pro eye tracking was designed quite with the same thing in mind as the Aero's was.

[Dogmanbird]

^ Thank you.

just went to check if there were eyetracking options and the headset is still flat after sitting on the dock all day. I can't get it to charge from it. However the hand controllers still do. It still seems to be charging from the USB port

 

[slughead]

1 minute ago, Dogmanbird said:

^ Thank you.

just went to check if there were eyetracking options and the headset is still flat after sitting on the dock all day. I can't get it to charge from it. However the hand controllers still do. It still seems to be charging from the USB port

 

I had something similar happen to me too. The headset had an update pushed through to it and, for some reason, discharged itself even though it was on the dock. I can't remember how I fixed it. I may have disconnected the cable from the charging station and then connected it directly to the headset to charge it. Then I may have power-cycled it. That's all I can think of at the moment.

[Dogmanbird]

Thanks  might let it do a full charge on cable then see what happens

[mbucchia]

2 hours ago, Dogmanbird said:

I thought the quest pro eye tracking would have been faster than it is and I can see the high res area catching up to my eye's movement

Are you using wireless? That could be cause for latency.

[Dogmanbird]

i'm using link cable.

I'm probably expecting too much of it based on early demo videos I saw online which might have exaggerated it's performance somewhat. I think it's lagging about 200 - 300ms behind my eye movements. The tracking is certainly still good enough that I'd have bought one for myself had the headset had a little better resolution. Fortunately my work owns at least 3 of them

[nikoel]

One thing I noticed when I started to play with the lower values inside the periphery vision, is that the resolution change is only mildly noticeable, however the shimmer especially on things like type-text is distracting even with MSAAx4.

I recon, if there was an effective way to subdue this shimmer (even at a cost of more blur) I think it could be worth for people with weaker cards

Is there an other way of getting rid of this periphery shimmer?

Edited June 23, 2023 by nikoel

[Dogmanbird]

^ yes I agree, even with msaa x4. By the time I up the perpixel res to eliminate significant peripheral shimmer, ASW must be then be running at an fps that causes the heat haze wobble 

Edited June 23, 2023 by Dogmanbird

[mbucchia]

On 6/22/2023 at 8:04 AM, dburne said:

Any chance of sharpening coming to the Aero DFR settings? Or would it even matter?

TL;DR: I'm still undecided about this one.

Long version (because y'all know I love to write up the details):

Varjo-Foveated was comparatively simple. It 100% relies on Varjo's own quad views support in their OpenXR runtime. All Varjo-Foveated does is:

- Add the necessary setup for foveated rendering on top of quad views - that DCS isn't doing otherwise - that's the primary bit of magic;

- Add the ability to override the focus region pixel density. size etc, which people love to play with;

- Implement a workaround for DCS incorrect frame metadata submission introduced by MT.

There is nothing that requires to touch any of the pixels. So there is no set-up in the code to do Direct3D stuff.

This is also why Varjo-Foveated took maybe 1/5th of the time to implement compared to Meta-Foveated. Meta-Foveated does all the above too, but it cannot leverage existing quad views support (it's not in the Oculus OpenXR runtime). So it has to reimplement it, including:

- Hooking into the eye tracker to tell DCS how to render the 4 views properly. Fortunately this code is 100% reused from OpenXR Toolkit for the eye tracker part and 100% reused from PimaxXR for the view projections calculations. So that bit was easy.

- Compositing the 4 views into a stereo view. Now that's the complex one. This one requires to do some drawing of my own. First because the Oculus OpenXR runtimes lacks of certain features (which Pimax supported for example, and therefore I did not have to implement them in PimaxXR) and second because there are edge cases in the Oculus OpenXR runtime (for example submitting multiple stereo layers seems to preclude ASW).

So for that last one, there is quite some significant code needed to set up drawing via Direct3D, pull data from the DCS backbuffers, setup new backbuffers to receive the produced views etc. That code does not exist in Varjo-Foveated, nor in PimaxXR. With all of that set up for Meta-Foveated, adding CAS was pretty trivial (I think it took an hour).

There's also Varjo being an odd-ball which caused quite a few issues in the past, the way they manage OpenXR backbuffers internall is 100% different than other OpenXR implementations, and it requires additional logic.

It would be complex to add this into Varjo-Foveated. Not impossible, but probably 2-3 days of work. I know, I sound terrible, because 2-3 days of work sound like nothing  but they are 2-3 days where I'm not doing something else. I'm literally that busy.

So wait and see... Right now I will focus on applying my learnings from Meta-Foveated back into PimaxXR for Crystal support. Ideally I can find a solution where Quest Pro/Varjo Aero/Pimax Crystal/others all share the same solution without having additional maintenance costs.

For example, I have Meta-Foveated working on G2 Omnicept - but releasing that is an additional maintenance cost (which I deem not worth it due to low volume of users). Ideally, there would be >1 maintainer to do work like this - eg someone passionate about G2 Omnicept support willing to spend a couple of hours a month to maintain it.

 

 

<long brain dump over>

 

[dburne]

Thanks much @mbucchia for all the info and all your great work! Believe me I am loving it as is also!

[freehand]

4 hours ago, Dogmanbird said:

^ Thank you.

just went to check if there were eyetracking options and the headset is still flat after sitting on the dock all day. I can't get it to charge from it. However the hand controllers still do. It still seems to be charging from the USB port

 

I had this after the update last night no headset orange light when sitting on the dock but controller ok the headset would charge directly from another usb port.

But to solve the problem I removed the charger plug from the wall socket "not just the cable lead from the dock " and done job all good.

You have to replug in the charger of course lol

 

Edited June 23, 2023 by freehand

[Dogmanbird]

Thanks again Slughead and Freehand - in the end it was because I had turned the rear head strap padding upside down to get the lenses tilted forward more. However it means the headset tilts too far back for the pins to make contact when on the dock. I just need to place something under the head strap to raise it about 4cm when charging 

[Hippo]

On 6/21/2023 at 11:34 PM, ismaeljorda said:

Today I have been doing tests and in the next few days I will try to make a video with the recommended settings.
Also, keep in mind that we can use the FOV MULTIPLIER (0.9 Horizontal and 0.9 Vertical) in OTT to gain extra performance.

Cheers,
Ismael
PS: Thanks @mbucchia
 

Very much looking forward to your video.  Nudge... nudge...   And whether you find the performance improvement as significant as with the Aero?

[VirusAM]

Using nvg there is a rectangle, with a circle inside it. It doesn't show with a screenshot only in the headset.
This rectangle follows the eyes so I guess it's the eye tracking part.
I don't know if there is something that can be done to hide it

[mimamema]

4 minutes ago, VirusAM said:

Using nvg there is a rectangle, with a circle inside it. It doesn't show with a screenshot only in the headset.
This rectangle follows the eyes so I guess it's the eye tracking part.
I don't know if there is something that can be done to hide it
 

That´s a well-known bug by Varjo users using Mbucchia´s quad views as well, I think there´s a mod somewhere that makes all your FOV have the night vision on.

Ok, I had a look and apparently is this one Tacca's NVG mod 1.2 for 2.8* (digitalcombatsimulator.com)

 

[Cytarabine]

Thanks @mbucchia for your work on this. It is fantastic, running it with my Quest Pro gave me a nice performance boost, and enough that I have gone from 90Hz forced to 45Hz to 72Hz with ASW switched off which is nice. Even over Sinai in a Mudhen with stuff happening all around.

Still tweaking the settings a little bit to find the exact right balance, have turned up the render scaling in the center a bit and down in the peripheries (it's so rare that I actually catch the foveated rendering missing).

It also makes me feel better about the Quest Pro compared to the upcoming Quest 3 so that's a bonus. 

Edited June 24, 2023 by Cytarabine

[VirusAM]

That´s a well-known bug by Varjo users using Mbucchia´s quad views as well, I think there´s a mod somewhere that makes all your FOV have the night vision on.
Ok, I had a look and apparently is this one Tacca's NVG mod 1.2 for 2.8* (digitalcombatsimulator.com)
 

Thanks