Native OpenXR (with mbucchia fix) vs SteamVR: reprojection question

[YoYo]

Hi @mbucchia,

firstly Thank You again for fix and 1.2.4 Toolkit. From your fix native DCS OpenXR works again with Motion Reprojection=Always On, no CTD. Also Turbo mode starts to work too without CTD. Im on WMR and Reverb G2. For VR I was using WMR+SteamVR+ReShade and I was testing WMR+OpenXR+Toolkit 1.2.4.

Both are working ok, however I noticed that OpenXR has a bigger problem to show right fast moving unit with MR=Always ON than with SteamVR (always on too). I didnt any digits test (so cant show any digrams with timing) but according me motion reprojection behaves worse in OpenXR than with SteamVR (maybe reprojection in Steam works better or is better optimalized?). I cant fly with stable 90 FPS in DCS or without MR, the stutter is visible between 45 to 90 (sometimes I'm 90, but just above the ground I'm already around 60-80) so I use 45FPS option (always on). Generally in both cases the picture from cockpit view (ground, trees, horizon, clouds etc) is very smooth and works very well but I noticed that with OpenXR the ghosting effect is more visible (fast moving objects). Test: F3 view, so flyby view from external, second test - rotor blades in a helicopter, e.g. Apache. The tearing on fast moving object is more visible on OpenXR. However SteamVR handles this much better, no tearing, ghosting effect like with OpenXR.

I'm still thinking about switching to OpenXR, especially since it works now, but I don't like that much ghosting on fast moving objects (F3 view, combat dogfight or rotor blades). It's strange, because theoretically OpenXR should be better, but SteamVR seems to have a better optimized reprojection function.

Someone has similar observations? Any suggestions? 

 

[A.F.]

vor 14 Minuten schrieb YoYo:

Hi @mbucchia,firstly Thank You again for fix and 1.2.4 Toolkit. From your fix native DCS OpenXR works again with Motion Reprojection=Always On, no CTD. Also Turbo mode starts to work too without CTD. Im on WMR and Reverb G2. For VR I was using

mbicchia is working on it, I think

[edmuss]

To date the reprojection in openxr has always been behind that of steamvr, I understand that it's because of how opencomposite operated in conjunction with openvr (steamvr) it was very difficult to accurately predict the reprojected frames, hence the increased artifacting.  Now that we have native openxr support without the translation layer to complicate things it should be possible to improve the accuracy of the reprojection, hopefully up to the level at which steamvr operates.

One huge advantage that openxr has over steamvr is that it is able to reproject at smaller fractions of the refresh rate, running at 90Hz it's able to produce a smooth reprojected image at 45, 30 and 22 Hz.  Essentially with steamvr, if your framerates are below 50 fps (half refresh rate + a couple of ms GPU overhead) then reprojection will disengage and you will have a very stuttery image; in openxr it will simply step down to the next refresh rate fraction of one third refresh rate.

[Glide]

25 minutes ago, A.F. said:

mbicchia is working on it, I think

He posted he's taking a family leave of absence on the other forum.  Here's hoping all is well!

[YoYo]

9 minutes ago, edmuss said:

it's able to produce a smooth reprojected image at 45, 30 and 22 Hz. 

Right, however if we talk about WMR (and G2) it works only in MSFS on every level like this, not like full or half refresh only, not yet in DCS (perhaps due of engine limitation?).

[Sile]

@YoYo

Thank you for this topic. What you describe in the first post is exactly what i oberserve as well.

In the past i have been checking out OpenXR from time to time to test, but still go back to SteamVR mainly because of the better motion smoothing it provides in DCS with my G2. I hope so much for some progress on this topic.

Btw.: The smaller fraction MRs 30 and 22 Hz worked for me in DCS last time i checked with opencomposite. Unless MR rate is not lock in OpenXR Toolkit...

Edited February 1, 2023 by Sile

[Baldrick33]

21 minutes ago, YoYo said:

Right, however if we talk about WMR (and G2) it works only in MSFS on every level like this, not like full or half refresh only, not yet in DCS (perhaps due of engine limitation?).

Not sure I understand that. The 22, 30 & 45 reprojection have all worked in OpenXR & DCS since the initial opencomposite release.

22 isn't great but 30 is very playable - almost indistinguishable from 45 in my experience with WMR & G1

[YoYo]

24 minutes ago, Baldrick33 said:

Not sure I understand that. The 22, 30 & 45 reprojection have all worked in OpenXR & DCS since the initial opencomposite release.

22 isn't great but 30 is very playable - almost indistinguishable from 45 in my experience with WMR & G1

I wrote about the same MR like it is in MSFS here. So you have MR near always, even if You have 33 or 27 fps (it works independent of refresh rate of headset). Full spectrum, DCS dosent have possibility to work like this but this is different subject.

[Glide]

I am getting great results with MR in OpenXR, with MR enabled via OpenXR Toolkit, Locked at 30fps.  I have my headset at 60hz with OpenXR set to 75% Custom Resolution.  I believe the lower custom resolution reduces the MR artifacts.  However, there need to be changes to the game to work perfectly.

If you turn on the OpenXR Toolkit FPS counter and pan your head back and forth, with MR off you will see double vision, but with MR on you can read the letters perfectly.  Your HMD, however, may still suffer from the wiggles with MR enabled depending on whats going on in front of you.  It seems to me that these MR artifacts are like tears in 2D.  The pixel is supposed to light up at a certain place and time, but it can't quite get there.  In 2D this would be a screen tear, but its more of a spraying affect in VR.  I just tell myself that's how the world looks through fancy canopy glass and fancy HMD lenses.

[edmuss]

42 minutes ago, YoYo said:

I wrote about the same MR like it is in MSFS here. So you have MR near always, even if You have 33 or 27 fps (it works independent of refresh rate of headset). Full spectrum, DCS dosent have possibility to work like this but this is different subject.

I'm not sure what you're describing but openxr reprojection has always run at 1/2, 1/3 and 1/4 refresh rate fractions - it's always implicitly linked to the refresh rate.  What you may have been seeing is framerate counter hysteresis whereby when the reprojection jumps from fraction to fraction it averages out the FPS counter for a few seconds.
At 90Hz refresh rate the FPS will lock to 90/45/30/22 and at 60Hz refresh rate the FPS will lock to 60/30/20/15.

[YoYo]

15 minutes ago, Glide said:

It seems to me that these MR artifacts are like tears in 2D.  The pixel is supposed to light up at a certain place and time, but it can't quite get there.  In 2D this would be a screen tear, but its more of a spraying affect in VR.  I just tell myself that's how the world looks through fancy canopy glass and fancy HMD lenses.

The truth is, in my experience, SteamVR reproduces it better, almost no "tearing", and here we have these artifacts like ghosting on fast moving objects (combat, rotor, F3 view). So all in all, I don't know why OpenXR is considered a more polished or less engaging system, when SteamVR looks better in the same situation (MR="Always on", stable 45fps here and here). Maybe OpenXR should still be optimized?

Edited February 1, 2023 by YoYo

[mbucchia]

See https://forums.flightsimulator.com/t/openxr-toolkit-upscaling-world-scale-hand-tracking-release-thread/493924/3034?u=mbucchia

(And no I do not have a release date, as someone else mentioned, I am on personal time off).

[YoYo]

This is exactly what I wrote about. Thanks for the link and maybe someday!

For the time being, I will probably stay with SteamVR and the current settings for this reason. Artifacts spoil the experience a bit, especially since we have a combat sim here, not a civil one, so "ghosting" effect can be quite an annoying thing for me. I'm still trying different options. It's nice to know that you know what it's about. You are my hero from years.

 Thanks!

Edited February 1, 2023 by YoYo

[mbucchia]

3 hours ago, edmuss said:

I understand that it's because of how opencomposite operated in conjunction with openvr (steamvr) it was very difficult to accurately predict the reprojected frames, hence the increased artifacting.  Now that we have native openxr support without the translation layer to complicate things it should be possible to improve the accuracy of the reprojection, hopefully up to the level at which steamvr operates.

That's both true and not entirely accurate.

For details on shortcomings of the motion reprojection algorithm itself today compared to the SteamVR implementation, refer to my post above. This isn't related to OpenComposite.

There was however an additional obstacle with OpenComposite, which was indeed related to frame prediction time not being reported properly, due to how OpenVR works.

This can be observed (if I recall properly) when running the game (or any game really) with OpenComposite and using OpenXR Tools for WMR developer option "Jiggle view rotations". When you use that, the view in the headset shall remain stable (except flickering in the corners). With OpenComposite, I recall this test failing: instead the whole view shakes really uncomfortably. This is evidence that the frame prediction times are misused by the application (here OpenComposite, but again not their fault, they are limited by OpenVR), and therefore will make motion reprojection much less accurate.

Now with DCS native OpenXR mode, you can run the same "Jiggle view rotations" test and you observe that the image does remain stable in the headset (not in preview windows, but that is expected since preview windows doesn't perform reprojection). This test passing now means that frame prediction times are properly used by the application (here the DCS engine itself). So this is definitely good news, because it should help make motion reprojection more accurate.

For a more detailed explanation of how motion reprojection works in general, see my other post here: Motion Reprojection explained - General Discussion & Interests / Virtual Reality (VR) - Microsoft Flight Simulator Forums.

[mbucchia]

50 minutes ago, YoYo said:

This is exactly what I wrote about. Thanks for the link and maybe someday!

For the time being, I will probably stay with SteamVR and the current settings for this reason. Artifacts spoil the experience a bit, especially since we have a combat sim here, not a civil one, so "ghosting" effect can be quite an annoying thing for me. I'm still trying different options. It's nice to know that you know what it's about. You are my hero from years.

 Thanks!

 

One test you could do and that would be interesting: try OpenXR+SteamVR (not OpenVR) and see if you get the same experience (that you like) on both.

This would give me some more information on where to look specifically. 

See my other post on how to toggle SteamVR OpenXR and OpenXR for WMR:

Thanks.

[Sile]

I run OpenXR with SteamVR Runtime and still find it smoother (and a little sharper) in Motion Smoothing compared to OpenXR with WMR Runtime and Motion Reprojection (and OpenXR Toolkit 1.2.4).

With smoother i mean the movement of objects like Jets. Fast moving objects are still "jittering" with WMR despite constant fps. 
Terrain and static Objects e.g. on the ground are fine.

But i also think Motion Reprojection in OpenXR with WMR Runtime got better now with the OpenXR Toolkit 1.2.4.

Edited February 1, 2023 by Sile

[YoYo]

17 hours ago, mbucchia said:

One test you could do and that would be interesting: try OpenXR+SteamVR (not OpenVR) ....

I did it and back with report. Yep, the behovior looks quite well! Interesting if you use SteamVR as OpenXR with Motion Reprojection on.

Maybe it doesnt look exactly the same like SteamVR only (in 100%), but a lot better than OpenXR only as native WMR/DCS. The ghosting effect is something about 80% of quality of picture in SteamVR only (so not the same exactly according me) but I didnt notice too many and very noticable artifacts like OpenXR only on fast moving objects. So maybe its a good way to solve it? 

[Sile]

After further testing of the following three VR Flavours for my G2 with regard to Motion Smoothing / Reprojection (MR) i see the following. (Didn't test opencomposite any more.)

OpenVR -> SteamVR -> WMR4SteamVR -> WMR:

+ MR good 

OpenXR native -> SteamVR -> WMR4SteamVR -> WMR:

+ MR good 

OpenXR native -> WMR:

- MR not as good as SteamVR ones. 
A good test is the F-16 Persian Gulf AAR Quick Mission. Fast rolling of the aircraft produces very noticable jittering of the cockpit side frame when viewed against the cloud with OpenXR->WMR. Same with objects moving very fast against the terrain or clouds. With any SteamVR "MR" these effects are very subtle. E.g. only a very small seam of the cockpit side frame jitters slightly when rolling fast.

Another test is showing the semi transparent DCS fps graph overlay with RCTRL+Pause. The overlay jitters very noticable when rolling the aircraft with OpenXR->WMR, but not with any SteamVR MR.

Didn't notice a difference between OpenVR->SteamVR... and OpenXR->SteamVR. 

Thank you.

Edited February 2, 2023 by Sile

[Glide]

With my G2 and 3080Ti, I get better MR with OpenXR native -> WMR.  I have mine locked at 30fps from within the toolkit.  I also am experimenting with OpenXR custom resolution @ 75% to make the MR artifacts less noticeable (you don't need any scaling or sharpening when you drop the resolution).  OpenXR native to WMR is the smoothest for me.

EDIT: Just an update, I enabled Fixed Foveated Rendering and Eye Tracking (Omnicept version of the G2), and that gave me the boost to go back to OpenXR 100%. Be sure to try these features as per the Toolkit website. 

Edited February 2, 2023 by Glide

[mbucchia]

Thanks all, your observations confirm that the fix I discussed earlier (in the MSFS thread) should be satisfactory for you once it is rolled out.

[YoYo]

3 hours ago, Sile said:

After further testing of the following three VR Flavours for my G2 with regard to Motion Smoothing / Reprojection (MR) i see the following. (Didn't test opencomposite any more.)

OpenVR -> SteamVR -> WMR4SteamVR -> WMR:

+ MR good 

OpenXR native -> SteamVR -> WMR4SteamVR -> WMR:

+ MR good 

OpenXR native -> WMR:

- MR not as good as SteamVR ones. 
A good test is the F-16 Persian Gulf AAR Quick Mission. Fast rolling of the aircraft produces very noticable jittering of the cockpit side frame when viewed against the cloud with OpenXR->WMR. Same with objects moving very fast against the terrain or clouds. With any SteamVR "MR" these effects are very subtle. E.g. only a very small seam of the cockpit side frame jitters slightly when rolling fast.

Another test is showing the semi transparent DCS fps graph overlay with RCTRL+Pause. The overlay jitters very noticable when rolling the aircraft with OpenXR->WMR, but not with any SteamVR MR.

Didn't notice a difference between OpenVR->SteamVR... and OpenXR->SteamVR. 

Thank you.

 

This part (parking hot start) is a good test too:

In the case of WMR>OpenXR, the problem with reprojection is clearly visible here. Clearly visible "waves" are created on frames when the rotor is running. With SteamVR aor OpenXR via SteamVR they are present still but with reduction about 80% which is a perfectly acceptable level.

[Glide]

Just got a new version of OpenXR Toolkit.  Thank you so much @mbucchia

So, I see now that the little bit of double vision you see panning your head back and forth when MR is turned off is the only real issue with having MR off.  The sim looks much better with MR off, no question.  And, looking out the cocking at 9 or 3 oclock will be jerky without MR (in real life it would be a blur).  I think it's better to turn it off and get used to the artifacts. 

I am running the lastest with Turbo Mode On, FFOV ON, Eye Tracking ON, 10% NIS Sharpening.  Still running OpenXR 75%.  So far, so good.

[Sile]

vor 13 Stunden schrieb mbucchia:

Thanks all, your observations confirm that the fix I discussed earlier (in the MSFS thread) should be satisfactory for you once it is rolled out.

Thank you very much, looking forward to the fix!

The example videos in the MSFS thread above look great and promissing, especially the second half of the second video. (+ Nvidia Optical Flow).

[YoYo]

1 hour ago, Sile said:

Thank you very much, looking forward to the fix!

The example videos in the MSFS thread above look great and promissing, especially the second half of the second video. (+ Nvidia Optical Flow).

Yep, the second option looks very promising .

[Sile]

Recently I'm reading many rumors about MS discontinuing WMR/VR, HP leaving the Market, ... employees being laid off. I'm not sure if an update is still in the pipeline or on top priority.

On a personal note i hope mbucchia isn't affected by all these developments. Such a great, dedicated and helpful developer for our community. I wish him only the best!

Edited February 6, 2023 by Sile