Asynchronous Reprojection explained aka ghosting, stuttering, etc

[VirusAM]

Asynchronous Reprojection explained aka ghosting, stuttering, etc

 

There is a troubling issue in modern flight sims with regards to VR that rarely receives the attention it should, and resulted in me misspending AU$1200 on a HP Reverb. Asynchronous Reprojection (hereafter referred to as AR), also known as synchronous timewarp, asynchronous spacewarp and SteamVR Motion Smoothing, is a technology used to “mimic” extra frames when the user’s system cannot send rendered frames at the refresh rate of the headset. Many VR users refer to the issues caused by AR as ‘ghosting’, ‘judder’, ‘stuttering’, or ‘smoothness’. I’m sure there are some other names I’m not familiar with, but the vague way in which it is described, not to mention the huge variety of “fixes” to deal with it, are part of the reason I’m writing this. The impending launch of the Reverb G2 is also another reason. When I encountered the issue, I was told I hadn't configured my PC correctly, so spent weeks trying to solve this problem. Turns out there isn't a solution. So I thought I’d make a post about it to help clear up some confusion, and potentially save some simmers from spending a great deal of money chasing a VR experience that currently does not exist. Note that I’m not a VR specialist, but I was a technology journalist focused on enthusiast PC gaming hardware for 20 years with a passion for VR. I have a well-configured 9700K w/ 2080 Ti and all the trimmings, and I can’t run the Reverb without this issue. I apologize if I make any inaccuracies in this post, please feel free to clear them up with your feedback. A huge thanks to Aurelius, who is far more knowledgeable in this field than I am, for reading over this post and correcting the many inaccuracies! This guy knows his stuff!

 

Please, please note that I’m not disparaging you if you are a VR user and enjoy it – lucky you! I wish I didn’t notice this problem, as VR is an incredible experience; however, I think if you’ve played other VR titles where AR is not used at all, it’s very hard not to notice the AR side-effects. Having said that, as evidenced by this forum, there are DCS VR users who do not notice AR, even in DCS. Whether you’ll be able to notice it or not can only be tested when you do put on your chosen HMD. I do believe it will become more of an issue with the Reverb G2, though the RTX 3090 (and possibly 3080) may have the requisite horsepower to overcome this problem.

 

AR is generally not visible on “through-the-lens” videos because of the manner in which cameras differ from the human eye. Because modern graphics cards lack the ability to send the requisite rendered frames to higher resolution HMD’s like the Reverb and Valve Index, AR becomes much more prevalent. You will know if DCS is running AR if the in-game fps counter does not match the HMD's selected refresh rate (which is 90Hz for the Reverb Pro and G2, though this can be lowered to 60Hz). AR automatically kicks in when the PC supplying the frames can not match a framerate equal to the refresh rate of the HMD. If you’re running a lower resolution HMD than the Reverb, such as the Rift or Vive, and can actually match your PC’s outputted framerate to the HMD’s refresh rate, AR is not used and will not appear as an issue at all. AR was conceived to be used in rare instances when the PC “hiccups”, and can’t match the refresh rate – it absolutely was not designed to be used 100% of the time, as is often seen in flight sims. This is why every other VR game is designed to ensure that the frame rate matches the refresh rate.

 

So, what actually is AR? It is a mathematical method for using coordinate transforms in the form of matrix computations to deliver a class of motion interpolation technologies aimed ensuring a virtual reality headset's responsiveness to user motion even when the GPU isn't able to keep up with the headset's target framerate, and to reduce perceived input lag at all times regardless of framerate. Reprojection involves the headset's driver taking one or multiple previously rendered frames and using newer motion information from the headset's sensors to "reproject" or "warp" the previous frame into a prediction of what a normally rendered frame should now look like to account for the change in all six degrees of freedom (6DOF). "Asynchronous" refers to this process being continuously performed in parallel with rendering, allowing reprojected frames to be displayed without delay in case a regular frame isn't rendered in time, and is used in all frames by default to reduce perceived latency.”

 

If your PC is outputting 45fps to a 90Hz HMD, the HMD needs to display 90 frames regardless. (VR requires higher Hertz because of the interleaving of two displays to fool your occipital lobe into believing depth is being seen.) Therefore, it uses a clever algorithm to match every frame from your PC with a fake second frame. This means that rotational image display (3DOF) remains smooth, even if the objects within the scene do not. Unfortunately, artificially creating 50% of the frames is not a miracle software hack that works without any issues. If it did, we’d see it in every game engine on the planet, 2D or 3D! Imagine being able to double your performance without needing the hardware to do so!

 

So then, what do AR’s side effects, or artifacts, actually look like? For me, the thing most noticeable was the ground “jumping” or “stuttering” the closer I got. Even worse was behavior in dogfights or formation flying; other aircraft have a very weird “shadow” image layered over the true 3D model which lags behind them by about a meter, yet also hovers over the top of them. It’s rather difficult to describe, and unfortunately I can’t find any illustrative diagrams. Please post some if you can find them!

 

As mentioned, some people don’t appear to notice AR’s artifacts. It also seems to be minimized if you’re looking at objects that aren’t moving across the scene quickly – such as the ground from high altitude or in slow-moving aircraft. The best way to remove its issues is to not use it – but to do this, your PC must output frames that match the HMD’s refresh rate. This can sometimes be achieved by lowering the graphical detail or resolution, but often it cannot. You can also forcefully disable AR by config tweaks in most platforms, but this usually results in the overall experience feeling less smooth when you rotate your head.

 

(Any time the relative 6DOF velocity (dy/dx) is high, the distortion grows because the reprojection algorithm must account for the larger difference, which results in the viewer seeing things as “warped” or “torn”. That is why an object close to your aircraft (a tree, .. another plane in formation, etc…) is noticeably degraded while something in the vast distance (a mountain) appears to be unaffected. The direct mathematical reason for this is the coordinate transform projection.)

 

I hope this description helps those who are struggling to understand how AR affects a smooth user experience. Also, if you’re looking to make the splash into VR, this is a rather large issue that you should be aware of. I implore you to road-test your chosen HMD in person before outlaying any monies, I certainly wish I had! Note that watching a through the lens video or hearing other people swear that they don’t notice it doesn’t mean that you will not (lucky you if you don’t!). I can’t speak to which version of AR is superior, be it Oculus’ ASW, or SteamVR’s Motion Smoothing, as I’ve only seen it thru the Reverb Pro, but I would love to hear the opinions of those who can.

 

(Up until about 2018, the code from Oculus was superior to Valve. Gradually, Valve reworked their algorithms (some say, reverse engineered Oculus’) and they are now generally seen as identical.)

 

If you notice any erroneous statements, please feel free to correct me. And don’t take it personally if you love the VR experience even with AR on; I’m not attacking you at all. I actually envy you, and hope to join you for a great VR experience when the 3090 releases.

 

 

 

Good post.

I personally don’t care about Motion Smoothing.

I notice it but I don’t care about it...better have it that have low fps in VR.

Also you don’t consider something that in DCS in quite important. The low fps often are not due to the graphic card. That depends from many factors, but let’s say that in a fairly complex dcs mission, with ai units and triggers, and scripting the cpu will be the bottleneck so even the 3090 won’t save us.

For that there is no solution. Of course Vulkan and multithread will help...but dcs calculate every single bullet, so complex missions won’t never be easy on the CPU.

[VirusAM]

I'm surprised you get it on the Rift S - I thought it was low enough resolution that you could run at the same framerate as the HMD's refresh rate?

 

 

 

They get on the Rift S (I got it also when I used it before the Index), because of what I wrote on my above post.

Many people fail to understand that in a flight simulator, and in DCS in particular, the CPU IPC count is more important then the Graphics card. And there is no CPU that can run average dcs missions in VR at 90 (neither 60 trust me) every time.

Of course the GPU has its own importance, but when the CPU cannot feed enough frames per second to the card to render, there is nothing it can do.

This is not easy to address for developers.

They would need to rebuild completely the dcs engine, and probably also to simplify some calculations in the background for far units for example.

I hope this clarifies many doubts people seems to have with VR performances.

[imacken]

Right. But if you turn all the graphics to minimum can you get that FPS? If not I’m sure it’s the CPU. That’s where the limits on VR performance come from. And especially in DCS

You basically have the same system as the OP and so would experience the same issues if you are run VR

Your sig says you’re running a 4K monitor, so am I with a 2080Ti and can max every setting.

 

Precisely my point. It all depends on your hardware and settings. If you look at my sig, you’ll see I run a Reverb. Of course if you turn the load way down on the GPU the CPU becomes the weak link.

I can only repeat, DCS is not CPU bound per se.

 

What headset do you run in VR, and what is your experience of motion reprojection?

[GunSlingerAUS]

Precisely my point. It all depends on your hardware and settings. If you look at my sig, you’ll see I run a Reverb. Of course if you turn the load way down on the GPU the CPU becomes the weak link.

I can only repeat, DCS is not CPU bound per se.

 

What headset do you run in VR, and what is your experience of motion reprojection?

 

This. DCS is CPU dependent... to a point. Run a Reverb with its incredibly high resolution and refresh rate, and the GPU becomes the bottleneck. It's pretty easy to test this - simply lowering the resolution or MSAA on the Reverb and you get better fps. But this comes at a huge trade-off, with poor image quality. This is why I'm hoping the 3090 has the horsepower to power my Reverb at 60fps without making massive image quality drops.

[imacken]

Exactly - hardware and settings!

 

Personally, I am happy on a smooth, rock solid 45fps on my Reverb at pretty high settings, including 170% SteamVR SS. I'd rather have higher settings @ 45, than lower settings @ 90.

[VirusAM]

Yes but even a 3090 won’t be enough to keep 90/80 or even 60 fps in an heavy mission full of AI and scripting especially in MP.

[Motomouse]

And there is no CPU that can run average dcs missions in VR at 90 (neither 60 trust me) every time.

 

Because you know: 90 FPS is all about that settings, no trouble, no trouble!

 

 

Who is afraid of half resolution?

1564x1528 versus 1200x1440 (HP half resolution vs Rift S full resolution per eye)

 

I fly 90 FPS in multiplayer and over big cities. But usually I prefer to fire up 60 hz mode and fly 60 FPS high quality.

 

1. Set your Reverb to half resolution

2. Uninstall Tacview

 

Enjoy!

3600X & 1080Ti & HP Reverb

 

https://forums.eagle.ru/showthread.php?t=285493

Edited September 12, 2020 by Motomouse
link added

[SharpeXB]

What headset do you run in VR, and what is your experience of motion reprojection?

I’ve used VR headsets professionally but not for gaming or flight sims. Discussions like this make it clear that they don’t perform well enough.

 

They would need to rebuild completely the dcs engine, and probably also to simplify some calculations in the background for far units for example.

This. Like I said, they would have to create a separate VR version of the sim in order for it to work perfectly. Maybe if VR gets a high enough usage rate that could be justified.

Edited September 12, 2020 by SharpeXB

[30mikemike]

There is a troubling issue in modern flight sims with regards to VR that rarely receives the attention it should, and resulted in me misspending AU$1200 on a HP Reverb. Asynchronous Reprojection (hereafter referred to as AR), also known as synchronous timewarp, asynchronous spacewarp and SteamVR Motion Smoothing, is a technology used to “mimic” extra frames when the user’s system cannot send rendered frames at the refresh rate of the headset. Many VR users refer to the issues caused by AR as ‘ghosting’, ‘judder’, ‘stuttering’, or ‘smoothness’. I’m sure there are some other names I’m not familiar with, but the vague way in which it is described, not to mention the huge variety of “fixes” to deal with it, are part of the reason I’m writing this. The impending launch of the Reverb G2 is also another reason. When I encountered the issue, I was told I hadn't configured my PC correctly, so spent weeks trying to solve this problem. Turns out there isn't a solution. So I thought I’d make a post about it to help clear up some confusion, and potentially save some simmers from spending a great deal of money chasing a VR experience that currently does not exist. Note that I’m not a VR specialist, but I was a technology journalist focused on enthusiast PC gaming hardware for 20 years with a passion for VR. I have a well-configured 9700K w/ 2080 Ti and all the trimmings, and I can’t run the Reverb without this issue. I apologize if I make any inaccuracies in this post, please feel free to clear them up with your feedback. A huge thanks to Aurelius, who is far more knowledgeable in this field than I am, for reading over this post and correcting the many inaccuracies! This guy knows his stuff!

 

Please, please note that I’m not disparaging you if you are a VR user and enjoy it – lucky you! I wish I didn’t notice this problem, as VR is an incredible experience; however, I think if you’ve played other VR titles where AR is not used at all, it’s very hard not to notice the AR side-effects. Having said that, as evidenced by this forum, there are DCS VR users who do not notice AR, even in DCS. Whether you’ll be able to notice it or not can only be tested when you do put on your chosen HMD. I do believe it will become more of an issue with the Reverb G2, though the RTX 3090 (and possibly 3080) may have the requisite horsepower to overcome this problem.

 

AR is generally not visible on “through-the-lens” videos because of the manner in which cameras differ from the human eye. Because modern graphics cards lack the ability to send the requisite rendered frames to higher resolution HMD’s like the Reverb and Valve Index, AR becomes much more prevalent. You will know if DCS is running AR if the in-game fps counter does not match the HMD's selected refresh rate (which is 90Hz for the Reverb Pro and G2, though this can be lowered to 60Hz). AR automatically kicks in when the PC supplying the frames can not match a framerate equal to the refresh rate of the HMD. If you’re running a lower resolution HMD than the Reverb, such as the Rift or Vive, and can actually match your PC’s outputted framerate to the HMD’s refresh rate, AR is not used and will not appear as an issue at all. AR was conceived to be used in rare instances when the PC “hiccups”, and can’t match the refresh rate – it absolutely was not designed to be used 100% of the time, as is often seen in flight sims. This is why every other VR game is designed to ensure that the frame rate matches the refresh rate.

 

So, what actually is AR? It is a mathematical method for using coordinate transforms in the form of matrix computations to deliver a class of motion interpolation technologies aimed ensuring a virtual reality headset's responsiveness to user motion even when the GPU isn't able to keep up with the headset's target framerate, and to reduce perceived input lag at all times regardless of framerate. Reprojection involves the headset's driver taking one or multiple previously rendered frames and using newer motion information from the headset's sensors to "reproject" or "warp" the previous frame into a prediction of what a normally rendered frame should now look like to account for the change in all six degrees of freedom (6DOF). "Asynchronous" refers to this process being continuously performed in parallel with rendering, allowing reprojected frames to be displayed without delay in case a regular frame isn't rendered in time, and is used in all frames by default to reduce perceived latency.”

 

If your PC is outputting 45fps to a 90Hz HMD, the HMD needs to display 90 frames regardless. (VR requires higher Hertz because of the interleaving of two displays to fool your occipital lobe into believing depth is being seen.) Therefore, it uses a clever algorithm to match every frame from your PC with a fake second frame. This means that rotational image display (3DOF) remains smooth, even if the objects within the scene do not. Unfortunately, artificially creating 50% of the frames is not a miracle software hack that works without any issues. If it did, we’d see it in every game engine on the planet, 2D or 3D! Imagine being able to double your performance without needing the hardware to do so!

 

So then, what do AR’s side effects, or artifacts, actually look like? For me, the thing most noticeable was the ground “jumping” or “stuttering” the closer I got. Even worse was behavior in dogfights or formation flying; other aircraft have a very weird “shadow” image layered over the true 3D model which lags behind them by about a meter, yet also hovers over the top of them. It’s rather difficult to describe, and unfortunately I can’t find any illustrative diagrams. Please post some if you can find them!

 

As mentioned, some people don’t appear to notice AR’s artifacts. It also seems to be minimized if you’re looking at objects that aren’t moving across the scene quickly – such as the ground from high altitude or in slow-moving aircraft. The best way to remove its issues is to not use it – but to do this, your PC must output frames that match the HMD’s refresh rate. This can sometimes be achieved by lowering the graphical detail or resolution, but often it cannot. You can also forcefully disable AR by config tweaks in most platforms, but this usually results in the overall experience feeling less smooth when you rotate your head.

 

(Any time the relative 6DOF velocity (dy/dx) is high, the distortion grows because the reprojection algorithm must account for the larger difference, which results in the viewer seeing things as “warped” or “torn”. That is why an object close to your aircraft (a tree, .. another plane in formation, etc…) is noticeably degraded while something in the vast distance (a mountain) appears to be unaffected. The direct mathematical reason for this is the coordinate transform projection.)

 

I hope this description helps those who are struggling to understand how AR affects a smooth user experience. Also, if you’re looking to make the splash into VR, this is a rather large issue that you should be aware of. I implore you to road-test your chosen HMD in person before outlaying any monies, I certainly wish I had! Note that watching a through the lens video or hearing other people swear that they don’t notice it doesn’t mean that you will not (lucky you if you don’t!). I can’t speak to which version of AR is superior, be it Oculus’ ASW, or SteamVR’s Motion Smoothing, as I’ve only seen it thru the Reverb Pro, but I would love to hear the opinions of those who can.

 

(Up until about 2018, the code from Oculus was superior to Valve. Gradually, Valve reworked their algorithms (some say, reverse engineered Oculus’) and they are now generally seen as identical.)

 

If you notice any erroneous statements, please feel free to correct me. And don’t take it personally if you love the VR experience even with AR on; I’m not attacking you at all. I actually envy you, and hope to join you for a great VR experience when the 3090 releases.

 

100% with you on this. Hence why I'm back to 1440p and TrackIR.

[imacken]

I’ve used VR headsets professionally but not for gaming or flight sims. Discussions like this make it clear that they don’t perform well enough.

 

Interesting that you seem to give a lot of opinions in VR threads with no actual experience of it yourself!

 

You’ll always get threads like this in forums like this.

 

You should try it. It’s a game changing experience which, in the main, with the right hardware, works very nicely in DCS.

[SharpeXB]

Interesting that you seem to give a lot of opinions in VR threads with no actual experience of it yourself!

 

You’ll always get threads like this in forums like this.

 

You should try it. It’s a game changing experience which, in the main, with the right hardware, works very nicely in DCS.

One day I will “borrow” the headset from the office and try it out :pilotfly:

[hansangb]

One day I will “borrow” the headset from the office and try it out :pilotfly:

 

 

I wouldn't. It may make you want to upgrade and spend a lot of money.

[Lt_Jaeger]

Yeap

 

Even with all the shortcomings, there is no way back.

 

 

Gesendet von meinem LYA-L29 mit Tapatalk

[eaglecash867]

You’ll always get threads like this in forums like this.

 

 

+1.

 

Its all a highly subjective thing, so you're naturally going to have a few with a "It'll never work. We're all gonna die." attitude, along with a bunch of us who make it work within the confines of what things are actually capable of. We try to help people with the reality of the situation, rather than telling them to just give up because it "can't be done". Some even flat-out refused to even try the settings I posted once. No technical problem has ever been solved by continually insisting that something can't be done, without even stripping things down to the basics and working up from there. Ideally, we'd all love to have DCS run at 90FPS with full retinal resolution, but until the day that happens, we have to work with what we've got. I have good clarity and performance, with an extreme minimum of ghosting/stuttering (it happens occasionally in ground-object-rich environments with both MFCDs in the A-10C displaying video). To tweak my settings to get that combination, I purposely used the A-10C "In The Weeds" mission and would immediately look to the side and roll the aircraft back and forth to look for ghosting of buildings and other ground objects. From there, I just dialed things down until that was eliminated with the MFCDs NOT displaying video. I turned shadows off and set cockpit display resolution to 256, which was just fine because I found that I don't spend a lot of time marveling at the shadows (which look weird in VR anyway, with them coming and going in one eye or the other) or staring at my mirrors when I'm flying.

[dburne]

I’ve used VR headsets professionally but not for gaming or flight sims. Discussions like this make it clear that they don’t perform well enough.

 

 

:megalol:

[TOYKILLA]

Why do some people keep saying this over and over. DCS IS NOT CPU bound. It totally depends on your hardware and settings in-game.

 

 

 

My DCS is GPU bound 90% of the time, and I am looking forward to my 3090!

 

Gesendet von meinem SM-G980F mit Tapatalk

[Lord_Pyro]

For the Rift (and Rift S) you can kill it in the Oculus Debug Tool. I (have to) do this every time i start oculus. If i forget about it you can't read Huds anylonger, or for example aim through revi sights.

But at least you can just alt tab out of the game and change it at any time, without needing to restart anything.

null

[Qcumber]

You can also use the Oculus Tray Tool. It's a bit easier to manage this way. 

https://www.apollyonvr.com/oculus-tray-tool

[Sile]

last post (toykilla) was from september 2020. 

why not let it R.I.P.

Edited May 14, 2023 by Sile