Asynchronous Reprojection explained aka ghosting, stuttering, etc

[GunSlingerAUS]

There is a troubling issue in modern flight sims with regards to VR that rarely receives the attention it should, and resulted in me misspending AU$1200 on a HP Reverb. Asynchronous Reprojection (hereafter referred to as AR), also known as synchronous timewarp, asynchronous spacewarp and SteamVR Motion Smoothing, is a technology used to “mimic” extra frames when the user’s system cannot send rendered frames at the refresh rate of the headset. Many VR users refer to the issues caused by AR as ‘ghosting’, ‘judder’, ‘stuttering’, or ‘smoothness’. I’m sure there are some other names I’m not familiar with, but the vague way in which it is described, not to mention the huge variety of “fixes” to deal with it, are part of the reason I’m writing this. The impending launch of the Reverb G2 is also another reason. When I encountered the issue, I was told I hadn't configured my PC correctly, so spent weeks trying to solve this problem. Turns out there isn't a solution. So I thought I’d make a post about it to help clear up some confusion, and potentially save some simmers from spending a great deal of money chasing a VR experience that currently does not exist. Note that I’m not a VR specialist, but I was a technology journalist focused on enthusiast PC gaming hardware for 20 years with a passion for VR. I have a well-configured 9700K w/ 2080 Ti and all the trimmings, and I can’t run the Reverb without this issue. I apologize if I make any inaccuracies in this post, please feel free to clear them up with your feedback. A huge thanks to Aurelius, who is far more knowledgeable in this field than I am, for reading over this post and correcting the many inaccuracies! This guy knows his stuff!

 

Please, please note that I’m not disparaging you if you are a VR user and enjoy it – lucky you! I wish I didn’t notice this problem, as VR is an incredible experience; however, I think if you’ve played other VR titles where AR is not used at all, it’s very hard not to notice the AR side-effects. Having said that, as evidenced by this forum, there are DCS VR users who do not notice AR, even in DCS. Whether you’ll be able to notice it or not can only be tested when you do put on your chosen HMD. I do believe it will become more of an issue with the Reverb G2, though the RTX 3090 (and possibly 3080) may have the requisite horsepower to overcome this problem.

 

AR is generally not visible on “through-the-lens” videos because of the manner in which cameras differ from the human eye. Because modern graphics cards lack the ability to send the requisite rendered frames to higher resolution HMD’s like the Reverb and Valve Index, AR becomes much more prevalent. You will know if DCS is running AR if the in-game fps counter does not match the HMD's selected refresh rate (which is 90Hz for the Reverb Pro and G2, though this can be lowered to 60Hz). AR automatically kicks in when the PC supplying the frames can not match a framerate equal to the refresh rate of the HMD. If you’re running a lower resolution HMD than the Reverb, such as the Rift or Vive, and can actually match your PC’s outputted framerate to the HMD’s refresh rate, AR is not used and will not appear as an issue at all. AR was conceived to be used in rare instances when the PC “hiccups”, and can’t match the refresh rate – it absolutely was not designed to be used 100% of the time, as is often seen in flight sims. This is why every other VR game is designed to ensure that the frame rate matches the refresh rate.

 

So, what actually is AR? It is a mathematical method for using coordinate transforms in the form of matrix computations to deliver a class of motion interpolation technologies aimed ensuring a virtual reality headset's responsiveness to user motion even when the GPU isn't able to keep up with the headset's target framerate, and to reduce perceived input lag at all times regardless of framerate. Reprojection involves the headset's driver taking one or multiple previously rendered frames and using newer motion information from the headset's sensors to "reproject" or "warp" the previous frame into a prediction of what a normally rendered frame should now look like to account for the change in all six degrees of freedom (6DOF). "Asynchronous" refers to this process being continuously performed in parallel with rendering, allowing reprojected frames to be displayed without delay in case a regular frame isn't rendered in time, and is used in all frames by default to reduce perceived latency.”

 

If your PC is outputting 45fps to a 90Hz HMD, the HMD needs to display 90 frames regardless. (VR requires higher Hertz because of the interleaving of two displays to fool your occipital lobe into believing depth is being seen.) Therefore, it uses a clever algorithm to match every frame from your PC with a fake second frame. This means that rotational image display (3DOF) remains smooth, even if the objects within the scene do not. Unfortunately, artificially creating 50% of the frames is not a miracle software hack that works without any issues. If it did, we’d see it in every game engine on the planet, 2D or 3D! Imagine being able to double your performance without needing the hardware to do so!

 

So then, what do AR’s side effects, or artifacts, actually look like? For me, the thing most noticeable was the ground “jumping” or “stuttering” the closer I got. Even worse was behavior in dogfights or formation flying; other aircraft have a very weird “shadow” image layered over the true 3D model which lags behind them by about a meter, yet also hovers over the top of them. It’s rather difficult to describe, and unfortunately I can’t find any illustrative diagrams. Please post some if you can find them!

 

As mentioned, some people don’t appear to notice AR’s artifacts. It also seems to be minimized if you’re looking at objects that aren’t moving across the scene quickly – such as the ground from high altitude or in slow-moving aircraft. The best way to remove its issues is to not use it – but to do this, your PC must output frames that match the HMD’s refresh rate. This can sometimes be achieved by lowering the graphical detail or resolution, but often it cannot. You can also forcefully disable AR by config tweaks in most platforms, but this usually results in the overall experience feeling less smooth when you rotate your head.

 

(Any time the relative 6DOF velocity (dy/dx) is high, the distortion grows because the reprojection algorithm must account for the larger difference, which results in the viewer seeing things as “warped” or “torn”. That is why an object close to your aircraft (a tree, .. another plane in formation, etc…) is noticeably degraded while something in the vast distance (a mountain) appears to be unaffected. The direct mathematical reason for this is the coordinate transform projection.)

 

I hope this description helps those who are struggling to understand how AR affects a smooth user experience. Also, if you’re looking to make the splash into VR, this is a rather large issue that you should be aware of. I implore you to road-test your chosen HMD in person before outlaying any monies, I certainly wish I had! Note that watching a through the lens video or hearing other people swear that they don’t notice it doesn’t mean that you will not (lucky you if you don’t!). I can’t speak to which version of AR is superior, be it Oculus’ ASW, or SteamVR’s Motion Smoothing, as I’ve only seen it thru the Reverb Pro, but I would love to hear the opinions of those who can.

 

(Up until about 2018, the code from Oculus was superior to Valve. Gradually, Valve reworked their algorithms (some say, reverse engineered Oculus’) and they are now generally seen as identical.)

 

If you notice any erroneous statements, please feel free to correct me. And don’t take it personally if you love the VR experience even with AR on; I’m not attacking you at all. I actually envy you, and hope to join you for a great VR experience when the 3090 releases.

Edited September 11, 2020 by GunSlingerAUS

[TOYKILLA]

Why you Post so mutch and ever and ever again in this Forum, about the same issue?

 

With the 3090 the performance should be better and ED is still working on Vulkan in the background, to make our VR Experience better.

 

Gesendet von meinem SM-G980F mit Tapatalk

[GunSlingerAUS]

Why you Post so mutch and ever and ever again in this Forum, about the same issue?

 

With the 3090 the performance should be better and ED is still working on Vulkan in the background, to make our VR Experience better.

 

Gesendet von meinem SM-G980F mit Tapatalk

 

Thanks for your great feedback on my post. With the Reverb G2 on the way, a lot of players are about to encounter Async Reprojection for the first time. I thought it would be good for them to have an informed idea of what to expect, spelt out clearly and accurately.

[TOYKILLA]

I also have my G2 Preorder running and can't wait to get my Hands on, but for now,the G1 makes also a good Job here as you know.

 

Gesendet von meinem SM-G980F mit Tapatalk

[Lange_666]

The whole thing to this is: can you live with AR or not?

I didn't know what it was before i bought my Rift S and had a bit of a WTF moment when i saw ghosting in action for the first time. But it doesn't bother me as much as it does for some people (you seem to be one of those), it's all individual.

So to be short: i can live with it (a lot better then with stuttering which i find a real showstopper).

[GunSlingerAUS]

The whole thing to this is: can you live with AR or not?

I didn't know what it was before i bought my Rift S and had a bit of a WTF moment when i saw ghosting in action for the first time. But it doesn't bother me as much as it does for some people (you seem to be one of those), it's all individual.

So to be short: i can live with it (a lot better then with stuttering which i find a real showstopper).

 

I'm surprised you get it on the Rift S - I thought it was low enough resolution that you could run at the same framerate as the HMD's refresh rate?

[Lt_Jaeger]

But anyhow, some people claim that they are not using AR and experience frame rates my kinda beefy system can't. That drives one, me at least, nuts and one starts to fiddel with the settings, in my case to no avail. There are so many rumors out there about achieving great results in dcs that one start to think too be to stupid to configure vr right (at least I do). At the end it all shows that the guys with the superb frame rates just sacrifi so much clarity and "eye candy" in order to avoid AR that, in my eyes, it's not worth it. Yes, AR bothers me, but all the sacrifices bother me more.

 

So thanx for the explanation and we will see if throwing more money at the system (replacing the 2080ti with a 3090) will rectifiers some of that. I will wait this time for some honest reviews before open my wallet again.

 

 

Gesendet von meinem LYA-L29 mit Tapatalk

[Lange_666]

I'm surprised you get it on the Rift S - I thought it was low enough resolution that you could run at the same framerate as the HMD's refresh rate?

To not getting it i need a constant 80 FPS, everywhere and on all occations, ain't gonna happen, at least not in VR (to much sacrifies as Lt Jeager points out). Capping it at 40 causes stutter, so AR (ASW) is the other option.

[some1]

You can also forcefully disable AR by config tweaks in most platforms, but this usually results in the overall experience feeling less smooth when you rotate your head.

 

Not rotate. Move sideways. Or when you look at something that moves sideways, like the terrain.

 

Smoothing head rotation is handled by lower-level smoothing algorithm that is always enabled, something Oculus calls Asynchronous Time Warp (ATW) but other companies use something similar. On the other hand, reprojection (or ASW) smooths also translational movement.

 

That's why reprojection on/off look basically the same if you just sit in the cockpit and pan your head around. But if you start moving the head sideways, without reprojection the instruments will blur.

[eaglecash867]

:yawn:

[tn_prvteye]

I just turn it off on my Rift. Is this not an option on the Reverb?

[RedeyeStorm]

I turn it of in my Index as I did with my old Rift. But then I am quit satisfied with 45 FPS and 30 FPS on the ground at take off. I do not have a beefy system but also more a SP player. My system is a Intel 6850k oc’ed tp 4.3 and hyperthreading disabled, 16 mb ram, 1070 video card.

[eaglecash867]

I just turn it off on my Rift. Is this not an option on the Reverb?

 

Its off by default on the Reverb. You have to turn it on in SteamVR for WMR. This whole thing boils down to personal preference and graphics settings, and I'm happy with how mine is set up with reprojection turned on. Everything has its limitations, and we have to work within those limitations, otherwise we have issues, so this is not any different than any other piece of computer hardware. Push it beyond its limits for the given technology combination, and it is going to give you problems.

[SharpeXB]

With the 3090 the performance should be better and ED is still working on Vulkan in the background, to make our VR Experience better.

I don’t think either of those will be a magic bullet. Performance in DCS and especially in VR is CPU bound, so a more powerful graphics card will not help.

Another flight sim which just updated to Vulkan shows no performance benefit for me. This may vary with hardware but it’s probably not a magic solution either.

[TOYKILLA]

I can Tell you on Releaseday, if the 3090 will push my overstocked 9900K ,to run DCS with the Reverb more Smoothly. But otherwise, I never will fly DCS on a Flat-screen, so I have to live with Motionreprojection, if nothing helps and I'm fine with it. :-)

 

Gesendet von meinem SM-G980F mit Tapatalk

[jaylw314]

For whatever reason, I think it's much more noticeable in dim or dark environments (like night flying) than in bright environments

[SharpeXB]

Certainly AR is something that’s not meant to run continually in games but it’s a stopgap to prevent the occasional stutter or drop which would be sickness inducing. DCS is simply too demanding for VR so it needs AR continually.

The only way for DCS or any game to run flawlessly in VR is for the game to be designed specifically for it. With lower poly counts, draw distances and numbers of objects. Lower graphics effects etc. That would require a separate VR version of DCS.

[imacken]

I don’t think either of those will be a magic bullet. Performance in DCS and especially in VR is CPU bound, so a more powerful graphics card will not help.

 

Why do some people keep saying this over and over. DCS IS NOT CPU bound. It totally depends on your hardware and settings in-game.

 

My DCS is GPU bound 90% of the time, and I am looking forward to my 3090!

[imacken]

Not rotate. Move sideways. Or when you look at something that moves sideways, like the terrain.

 

Smoothing head rotation is handled by lower-level smoothing algorithm that is always enabled, something Oculus calls Asynchronous Time Warp (ATW) but other companies use something similar. On the other hand, reprojection (or ASW) smooths also translational movement.

 

That's why reprojection on/off look basically the same if you just sit in the cockpit and pan your head around. But if you start moving the head sideways, without reprojection the instruments will blur.

 

+1 to that.

 

I am very happy with motion reprojection on my Reverb G1. Solid, smooth 45fps and no ghosting or stuttering!

[Motomouse]

What is the fuzz all about?

 

I enjoy my Reverb at 90 FPS :-)

 

3600X & 1080Ti

[SharpeXB]

Why do some people keep saying this over and over. DCS IS NOT CPU bound.

Can you keep a constant 90fps over big cities and in large scenarios?

[imacken]

Can you keep a constant 90fps over big cities and in large scenarios?

 

Best not to quote out of context. The next part of my post was the most important. DCS can be CPU or GPU bound. Depends on your hardware and settings which it is.

[SharpeXB]

Best not to quote out of context. The next part of my post was the most important. DCS can be CPU or GPU bound. Depends on your hardware and settings which it is.

Right. But if you turn all the graphics to minimum can you get that FPS? If not I’m sure it’s the CPU. That’s where the limits on VR performance come from. And especially in DCS

You basically have the same system as the OP and so would experience the same issues if you are run VR

Your sig says you’re running a 4K monitor, so am I with a 2080Ti and can max every setting.

Edited September 11, 2020 by SharpeXB

[GunSlingerAUS]

Hey folks, thanks for your thoughts on this subject. Please don't turn this into a flame thread - I simply wanted to have a clear guide on the topic for newcomers to VR. To be very clear, AR is fine for some people, I'm not criticizing them at all.

 

However, I was told for months that I'd configured my PC incorrectly, which caused me much stress and wasted time; if I can save just one other VR user from having this poor experience, it was worth the time posting this information. It'd be awesome if it could be stickied, as I think a LOT of new Reverb G2 owners will find this post relevant.

[The_Nephilim]

Hey folks, thanks for your thoughts on this subject. Please don't turn this into a flame thread - I simply wanted to have a clear guide on the topic for newcomers to VR. To be very clear, AR is fine for some people, I'm not criticizing them at all.

 

However, I was told for months that I'd configured my PC incorrectly, which caused me much stress and wasted time; if I can save just one other VR user from having this poor experience, it was worth the time posting this information. It'd be awesome if it could be stickied, as I think a LOT of new Reverb G2 owners will find this post relevant.

 

 

OK, I read the OP and still unsure what you are trying to say here? I think maybe you are trying to say someone told you it was your settings and you spent countless hours adjusting and tweaking in a vain attempt to fix the problem and it turned out it was AR setting making it look that way and basically everything was fine? I understand about AR on my Samsung, they call it Motion Reprojection on Odyssey Plus basically the same thing..

 

So you say you can not fix. Well I am sure if I had your Video card I would be running at 90FPS or damn close to it all the time currently..

 

I, have a 1080GTX and found out that Steam VR Reports the wrong native resolution for a Odyssey Plus. and in turn I ran a Way higher res then I thought I was or should have been which in turn made the FPS Poor.

 

I set the One graphic setting on Auto and then went to the Per application setting and at the native res my Percentage was 66% and that was the odysseys native resolution.

 

I am pretty sure if I get a 3080 I would be right at 90fps, as I could be in most missions.. I am unable to get a 3080 cost prohibitive currently and will just need to run at Native resolution of my current headset which is 1440x1600.. IF I do crank it up to 100% yes it looks a bit clearer but runs at 45fps with MR.

 

I would rather run with motion reprojection OFF and native res of my headset then run higher rez with MR ON..

 

in Single player it really runs much better but with MP it may require MR sometimes I am still testing but for the most part as long as the FPS are at 60 or above it does not look too bad and when it is running 90FPS it is smooth as butter..

 

And for the most Part DCS is or was with me, GPU Bound.. I bought a new CPU and found out the DCS is not CPU bound with most CPU's and is GPU bound most times then not..

Edited September 12, 2020 by The_Nephilim