VR cursor and HMS symbology jitter

[St4rgun]

Several users observed a HMS symbology jitter, which is most conspicuous with the IHADDS of the Apache.

It turned out, that this phenomenon is not only HMS related, it's the same with the VR cursor.

The head tracking of the VR headsets is so precise, that even my pulse can move the crosshairs altough I hold my head very stable. If someone only want to ocassionaly use the crosshairs then it's fine, but as long as you are starting to be aware of this problem and trying to observe these fine details then it will be obvious that something is basically wrong with the HMS crosshairs movement (including all helmet sights of the A-10, F-16, F/A-18, Ka-50 and the Apache as well).

The same goes with the mouse VR crosshairs: because the VR cursor is linked to your head so it also moves this way. The slightest movement (pulse) can jitter it around while trying to fight it with the mouse. It's more emphasized in the Apache when typing a lot of data on the keyboard.

The main cause can be the method, that the movement of the HMD sensed at position of the head, while the movement of the VR cursor and HMS symbology placed in the space further in a distance which means a given angular displacement of the head makes fairly large displacement in space at the distance of the displayed VR cursor or HMS symbology. IRL this would not happen, because the HMS placed close to the eye, just collimated to infinity.

IMHO this issue is serious enough to be corrected to eliminate fatigue caused by the need of unnatural head fixing meanwhile improving the aim experience with the crosshairs.

I think some deadzone solution for the head tracking is generally not appropriate, because sometimes we want to move the crosshairs precisely, so a deadzone would introduce some artificial lag. Probably filtering the jitter of the HMDs head tracker sensor raw data is the solution to go. But these all should be applied only to HMD symbology and VR cursor, in every other aspect the head tracking is perfect now.

@BIGNEWY Any toughts on this?

[BIGNEWY]

Personally I think it would lead to more issues if a deadzone or artificial stabilisation was used, but I will pass on your comments to the team. 

[5ephir0th]

I don´t think a deadzone would be the correct fix but some kind of smoothing for the head movement, with a slider to adjust it

[Baldrick33]

I agree, some form of damping or smoothing with a slider to set as an option would be a great addition.

[St4rgun]

On 4/29/2022 at 2:05 PM, BIGNEWY said:

Personally I think it would lead to more issues if a deadzone or artificial stabilisation was used, but I will pass on your comments to the team. 

Maybe two different methods could be used for the different tasks:

• VR cursor stabilization: even a bigger deadzone can be highly usable for the head tracking not to interfere with precise mouse movement
• HMS symbology stabilization: the deadzone is contraproductive, special noise filtering is required instead like some tuned low-pass filter to avoid lag (i.e:https://hal.inria.fr/hal-00670496/document)

[Fuggzy]

I concur completely.  I've done more testing, comparing Apache IHAADS and Viper HMCS -- it's not aircraft specific.  It's the technology.

It works smoothly and perfectly with TrackIR, because that is a much lower resolution tracking than the VR headsets with internal gyros/whatever they do.  TrackIR can't see the constant, tiny movements of a person's head, but the Quest2 sure can feel them.  The headset and controllers are astounding at how precise they are, you cannot move them any perceptible amount in space without it being detected.  When I play in 2D with TrackIR, it works exactly like I'd expect, and it feels like the symbology is tracking my true head movements.  In VR though, I'm constantly trying to focus like a jedi to keep my head more still, and I end up deciding that head-pointing just makes things difficult.

The solution would be an app or a mod that could filter the noise through a number of schemes -- a movement threshold "limp zone", a continuous averaging of polled values, etc.  I think ED could implement a slider in the VR tab that does this in-game to dial in an amount of smoothing from none to very smooshy, and have this apply universally to the modules.  That's really the only problem I have in VR right now, too much head tracking precision.  Heh.  

 

 

[St4rgun]

To say some news for this topic: I've just tried the "Inputs" option of OpenXR Toolkit (I'm using Reverb G2 with OpenXR).

There's an option which was pretty much under my radar before: "Shaking reduction".

OMG, what a difference! I set it to -70% and EVERYTHING went smooth, without any annoying lag.

I mean REALLY smooth. Not only the crosshairs will be so much better, but the whole scene inside and outside the cockpit. WOW, it's a jaw dropping moment for me.

I highly urge all WMR usees who're using OpenXR to try this out, it's a really game changer.

Edited May 17, 2022 by St4rgun

[dburne]

2 hours ago, St4rgun said:

To say some news for this topic: I've just tried the "Inputs" option of OpenXR Toolkit (I'm using Reverb G2 with OpenXR).

There's an option which was pretty much under my radar before: "Shaking reduction".

OMG, what a difference! I set it to -70% and EVERYTHING went smooth, without any annoying lag.

I mean REALLY smooth. Not only the crosshairs will be so much better, but the whole scene inside and outside the cockpit. WOW, it's a jaw dropping moment for me.

I highly urge all WMR usees who're using OpenXR to try this out, it's a really game changer.

 

 

Nice! I can't say I notice any jitters with my Aero running through Steam VR but good to know as there are a lot of WMR users out there.

[St4rgun]

Dear @BIGNEWY, sorry to disturb.

You mentioned that it's possible to forward the ideas to the team. Based on my previous reply at least the WMR headsets CAN apply stabilization right now when using OpenXR.

15 hours ago, St4rgun said:

To say some news for this topic: I've just tried the "Inputs" option of OpenXR Toolkit (I'm using Reverb G2 with OpenXR).

There's an option which was pretty much under my radar before: "Shaking reduction".

OMG, what a difference! I set it to -70% and EVERYTHING went smooth, without any annoying lag.

I mean REALLY smooth. Not only the crosshairs will be so much better, but the whole scene inside and outside the cockpit. WOW, it's a jaw dropping moment for me.

I highly urge all WMR usees who're using OpenXR to try this out, it's a really game changer.

 

Based on that it's worth checking the existing solutions on other VR platforms as well, but it can still provide a proof that the HMD movement stabilization is not only usable, but MANDATORY for DCS. If all platforms already have proper solution then it's great, it will not need any further development.

[BIGNEWY]

Hi, again I will mention it to the team, but can not promise anything. 

thanks

[Fuggzy]

Please please I hope ED can provide a universal shakiness reducer for VR.

It is not just the IHAADS but the mouse cursor too. It is tiresome focusing my head still and wrestling the cursor with my mouse to push the intended button.  The rest of the world is smooth and no jitters, only what is virtually connected to my pilot's head is shaky.

I agree with @St4rgun ... Something akin to a head tracking noise filter is desperately needed for VR.  Not a dead zone or resolution gimper, a filter to get rid of the micro movements like the vein pulsing in my temple because of the rage. 

[St4rgun]

22 hours ago, Fuggzy said:

Please please I hope ED can provide a universal shakiness reducer for VR.

It is not just the IHAADS but the mouse cursor too. It is tiresome focusing my head still and wrestling the cursor with my mouse to push the intended button.  The rest of the world is smooth and no jitters, only what is virtually connected to my pilot's head is shaky.

I agree with @St4rgun ... Something akin to a head tracking noise filter is desperately needed for VR.  Not a dead zone or resolution gimper, a filter to get rid of the micro movements like the vein pulsing in my temple because of the rage. 

Fuggzy, if you are using WMR with OpenXR then you"re lucky too.

Check out my previous post about OpenXR Toolkit's shaking reduction feature, it's a blessing. It really makes wonders.

If using Oculus or Varjo then I can't help, maybe someone have a solution for those users too.

Edited May 25, 2022 by St4rgun

[dburne]

2 minutes ago, St4rgun said:

Fuggzy, if you are using WMR with OpenXR then you"re lucky too.

Check out my previous post about OpenXR Toolkit's shaking reduction feature, it's a blessing. It really makes wonders.

If using Oculus or Varjo then I can't help, maybe someone have a solution for those users too.

 

Not seeing this behavior in my Aero, least that I can tell.

[Fuggzy]

13 hours ago, St4rgun said:

Fuggzy, if you are using WMR with OpenXR then you"re lucky too.

Check out my previous post about OpenXR Toolkit's shaking reduction feature, it's a blessing. It really makes wonders.

If using Oculus or Varjo then I can't help, maybe someone have a solution for those users too.

 

Hi St4rgun, I got super excited about your other thread talking about your OpenXR find...  but I'm on Oculus.  And I found it interesting the parameter used to be called "Prediction".  So I researched more, lol.  The problem we're having is Prediction... the headset IS seeing tiny movements and trying to helpfully anticipate bigger movements, in an effort to cover the latency gap between a person's real head movement and virtual.  In fact, Quest Oculus Meta touts on their website how awesome their Prediction is.  It's very smart and works well, except for titles like DCS where it becomes apparent.  It just happens that the concept of HMCS/IHAADS -- a face mounted pointer with a distant sight picture, is a new thing to the VR problem with this prediction tech.  Every HMD maker needs to wise up and make this configurable.  I hope some super smart code monkey wishes to develop a 3rd party solution in the meantime.  

I did make some minor progress forward in this area by simply turning on all the lights in the room and turning off my monitor.  The hypothesis goes that this helps the tracking predict more reliably with less false movements, I think.  If it does help, it's so subtle that I admit it might be placebo, I don't know.  It seems to jitter less when the headset cameras have a stable, well-lit environment and no movement from a monitor 3 feet away.  Still not enough help though.

 

Edited May 26, 2022 by Fuggzy

[St4rgun]

It's a pity that Meta's headsets lagging behind on this field. On the Reverb G2 it is obvious that the small jitter is caused by the very fine movements (pulse mainly), but it's aggregated by the IHADSS implementation of ED as we discussed before. I hope that ED can think about some standardized solution, as @BIGNEWY also uses Reverb G2, so he can try out the existing shaking reduction of OpenXR Toolkit to convince the team about the feasibility and usefulness of the filtering.

[lefuneste01]

I never notice this kind of issue with my G2,  without using any smoothing.

[zildac]

Occasionally I do see this issue, and it can be as subtle as an elevated heartbeat/pulse causing a slight movement of the HMD. It's mainly noticeable to me at least, when stationary. I have just upped the shaking reduction to -20%, and it seems to alleviate the issue for me.

[St4rgun]

1 hour ago, zildac said:

Occasionally I do see this issue, and it can be as subtle as an elevated heartbeat/pulse causing a slight movement of the HMD. It's mainly noticeable to me at least, when stationary. I have just upped the shaking reduction to -20%, and it seems to alleviate the issue for me.

Try out the -50% or even -70% reducion. All the symbology will be silky smooth and stable in air.

[zildac]

Try out the -50% or even -70% reducion. All the symbology will be silky smooth and stable in air.

Will try it out, thanks.

[Marco Schaap]

@BIGNEWY would it also be possible to change size and color of the cursor? I woul love to be able to change the size and color of it.

[Fuggzy]

This whole problem has consumed me, I find it a very interesting problem that certainly wasn't forseen in VR world. I've figured out a way to illustrate the gist of it:

First, a fact:  VR headset motion/position tracking resolution is an order of magnitude more precise than TrackIR.  e.g. Quest2 is particularly proud of their ultra high resolution tracking, and they are not lying.

OK, let's begin the experiment.

• On a dark evening, duct tape a laser pointer to your head such that it shines directly forward. Proceed to your backyard.
• Stand at one end of the yard such that you can shine the laser dot on a flat surface as far as you can get from it.  30yds/m is enough.
• Now, hold that laser dot exactly still, using your head.  ...  It's ok, you can sit and try again.  ...  Prone? Of course, go for it. ... Dang that's impossible, IKR?

Ok now we ponder this

• Observe that while your head jitters continually, one's brain is continually stabilizing the full-world FOV.  The world around the backyard is perfectly stable, as if none of these tiny head movements exist.  It gets even crazier that our eyes are also going crazy micro-moving without ever stopping, and we have zero perception of it.  That old meatball up there is doing camera stabilization of GoPro's wildest dreams, all the time.  But the brain cannot smooth out that laser dot, it is an external metric that exposes the movement that is countered automatically out of our vision.
• The helmet mounted devices in DCS are like laser pointers, affixed to our virtual heads which are married to our high-resolution VR head trackers. If you really watch closely at the outside edge of your FOV where the mask is, you'll notice that yep your head is moving relative to the view, and it matches the IHAADS... but the freaky thing here is that your brain is making the VR world view smooth as silk exactly like it does in RL view. -- BUT it cannot mask the motion of the laser pointer, in this case the helmet symbology.

In summary, the problem here is that everything -- the VR headsets, DCS, implementation -- work perfectly, exactly as designed, and better than expected. This is a problem that I believe only dynamic focus lens headsets of the future will truly solve.  Pretty neat.  But man the jittering.  I just map a control on HOTAS to on/off and use it only for combat.  I still think the headset makers could add a slider into the headset tracking setup that detunes the tracking resolution to be in larger steps or something. But that would never happen, because in 99% of games you'd never see this phenomenon, only those where there happens to be a pointer thing attached to your virtual head in the game.

 

Edited July 9, 2022 by Fuggzy

[Dragon1-1]

How does the real aviation HMCS systems deal with that, I wonder? Because the HMCS is, in fact, a display affixed in front of your head, almost just like a VR headset. It's just as precise in its tracking as a VR headset is (it uses magnetic sensors, as opposed to IR lasers or cameras, but this system is just as good), and the symbology is just as much a "laser pointer" as it is in DCS. This might be just the way these displays are, and there's no "problem" to solve, other than the fact that real displays of that sort have their peculiarities. IRL, you usually have a hands-on switch to blank the HMCS (unless you're flying the Apache, but then, the monocle is known to take some getting used to, to say the least). 

Myself, I don't have any problems with jittering on my G2, it follows the movement of my head exactly like I'd imagine a real HMCS does. Having it in only one eye is weird, particularly since I'm somewhat ambidexterous so my brain on occasion can't quite decide if it's there or not, but it stopped bothering me. It's not stable in relation to the outside world, but neither is the real thing, since it's literally on your head, and even weighs it down in a similar way to a VR headset.

Edited July 9, 2022 by Dragon1-1

[Eldur]

I've notices the massive jitter of the IHADSS right away with the AH-64 and was wondering why noone reported it at all. It's typically the headtracker symbol that keeps jumping around constantly and as there's no such thing in other Hammickses, I never saw an issue there except for jittering datalink related things that I couldn't see on pancake footage on the Tube either. Now the real question is: Why does this only happen VR, but not with TrackIR, mouse movement or whatever?

[5ephir0th]

On 7/14/2022 at 1:25 PM, Eldur said:

I've notices the massive jitter of the IHADSS right away with the AH-64 and was wondering why noone reported it at all. It's typically the headtracker symbol that keeps jumping around constantly and as there's no such thing in other Hammickses, I never saw an issue there except for jittering datalink related things that I couldn't see on pancake footage on the Tube either. Now the real question is: Why does this only happen VR, but not with TrackIR, mouse movement or whatever?

Because TrackIR has an inbuilt smoother and it looks like VR headsets tracking has way higher resolution than TrackIR or similar

[St4rgun]

As I wrote previously the OpenXR WMR headsets (G2) can use the OpenXR Toolkit's inbuilt shaking reduction to smooth out the jitter of the IHADSS perfectly.

And this is not only IHADSS related, it also helps a LOT in general to stabilize the view. All other modules with HMDs (A-10C II, F-16, F/A-18) are night and day when this shaking reduction is switched on.