*Update* Meta Quest Hand Tracking & Hand controller as a cursor on index finger (NOT as Pointer) to interact with DCS world virtual cockpit

[hannibal]

***UPDATED THOUGHTS***

After trying out some things, especially trying out hand tracking on quest 3 via virtual desktop... 

asking ED if if possible to have option to set interaction of hand tracking or motion controller to have the white dot / yellow cross cursor ON THE TIP of the virtual index finger (NOT AS A POINTER) 

i find that the pointer increases the inaccuracy of the spot being pointed at, even more inaccurate on hand tracking with the headset trying to track the hands.

i find that pointctrl works great because it is keeping track of cursor on your finger, rather than a pointer.

would be great if the DCS vr hands could do the same

 

 

Hello

I am a DCS enthusist and a PointCtrl user

i find that the PointCtrl finger tracker devices have allowed me to descently interact with the DCS world virtual cockpit

I like that the IR tracked finger controllers will move the in game cursor and i can press a button to perform a function, with PointCtrl substituting as mouse control.

as a VR user, i also have the meta quest touch controller, and have the hand controllers enable in game 

I find that performing functions with the hand controllers to be cumbersome.. pulling the hand controller trigger displaces the pointer, or trying to physcially rotate the hand controller up-side down to reach somewhere awkward in cockpit makes it diffifuclt to perform a function.

for me PointCtrl just works.

I ask Eagle dynamics if we can have the best features of both PointCtrl and VR motion hand controller implemented so that VR users like me can more intuitively interact with DCS world high fidelity modules!

what am i asking specifically?

VR hand controllers: have the pointer (OR THE TIP of the INDEX FINGER) of the in game hand controller function like the normal mouse cursor (as it can be the white dot or yellow cross). in addition to this, have UIlayer controllers to map a "left-click", "right-click", middle click, srcoll up and scroll down function..  i feel it would be more convienent to, for example, i am using my right motion controller  with my right hand to point at a switch, and with my left hand using my HOTAS or some kind of joystick to press a button, and if there was a "left-click" assignement mapped to my joystick in which i can press with my left hand.

VR hand tracking: ***EVEN BETTER*** as mention above, with avent of VR headsets to have hand tracking feature, it would be great that a pointer, or the tip of the index finger on the in game virtual hand would function like the normal mouse cursor... and left click, right click and so on facilitated by a joystick, should DCS World Control settings under UIlayer to have  "left-click", "right-click", middle click, srcoll up and scroll down button assignment..

I THINK THIS WOULD BE VERY FUNCTIONAL and DOABLE.... and allow DCS world pilots to better interact with the amazing DCS full fidelity modules..

 

 

Edited October 13 by hannibal

[nessuno0505]

I have a Quest 3 and a PointCTRL and I think quest 3 finger tracking is too imprecise to work with tiny switches like the ones in cockpit. I'd prefer an improved tracking with pointCTRL (a pointCTRL v2 with a better sensor maybe?).

[hannibal]

ok, i tried to see what was possible in DCS world.

i found that there are mappings for left click right click and scroll assignable to joysticks.

with a VR motion controller, it fairly works.. i havent tried the scroll function.

when it comes to interacting with levers, i find that left click isnt grabbing or holding on to the lever...possible that the joystick input is pulsing instead of a un-interruped input 

unfortunately, meta link app did not pass hand tracking over to PC..so when i am in the PC meta link home environment, i do not see the hand tracking... i have to see if hand tracking works over virtual desktop

[mkel2010]

14 hours ago, hannibal said:

 

unfortunately, meta link app did not pass hand tracking over to PC..so when i am in the PC meta link home environment, i do not see the hand tracking... i have to see if hand tracking works over virtual desktop

Hand tracking most definitely works with Virtual Desktop. I use an add on called HTCC (link below) which allows me to interact with all the buttons and switches in the cockpit - although it sometimes is a bit finicky. It also works with PointCTRL as well.

https://htcc.fredemmott.com

[hannibal]

11 minutes ago, mkel2010 said:

Hand tracking most definitely works with Virtual Desktop. I use an add on called HTCC (link below) which allows me to interact with all the buttons and switches in the cockpit - although it sometimes is a bit finicky. It also works with PointCTRL as well.

https://htcc.fredemmott.com

Sounds great!! Im going to take a look. 

[mkel2010]

2 minutes ago, hannibal said:

Sounds great!! Im going to take a look. 

I also use Voice Attack and Viacom Pro in VR. Over time, I've created about 375 Voice Attack commands for the F-16 that I commonly use in place of activating switches and buttons.

[hannibal]

if the virtual desktop thing actually is functional to have hand tracking, i am thinking of chopping up a mouse just to have native mouse control along with the had tracking.

i just want to use something that is functional, and demostrated it.

not everyone has pointctrl... though i as lucky to acquire two sets. and it works great. 

 

[hannibal]

ok so i just tried out hand tracking on quest 3. pretty cool, but still janky to use

update first post, asking if ED can have the point of cockpit interaction on the index finger rather than as a pointer

[Dogmanbird]

the q3 hand tracking is pretty noisy and they've used a lot of averaging to try and stabilise it, resulting in lag. I think you'd be pretty disappointed if you could try  it with the crosshair or pointer attached to the finger 

the leapmotion device is about as good as I've seen for finger tip control, but even that has it's own caveats due to problems with reflections, shadows and any nearby cockpit objects affecting it's performance 

The HTCC app and Q3 is a reasonable compromise once it's setup well

[sleighzy]

On 10/12/2025 at 6:06 AM, hannibal said:

I find that performing functions with the hand controllers to be cumbersome.. pulling the hand controller trigger displaces the pointer, or trying to physcially rotate the hand controller up-side down

Not sure if it helps but you don’t have to pull the trigger, or rotate the controller. If you just push the joystick on the controller up it does a left click, and down is a right click. Moving the joystick on the controller right and left turns dials as well. 

[hannibal]

4 hours ago, sleighzy said:

Not sure if it helps but you don’t have to pull the trigger, or rotate the controller. If you just push the joystick on the controller up it does a left click, and down is a right click. Moving the joystick on the controller right and left turns dials as well. 

yes, this is already understood! thanks!

On 10/27/2025 at 7:53 AM, Dogmanbird said:

the q3 hand tracking is pretty noisy and they've used a lot of averaging to try and stabilise it, resulting in lag. I think you'd be pretty disappointed if you could try  it with the crosshair or pointer attached to the finger 

the leapmotion device is about as good as I've seen for finger tip control, but even that has it's own caveats due to problems with reflections, shadows and any nearby cockpit objects affecting it's performance 

The HTCC app and Q3 is a reasonable compromise once it's setup well

is there any links or videos about the HTCC and Q3 setup? 

hows it going @Dogmanbird! are you still using your doorgun joystick?

[Dogmanbird]

Hi @hannibal, I'm good thanks and hope you are too!

I had to poach the parts to use on another project (I'm bad for that) but I plan to build a new version of it   Are you still using yours?

I recall seeing a youtube video on how to use an older version, but I think the latest version might have had some of the older features removed from it's interface, unless I've overlooked something. There was once a tablet behaviour mode as sell as a general controller mode, but I can't see that in the version I've just updated to.

Although I've tried both modes, I prefer the controller mode (if that's what it was called).  There are registry settings you can tweak to alter the distance from hand to headset to horizon. Once you get this set right, there's little movement of the cursor when moving your head around and you can literally move switches while looking about. It takes some fiddling to get it right. I haven't quite got the latest version working as well as an older version yet.

If you haven't already, check this link and scroll down to "VR Offset" and have a go at tweaking until you get results you're happiest with.  https://htcc.fredemmott.com/settings.html

HTCC introduces a little bit of cursor lag but I still manage to use it okay. Let us know how it goes