Leap Support!!!

[Ala12Rv-Tundra]

Does Leap motion work with all DCS modules?

[edmuss]

48 minutes ago, Ala12Rv-Tundra said:

Does Leap motion work with all DCS modules?

I believe that the gloves will show up in any module, however I think it's only in ED modules that the cockpit interaction is implemented. I only have A10C 2 and BS 2 and it works perfectly in both.

I could be wrong though, it could be that the implementation has been carried across by the 3rd party devs.

[A.F.]

vor 12 Stunden schrieb Ala12Rv-Tundra:

Does Leap motion work with all DCS modules?

I wrote to ED about this.
They said that the API is available to all, but it is up to the companies to implement Leap motion.

Heartblur writes that they are not thinking about it at the moment, but maybe in the future.

[Ala12Rv-Tundra]

I guess 3rd party devs need some time to catch up

[edmuss]

I guess it should 'only' be a case of mapping a 3D coordinate to each control node point out similar. Probably completely wrong with that guess though

At least the toolkit is there, with the recent improvements to leap motion within DCS they will hopefully be more uptake and increased demand to get the 3rd parties moving on it. I only own ED modules (which I don't have enough time to learn properly!), so chances of me using others in the near future is remote!

[bjornvil]

Hey guys! I've been aware of Leap Motion for a while now, but only recently became interested in trying it. I have a numpad mounted on my Monstertech stick mount as a makeshift Hornet UFC, which works quite well even thought it's not in the correct position relative to the actual UFC. I was going to get a pair of the Thrustmaster MFDs to mount beside the numpad and with that I would not need to use my mouse in VR for the most part, apart from flicking some switches if needed. However I've seen that the Leap Motion controller is not very expensive, about the same as a pair of MFDs and I reckon if the Leap Motion works well to manipulate the UFC and DDIs in the Hornet, everything else is a bonus. 

I would greatly appreciate any feedback you guys can share! From what I can gather there's been some developments recently which has improved the experience in DCS. Is it solid enough to go for the Leap Motion rather than the MFDs? How is setting up the LM Controller with Oculus Quest 2? Is the cable restrictive? Can you feel a difference now that there are two cables running from the headset?

[Sielu]

2 hours ago, bjornvil said:

Hey guys! I've been aware of Leap Motion for a while now, but only recently became interested in trying it. I have a numpad mounted on my Monstertech stick mount as a makeshift Hornet UFC, which works quite well even thought it's not in the correct position relative to the actual UFC. I was going to get a pair of the Thrustmaster MFDs to mount beside the numpad and with that I would not need to use my mouse in VR for the most part, apart from flicking some switches if needed. However I've seen that the Leap Motion controller is not very expensive, about the same as a pair of MFDs and I reckon if the Leap Motion works well to manipulate the UFC and DDIs in the Hornet, everything else is a bonus. 

I'd say go for it.  The native leap support right now still leaves a lot to be desired for me, although YMMV and just manipulating UFC and DDI may be fine.  I wasn't nearly satisfied with the out-of-the-box support, but I've had tremendous luck with the app @frenzon built: 

This approach requires you have to either make some custom buttons to attach to your hands/fingers, or map right click, left click, and possibly mouse scroll to your HOTAS.  That being said it's much, much more reliable than the basic leap support and--with a bit of extra work--extremely immersive.  Especially if, like many of us, you're on the waitlist for a PointCTRL this can help bridge the gap!

[edmuss]

It's not going to be as quick and tactile as pressing a physical button, but then if you can't see the physical button then you're working on muscle memory to get the right one each time.  I don't think trying to produce a AR solution with the leap motion because getting the scaling and motion to line up would be a nightmare.  Going purely leap motion would be the way to go I think, it also means you're not tied to one physical setup i.e: F18 vs A10 UFC.  For the cost of a leap controller it's worth a shot, I paid about 30 quid for mine from ebay.

I can't comment on the hornet useablilty (as I've never flown it) but I find it's very good in the A10 and KA50, from my observations the hornet UFC is massive so should be easy to get all the controls.  OSBs on the MFDs are really easy to hit, I struggle with some of the toggle switches on panels to the sides but that's more of a physical constraint of my setup.

The additional cable does add weight to the already hefty G2 cable but I don't really notice it as I'm seated and the cables are cleated to the wall and hanging next to me; I do fold a loop of cable onto my lap so that it doesn't drag over my right arm on the centre stick, I can feel the weight of the cable on my belly but it doesn't put any additional restriction of moving my head.

As pointed out, it's not perfect yet but getting better with each iteration

edit: For the controls that you can't get with the leap controller then binding mouse functions to the hotas and using VR mouse slaved to the headset view works perfectly.

Note that if you have any stick input applied you can't use your right hand and similarly for the throttle - there is a timeout from the last control input that has to elapse before the leap hands will activate, sometimes this can be a bit bugged if you have the tiniest amount of stick input.  I have no issue reaching my left hand over to the right hand side though.

Edited January 27, 2022 by edmuss

[bjornvil]

9 hours ago, Sielu said:

I'd say go for it.  The native leap support right now still leaves a lot to be desired for me, although YMMV and just manipulating UFC and DDI may be fine.  I wasn't nearly satisfied with the out-of-the-box support, but I've had tremendous luck with the app @frenzon built: 

This approach requires you have to either make some custom buttons to attach to your hands/fingers, or map right click, left click, and possibly mouse scroll to your HOTAS.  That being said it's much, much more reliable than the basic leap support and--with a bit of extra work--extremely immersive.  Especially if, like many of us, you're on the waitlist for a PointCTRL this can help bridge the gap!

 

9 hours ago, edmuss said:

It's not going to be as quick and tactile as pressing a physical button, but then if you can't see the physical button then you're working on muscle memory to get the right one each time.  I don't think trying to produce a AR solution with the leap motion because getting the scaling and motion to line up would be a nightmare.  Going purely leap motion would be the way to go I think, it also means you're not tied to one physical setup i.e: F18 vs A10 UFC.  For the cost of a leap controller it's worth a shot, I paid about 30 quid for mine from ebay.

I can't comment on the hornet useablilty (as I've never flown it) but I find it's very good in the A10 and KA50, from my observations the hornet UFC is massive so should be easy to get all the controls.  OSBs on the MFDs are really easy to hit, I struggle with some of the toggle switches on panels to the sides but that's more of a physical constraint of my setup.

The additional cable does add weight to the already hefty G2 cable but I don't really notice it as I'm seated and the cables are cleated to the wall and hanging next to me; I do fold a loop of cable onto my lap so that it doesn't drag over my right arm on the centre stick, I can feel the weight of the cable on my belly but it doesn't put any additional restriction of moving my head.

As pointed out, it's not perfect yet but getting better with each iteration

edit: For the controls that you can't get with the leap controller then binding mouse functions to the hotas and using VR mouse slaved to the headset view works perfectly.

Note that if you have any stick input applied you can't use your right hand and similarly for the throttle - there is a timeout from the last control input that has to elapse before the leap hands will activate, sometimes this can be a bit bugged if you have the tiniest amount of stick input.  I have no issue reaching my left hand over to the right hand side though.

 

Thanks guys! I actually got a hold of a controller already and I'm trying to download the software, but I'm unable to make an account with Ultraleap for some reason, the sign up page is bugged. I can't get through the sign up process. Any way to download the software without an account?

[edmuss]

Ask and ye shall receive

https://drive.google.com/file/d/1hFhbx9rSQ6kZbCiepv1s-DqBV8OtHA1a/view?usp=drivesdk

edit: updated sharing on the file, hopefully you will be able to download it now!

Edited January 28, 2022 by edmuss

[Sielu]

2 hours ago, bjornvil said:

Thanks guys! I actually got a hold of a controller already and I'm trying to download the software, but I'm unable to make an account with Ultraleap for some reason, the sign up page is bugged. I can't get through the sign up process. Any way to download the software without an account?

Yeah I had a hell of a time too.. turns out the Pi-Hole I had on my network (which blocks trackers) was blocking the login server.  I disabled the Pi-Hole temporarily and was able to get in... if @edmuss's download link doesn't work, try disabling whatever VPN/Ad-blockers/etc. you have.

[A.F.]

Hi,
I would like to briefly share my ideas here regarding the mounting of Leap Motion Controller on the Hp Reverb G2 and the possibility - despite the lack of buttons and switches - to still have the feeling of pressing a button.

For the mounting of the Leap Motion Controller on the Hp Reverb I use "Worbla". This material can be heated with a heat gun and then bent into a desired shape. When it cools down, the shape remains and the material becomes solid again.
This way you can also glue two layers of Worbla together to give the material more strength. (See photos) 

To press a switch, I don't use real switches, but soft foam, which I glue on a hard foam plate. Here for the Viper.
This way I am not forced to find exactly the point (in height, right/left flush in depth), where to find the button in the VR cockpit. Nevertheless, I have the feeling of pressing a button.

It's a bit difficult to find the right distances while wearing the HMD.
But with the help of my lovely wife it worked
I point where something is in VR and she draws a line or says the distances.

Maybe this will give you some ideas of your own

 

Photos.zip

[edmuss]

I really like the foam idea with the mock up cockpit, how variable is your chair position relative and does it change the reach distance much if you reset you headset position?

I have previously used the 3D printed mount downloaded from ultraleap along with a bracket to suit the curved face of the G2 from thingiverse.

I'm currently trying out with a top mount consisting of a bit of folded aluminium plate, 3M VHB tape to hold the mount to the plate and then some heavy duty velcro to stick it to the headset.  I tried using the VHB directly onto the leap motion casing but the heat caused it to lose hold quite quickly.

Edited January 30, 2022 by edmuss

[Harlikwin]

Can someone give me a quick rundown of this in DCS. is it like usable in all modules, does it work reliably/decently, do you see your hands in the pit?

I'm currently using point control, but always on the lookout for something better.

[edmuss]

You see a pair of disembodied flight gloves in the pit, fully tracking* what your hands are doing. You can press, twist and flick any controls that you could normally interact with the mouse.

By default, only ED modules are fully mapped for leap support but the toolset is available for 3rd party developers to add it to their own pits. I don't know how many of the 3rd party pits are compatible.

There are still some bugs to iron out but it's actually really quite useable although obviously not as quick as a mouse or physical button to press because there's no physical feedback. Also some things in the play area can cause tracking to become obscured or confused but that's fairly easily sorted.

*there are a couple of tracking issues but they're not game breaking. Things like thumb and forefinger not touching on an ok sign.

 

 

[Harlikwin]

5 hours ago, edmuss said:

You see a pair of disembodied flight gloves in the pit, fully tracking* what your hands are doing. You can press, twist and flick any controls that you could normally interact with the mouse.

By default, only ED modules are fully mapped for leap support but the toolset is available for 3rd party developers to add it to their own pits. I don't know how many of the 3rd party pits are compatible.

There are still some bugs to iron out but it's actually really quite useable although obviously not as quick as a mouse or physical button to press because there's no physical feedback. Also some things in the play area can cause tracking to become obscured or confused but that's fairly easily sorted.

*there are a couple of tracking issues but they're not game breaking. Things like thumb and forefinger not touching on an ok sign.

 

 

I am tempted. But so far point control is 100%...

Where is a good place to get leap kit?

 

[A.F.]

vor 16 Stunden schrieb edmuss:

I really like the foam idea with the mock up cockpit, how variable is your chair position relative and does it change the reach distance much if you reset you headset position?

 

I am sitting in a car seat.
I can push it forward and backward.
If my virtual fingers can't reach the buttons, I can use "Move cockpit view forward" and a keybinding to move closer to the control buttons. 
That's the first thing I do when I go into the HMD.
I will still paint the cockpit mockup black. That way I will have a better tracking.

The Leap Motion I had also tried to attach via Velcro. Also with me the devices were too hot and the tape did not hold.

Note that the virtual hands are tracked in relation to the headset. It could be that the hands are displayed in the wrong position due to the high position on the HMD.

[edmuss]

39 minutes ago, A.F. said:

I am sitting in a car seat.
I can push it forward and backward.
If my virtual fingers can't reach the buttons, I can use "Move cockpit view forward" and a keybinding to move closer to the control buttons. 
That's the first thing I do when I go into the HMD.
I will still paint the cockpit mockup black. That way I will have a better tracking.

The Leap Motion I had also tried to attach via Velcro. Also with me the devices were too hot and the tape did not hold.

Note that the virtual hands are tracked in relation to the headset. It could be that the hands are displayed in the wrong position due to the high position on the HMD.

Good thinking with the move cockpit view, on the fly adjustment!  My car bucket seat is hacked onto a gas lift, tilting office chair base so the seat position isn't quite as rigidly set; I can tip the seat back to avoid physical objects such as the wheel servo base, gearshift and desk edge but by it's nature the seat position is not very regular.  Before you paint the mockup, make sure that you test the paint to ensure that doesn't reflect the IR light, if it does then your tracking will go to pot.  Open the ultraleap visualizer and see if the paint is reflecting white (bad) or black (good).  I used 3mm self adhesive closed cell foam to sheet over the reflective surfaces (wooden desk and painted wall mainly) and my tracking became infinitely better.

The tracked position is independant of the centre of head rotation and that isn't factored into the tracking (Dan @ ultraleap confirmed this to me).  So it can be above/below/front of the headset and still track the same.  There is a little tracking occlusion because I've set it back from the front edge but that just means that the hands hide more reliably when I'm hotas and not eyes down; when eyes down or using the UFC they retrack without any issue, saves the hands tracking badly on the edge of the tracking envelope and jittering around the stick in your peripheral vision.

 

6 hours ago, Harlikwin said:

I am tempted. But so far point control is 100%...

Where is a good place to get leap kit?

 

I would suggest that point control might be quicker and more accurate but I do like the fact that there is no additional button pressing to activate the switches.

I got my leapmotion off ebay for 30 quid

Edited January 31, 2022 by edmuss

[Sielu]

15 hours ago, Harlikwin said:

I am tempted. But so far point control is 100%...

Where is a good place to get leap kit?

 

If you already have PointCTRL, I would stick with that.  I see a few fundamental barriers to native, 3D leap support that PointCTRL and other hand-to-2D Plane solutions don't have:

1. No feedback: gestures are great but having tactile feedback, even if it's just a button push against the side of your finger is REALLY nice.  From what I understand it's one of the complaints of actual F35/F22 drivers that the touch screens they have don't give that tactile "confirmation" of a button push.  I think LEAP is the same way, even if you're looking right at the switch it's nice feeling the actuation.

2. Module support: as has been mentioned, 3rd party modules have the option of implementing LEAP, but to my knowledge none have so far.  Anything that works with a mouse cursor will work with PCTRL though.

3. Clearance: This, I'm realizing, is the biggest issue for me.  I regularly fly the Viper and the Hog, and I'm starting to dabble in the Hornet as well.  These three have different clearance requirements depending on what you're doing.  Since PointCTRL maps your hand to a 2D plane, you don't have to actually put your hand at the correct distance to actuate a switch... whereas with a true 3D solution you need to have full clearance to all the buttons and switches you want to actuate.  Building a replica cockpit, as have some folks above, is an option but given you fly a variety of aircraft, this could pose a challenge.  

 

[Harlikwin]

48 minutes ago, Sielu said:

If you already have PointCTRL, I would stick with that.  I see a few fundamental barriers to native, 3D leap support that PointCTRL and other hand-to-2D Plane solutions don't have:

1. No feedback: gestures are great but having tactile feedback, even if it's just a button push against the side of your finger is REALLY nice.  From what I understand it's one of the complaints of actual F35/F22 drivers that the touch screens they have don't give that tactile "confirmation" of a button push.  I think LEAP is the same way, even if you're looking right at the switch it's nice feeling the actuation.

2. Module support: as has been mentioned, 3rd party modules have the option of implementing LEAP, but to my knowledge none have so far.  Anything that works with a mouse cursor will work with PCTRL though.

3. Clearance: This, I'm realizing, is the biggest issue for me.  I regularly fly the Viper and the Hog, and I'm starting to dabble in the Hornet as well.  These three have different clearance requirements depending on what you're doing.  Since PointCTRL maps your hand to a 2D plane, you don't have to actually put your hand at the correct distance to actuate a switch... whereas with a true 3D solution you need to have full clearance to all the buttons and switches you want to actuate.  Building a replica cockpit, as have some folks above, is an option but given you fly a variety of aircraft, this could pose a challenge.  

 

All good points, thank you, and yeah didn't think of the clearance issue. I fly with a desk+mounts, so that might be challenging.

[Trufret]

I just picked up a leap motion after checking out some videos but am seeing some odd behaviour. Did anything change in the last OB patch as what i saw in the videos of people using it (just touching buttons with the virtual hands) is not what I'm experiencing?

I'm currently on 2.7.10.18996 Open Beta

I've done a Cleanup and repair with the DCS updater tool

I've tried the Gemini 5.2.0 and 5.3.1 versions with the same results.

Obviously enable is selected under Special >leap

For the most part i only have a left hand, I can get a right hand but the only way is to enable the allow use of stick checkbox. No other setting will show a right hand.

I have show arm enabled as without it I only have a pointer appearing out of nowhere (that makes sense)

I usually have Pointers set to none which removes the pointer laser but then I cannot interact with anything in the cockpit. virtual hands just go through the model with no interaction.

If i set pointers to left hand, right hand, or both I only see the left hand but there is a pointer coming off my index finder about 30 degrees to the right. if I move my hand close to the cockpit buttons I can press them without having to worry about the mouse click setting of index bend or thumb pressure but the action point is at a point along the axis of the pointer laser and not on my finger tip.

trying to use the pointer from further away (as a pointer instead of touch interaction) via index bends on thumb movements has no effect.

I've tried the F-18, A10CII, and F-16 and all have the same behaviour.

Is anyone else using leap experiencing this?

 

[edmuss]

I think to get the right hand to be more reliably shown you might need to give the stick a tiny waggle, if there is any control input on it DCS considers you to be hotas so hides the hand (this is assuming that you have the stick/throttle hand interaction turned off). Try setting a massive stick dead zone to see if that helps to alleviate it, also try with vr mouse enabled.

The settings are bugged at this time, they need to be reset each time you restart DCS and in order to set the pointer condition you need to set a different option and then set what you want.

The hands will clip through the cockpit and buttons. I find that there's a certain knack to getting the finger in the right place before pressing, sometimes it helps to lean inline with the button so that the parallax errors are minimised. I quite quickly developed muscle memory to lean to position the finger and then make a distinct actuation; it is slower than pushing a physical button though.

I have best results using a "gun finger" hand pose to keep the single fingertip out on its own to help with tracking the point. Make sure that the fingers are actually still tracking movement and it's not frozen the fingers in place, at which point I don't think the hand can interact with the cockpit.

Finally check that you have minimal IR reflective surfaces near where you press controls. Open the ultraleap visualiser and look at the background behind your hands, if it's white then it's reflecting the IR and the leap motion will have a harder time spotting the outline of the hand. Ideally you want white hands on a black background. I used closed cell foam sheet to absorb the background IR and it made the tracking infinitely better

Edited February 3, 2022 by edmuss

[A.F.]

vor 5 Stunden schrieb edmuss:

I think to get the right hand to be more reliably shown you might need to give the stick a tiny waggle, if there is any control input on it DCS considers you to be hotas so hides the hand (this is assuming that you have the stick/throttle hand interaction turned off). Try setting a massive stick dead zone to see ….

Good points!

I still have the problem that when I deactivate "Hands move stick " (I don't know what the function is called), one of my hands disappears as soon as I move the stick or throttle.
That's why I leave this function on. You just have to be careful that your real hands don't get too close to the virtual sticks and throttles in the cockpit. Otherwise the virtual hands will make the control inputs.

[edmuss]

Yeah, it's somewhat disconcerting when your pretend hands move the pretend joystick when your real hands are moving the real joystick in almost the exact same position.  Instinctly you move your hands away from the virtual stick but that's difficult whilst holding onto the physical stick

This is why I turn off the hands move stick/throttle!

The left hand is always far more robust when appearing but the controller inputs are far less likely than the stick to show a delta so tends to come back straight away.  It got better with the last but one OB patch and I expect it will get better again soon enough. I don't actually use any deadzone on my warthog but generally the stick doesn't give any input if I release it, I can also use my left hand to control most of the right hand side of the cockpit and keep input on the stick with my right hand.

Perhaps a workaround might be to have a bindable pause tracking toggle button, when toggled off the hands appear (when within the tracking envelope) and either hand can be used to manipulate controls regardless of hotas input, when tracking is paused then the hands are simply hidden.  This would allow you to hold a stick input with the left hand and use the right to manipulate the right MFD for example.

[A.F.]

I just tried it again: if I deactivate the option "Hands move the stick/throttle, my virtual left hand disappears.
It does not come back then either.
So I have to activate the option "Hands move the stick/throttle" - but I have to be careful that the virtual hands don't grab the stick/throttle.
When I move my head back and forth, my virtual hands move as well, which causes the stick/throttle to move - but I don't want that.

This still seems to be a bug - I hope anyway