Jump to content

Leap Support!!!


Recommended Posts

On 4/2/2022 at 9:17 AM, Avalon66 said:

Hi

sorry if it has been already discussed but I can't find it through the search.

Do you think that pairing leap motion with pointctrl should be a viable option?

I was wondering to do that in order to keep having the awareness where your finger is moving and use pointctrl to act for pushing buttons and rotating knobs

 

There's no reason why it shouldn't work as long as the IR from the point control isn't thrown out by the IR of the leap motion; the leap controller casts out IR light and relies on the bounce back to identify and track the hands.  I've a feeling that the flood of IR might break point control.

A neater solution is just to have the VR mouse enabled with mouse controls bound to the hotas, that way you can use that if the leap controller doesn't work well enough.

 

On another topic, over the last couple of OB updates has anyone noticed that the rotary knobs are now easier to actuate?

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

Ok, I've pulled my leap motion off the shelf after trying to make it work many moons ago (prior to official support). I *still* have Mickey Mouse hands in SteamVR Home that I can't seem to get rid of.

Anyway, I'm using OpenXR now instead of SteamVR, and saw edmuss' signature there had leap, so I stalked him to here. 😉 So my issue right now is (aside from having to dis-re-enable Leap in the Special tab of the Options menu) is that when the hands show up and I try to touch anything, the hand refuses to. As in, balling up into a fist when I try to press a button. I'll have my index finger extended to touch one of the buttons on the Apache MPD, and as I approach it with my finger, it bends back into a fist, *and* the hand slows down and won't extend fully.  EDIT: It's like there's a force field keeping the hand/finger from interacting with the virtual buttons.

For those of you using the OpenXR add-on, are there additional adjustments that must be made in the OpenXR Toolkit Hand-to-controller Config Tool? I see offsets and gestures that can be edited, but nothing that would alter the distance that the gloves would travel.

I've disabled the mouse in VR settings, and am dealing with having the cursor or blue + visible all the time, but would like to see some improvement there. Honestly, if I could just be able to push buttons on the MFDs without issue, I'd call that a win.

Any advice?

Erik


Edited by Ainuke

i9 11900K :: 32GB G.Skill Trident DDR4-3200 :: Corsair water loop ::  Samsung 2TB 980pro nvme :: Gigabyte Aorus 3090 Waterblock :: Gigabyte Aorus Z590 Xtreme MoBo :: HP Reverb G2 VR

VKB Gunfighter flightstick and collective :: TM Warthog throttle & MFDs :: MFG Crosswind pedals :: 5 Dayton Audio bass shakers on 5.1 home theater amp

Win10 Pro x64 :: DCS 2.7 (Huey & P-51, mainly; have all the other rotors, but haven't explored their depths as yet)

Link to comment
Share on other sites

That sounds like a tracking/occlusion issue.

Leap motion relies on the hands being clearly outlined with the IR light cast out by the controller. That is to say, any background surfaces that reflect IR light will act as chaff and prevent the sensor getting a good, robust lock on the hands.

Open the ultraleap visualiser and move your hands around your rig setup, make a mental note of how many white backgrounds there are behind your hands. This is IR reflective surfaces and bad, ideally you want black with your hands only in white.

I ended up covering my keyboard tray, desk edges and the shelf/wall where my throttle is with closed cell foam sheet and it's been perfect ever since.

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

Hi,

I just received my leap motion, and I have to say that it's really easy to use ! I used to blind click on my ICP and MFD, but it's way easier now.

I just hard for me to not touch something when clicking, but it's really fine !

2 questions :

1 -how to down click on buttons ? Master arm Off for example ?

2-can I enable/disable it without going into menu ? when I loose one hand

 

THanks !

Link to comment
Share on other sites

Clicking toggle switches should be exactly as you would actuate it in real life, rest the finger above it and push downwards and below and upwards to go the other way.

Similar with rotaries, rest the finger on the top and slide it sideways. They can be more finicky though if they're smaller diameter. For example the HUD brightness knob on the KA50 is quite large and really easy to manipulate.

It does take a little practise to get the muscle memory but generally it's very good.

If you've lost a hand then it's almost certainly because you have a tiny off centre input on a controller (normally the stick), give it a wiggle to re-centre and you should get the hand back.

Glad that it's working ok though.


Edited by edmuss

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

18 hours ago, edmuss said:

That sounds like a tracking/occlusion issue.

Leap motion relies on the hands being clearly outlined with the IR light cast out by the controller. That is to say, any background surfaces that reflect IR light will act as chaff and prevent the sensor getting a good, robust lock on the hands.

Open the ultraleap visualiser and move your hands around your rig setup, make a mental note of how many white backgrounds there are behind your hands. This is IR reflective surfaces and bad, ideally you want black with your hands only in white.

I ended up covering my keyboard tray, desk edges and the shelf/wall where my throttle is with closed cell foam sheet and it's been perfect ever since.

Thanks for the help. Through the visualizer I'm getting spot-on tracking with the headmounted option. I'd hoped I could use the screentop positioning, as that would've allowed me to just have the sensor mounted above my pit, but DCS seems to expect the head mount orientation; deviation from that has the hands moving on a plane about 90 degrees off from where my hands are. But anyway, I didn't see any white or IR reflective surfaces through the visualizer; and my screen is actually 180 degrees behind my pit (which is AWESOME to have to turn to launch and app that I forgot to do before I climbed into the seat). I'll double-check again, though.

Erik

i9 11900K :: 32GB G.Skill Trident DDR4-3200 :: Corsair water loop ::  Samsung 2TB 980pro nvme :: Gigabyte Aorus 3090 Waterblock :: Gigabyte Aorus Z590 Xtreme MoBo :: HP Reverb G2 VR

VKB Gunfighter flightstick and collective :: TM Warthog throttle & MFDs :: MFG Crosswind pedals :: 5 Dayton Audio bass shakers on 5.1 home theater amp

Win10 Pro x64 :: DCS 2.7 (Huey & P-51, mainly; have all the other rotors, but haven't explored their depths as yet)

Link to comment
Share on other sites

Yeah I think DCS is only setup for head mounted sensor, desk mounted wouldn't work because of the amount of switches to the side and below; they would fall outside of the tracking envelope.

You can also mess about with the sensor position to alter the tracking envelope, mine is on top of my G2 and set back a bit which moves the envelope backwards a little at the expense of downwards vision.

This works nicely because if I'm using a control low down I'm going to be looking at it but when heads up the hands stop tracking because they're occluded by the front of the headset.

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

21 hours ago, edmuss said:

Clicking toggle switches should be exactly as you would actuate it in real life, rest the finger above it and push downwards and below and upwards to go the other way.

Similar with rotaries, rest the finger on the top and slide it sideways. They can be more finicky though if they're smaller diameter. For example the HUD brightness knob on the KA50 is quite large and really easy to manipulate.

It does take a little practise to get the muscle memory but generally it's very good.

If you've lost a hand then it's almost certainly because you have a tiny off centre input on a controller (normally the stick), give it a wiggle to re-centre and you should get the hand back.

Glad that it's working ok though.

 

Ok thanks a lot

 

is that normal that I have to disable/enable Leap each time I launch DCS ? thanks

Link to comment
Share on other sites

Unfortunately yes, you would imagine that it works be a relatively simple bug to fix but it's obviously waaay down the list 😞

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

43 minutes ago, edmuss said:

Unfortunately yes, you would imagine that it works be a relatively simple bug to fix but it's obviously waaay down the list 😞

ok

Not a big problem, just have to think about it. What is my main problem actually is that my right hand is not seen by DCS one time over 2. Left hand is really nice. I can move my right hand in front of the camera, shake it but no succes. And sometimes it works fine.

I tried with leap software in sametime and it's fine, nothing on DCS but hand on leap ;(

 

same for you ? thanks

Link to comment
Share on other sites

20 hours ago, edmuss said:

Yeah I think DCS is only setup for head mounted sensor, desk mounted wouldn't work because of the amount of switches to the side and below; they would fall outside of the tracking envelope.

You can also mess about with the sensor position to alter the tracking envelope, mine is on top of my G2 and set back a bit which moves the envelope backwards a little at the expense of downwards vision.

This works nicely because if I'm using a control low down I'm going to be looking at it but when heads up the hands stop tracking because they're occluded by the front of the headset.

Ok, so here’s where I am:

I went back and double checked for reflective surfaces and threw a blanket over everything just to be sure. In visualizer everything is perfect. I did notice the curling of the fingers there, and it appears to happen at the edge of the sensor/camera range; it appears usable if I lean forward. I’m 6’7” with suitable long arms, so it wouldn’t be the first time I had to adjust. 
 

but…

when I’m in DCS, for the “weapon” button on the left MFD, I have to reach my physical arm/hand down lower than my knee to even get close to the button. It’s like there’s a problem with how DCS is interpreting the scale of my movements; like instead of 1:1 hand movement, it’s 2:1. The underlying problem has been that I have to reach *really* far to touch the buttons, which isn’t what the world scale would indicate. 
I will say that in Visualizer (and projecting it into the HMD), everything is magnified, which may be what’s messing with DCS?

Does any of this make sense?

erik

i9 11900K :: 32GB G.Skill Trident DDR4-3200 :: Corsair water loop ::  Samsung 2TB 980pro nvme :: Gigabyte Aorus 3090 Waterblock :: Gigabyte Aorus Z590 Xtreme MoBo :: HP Reverb G2 VR

VKB Gunfighter flightstick and collective :: TM Warthog throttle & MFDs :: MFG Crosswind pedals :: 5 Dayton Audio bass shakers on 5.1 home theater amp

Win10 Pro x64 :: DCS 2.7 (Huey & P-51, mainly; have all the other rotors, but haven't explored their depths as yet)

Link to comment
Share on other sites

11 hours ago, dureiken said:

ok

Not a big problem, just have to think about it. What is my main problem actually is that my right hand is not seen by DCS one time over 2. Left hand is really nice. I can move my right hand in front of the camera, shake it but no succes. And sometimes it works fine.

I tried with leap software in sametime and it's fine, nothing on DCS but hand on leap ;(

 

same for you ? thanks

 

Right hand MIA will almost certainly be due to some control input on the stick, if there is active delta on the control inputs then the hands are disabled (unless you have leap hands control the stick and throttle which is terrifying 😄 ).  Basically if you have ANY stick input your right hand is disabled, this does unfortunately mean you can't hold stick position with the left and then use the right hand for switches etc.

It might take a couple of attempts to get the stick centred, setting a slightly larger deadzone on the stick axes might help as well; I've not specifically tested that one I'm afraid.

 

5 hours ago, Ainuke said:

Ok, so here’s where I am:

I went back and double checked for reflective surfaces and threw a blanket over everything just to be sure. In visualizer everything is perfect. I did notice the curling of the fingers there, and it appears to happen at the edge of the sensor/camera range; it appears usable if I lean forward. I’m 6’7” with suitable long arms, so it wouldn’t be the first time I had to adjust. 
 

but…

when I’m in DCS, for the “weapon” button on the left MFD, I have to reach my physical arm/hand down lower than my knee to even get close to the button. It’s like there’s a problem with how DCS is interpreting the scale of my movements; like instead of 1:1 hand movement, it’s 2:1. The underlying problem has been that I have to reach *really* far to touch the buttons, which isn’t what the world scale would indicate. 
I will say that in Visualizer (and projecting it into the HMD), everything is magnified, which may be what’s messing with DCS?

Does any of this make sense?

erik

A whole foot taller than me 😄

I do have +3" ape index though so arms are relatively long.

It might help to play around with the height/position/angle of the sensor as I think this can have a bearing on the positional tracking of the hands.  I find that if I lean right back in my chair and recentre HMD before sitting forwards to normal position then it brings me closer to the front panels and means I don't have to reach so far to hit the MFDs (I get physical finger clashing with my desk/steering wheel base/shifter otherwise).

Also have a play with lighting, leap motion will work in perfect darkness but over bright light sources may wash it out.  I have a set of led strip lights arranged around my cupboard (yes I fly in a walk in cupboard in the bedroom!) and setting them to red gives enough light for the G2 to track, enough light to see to type and not cause too much light pollution for late night use.

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

If this could be used along with leap motion to actually "feel" the controls in DCS while you interact with them, I'd be very happy....

The open question is whether or not enough is exposed on the DCS side to write some code to do the integration with DCS...

From their website... "Note that TactGlove is compatible with Meta Quest and Ultraleap’s Leap motion camera-based hand tracking systems."

 


Edited by j9murphy
Link to comment
Share on other sites

2 hours ago, j9murphy said:

If this could be used along with leap motion to actually "feel" the controls in DCS while you interact with them, I'd be very happy....

The open question is whether or not enough is exposed on the DCS side to write some code to do the integration with DCS...

From their website... "Note that TactGlove is compatible with Meta Quest and Ultraleap’s Leap motion camera-based hand tracking systems."

 

 

This might make you piss with excitement but ultraleap do touch free mid air haptics 😄

Super spendy though and unlikely to get applied to DCS 😞

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

7 hours ago, edmuss said:

 

Right hand MIA will almost certainly be due to some control input on the stick, if there is active delta on the control inputs then the hands are disabled (unless you have leap hands control the stick and throttle which is terrifying 😄 ).  Basically if you have ANY stick input your right hand is disabled, this does unfortunately mean you can't hold stick position with the left and then use the right hand for switches etc

 

Thanks a lot, it was that. I checked right hand on stick because my stick is on the middle (what a shame), so my hand doesn't go on F16 right stick !

 

No more problem, thanks a lot

  • Like 1
Link to comment
Share on other sites

Because I use a physical centre stick and fly the A10 having the leap motion suddenly take control authority of the stick axes whilst simultaneously fighting against the inputs bound to those axes it's almost always an instacrash 😄

I really like the hand snapping to the stick visually, I just wish it was possible to not have it give the stick inputs as well.

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

Another question:
1.
I have noticed that my virtual right hand is displayed 5 cm lower than my left hand.
Does this happen to you too?

2.
And if I hold my real hands still in front of me and move my head to the right/left/up or down, my virtual hands follow this movement to a certain level.

Link to comment
Share on other sites

Now I have a really stupid question.
Does the usb cable on your Leap motion come out on the left or right side when you wear the vr glasses?

For me, it comes out on the right side.
But in the YouTube videos of DCS pilots, it comes out of the left side…?

Link to comment
Share on other sites

5 hours ago, A.F. said:

Another question:
1.
I have noticed that my virtual right hand is displayed 5 cm lower than my left hand.
Does this happen to you too?

2.
And if I hold my real hands still in front of me and move my head to the right/left/up or down, my virtual hands follow this movement to a certain level.

Not seen offset hands myself, I get the slight tracking lag as well but it's not an issue for normal usage.

5 hours ago, A.F. said:

Now I have a really stupid question.
Does the usb cable on your Leap motion come out on the left or right side when you wear the vr glasses?

For me, it comes out on the right side.
But in the YouTube videos of DCS pilots, it comes out of the left side…?

Either, you can flip the orientation in the leap motion software so cable can come out left or right 🙂

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

vor 30 Minuten schrieb edmuss:

Not seen offset hands myself, I get the slight tracking lag as well but it's not an issue for normal usage.

Either, you can flip the orientation in the leap motion software so cable can come out left or right 🙂

I did not know that. Thank you 

  • Like 1
Link to comment
Share on other sites

22 minutes ago, A.F. said:

It's just like reality.
You have to take your hand off the throttle to operate a switch.

Or did I misunderstand your comment?

I mean, my arm is on real throttle, and it switch cockpit gear button or seat arm

 

when you are a mach 1, not a good choice 😛

Link to comment
Share on other sites

It should only be the tip of your index finger that can activate switches, and I think then only if you're making a gun finger. Keep your index finger tucked in/bent whilst traversing panels and it shouldn't accidentally trigger anything.

I would anticipate that if your hand is on the the throttle then the finger shouldn't be extended and/or be occluded.

Obviously every setup will be different and I've adjusted my throttle to be in the correct nominal position so I never accidentally flick anything.

Ryzen7 7800X3D / RTX3080ti / 64GB DDR5 4800 / Varjo Aero / Leap Motion / Kinect Headtracking
TM 28" Warthog Deltasim Hotas / DIY Pendular Rudders / DIY Cyclic Maglock Trimmer / DIY Abris / TM TX 599 evo wheel / TM T3PA pro / DIY 7+1+Sequential Shifter / DIY Handbrake / Cobra Clubman Seat
Shoehorned into a 43" x 43" cupboard.

Link to comment
Share on other sites

vor 20 Stunden schrieb edmuss:

 

 

Obviously every setup will be different and I've adjusted my throttle to be in the correct nominal position so I never accidentally flick anything.

I would like to be able to do that too!

When I make an input on the stick or throttle, either my right or my left hand disappears. (In the "Hands do not move the stick/throttle" setting).
Why is this the case for me and not for you? 
Hm.... Maybe I have to ask Leap.

Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...