leapMotion implementation possibility

[streakeagle]

Leap Motion should arrive on May 28th... 2 days. I just got a WinWing F/A-18C HOTAS today, so it is like Christmas in May :)

[Fri13]

For years it has been no reaction from the Eagle Dynamics to support Leap Motion. While it was cheap (under 20€) to buy. When the company was bought, the device price rocketed to over 100€ price range.

 

Sure it wasn't amazing in the early time, but when in 2018 the "Orion" version came out, its tracking accuracy and latency capabilities became amazing.

The DCS was exactly what the Orion would have improved, if just officially supported.

 

I could almost bet that ED must have few of these devices laying around in their office, next to some VR systems.... But then a claptoglove gets implemented with a hand and finger tracking.... like, what the....?

 

Today we should be able go to DCS settings and tick box "Use Leap Motion in VR mode" and just attach sensor to HMD front and have a virtual hands working (even the skeleton model from Orion would be amazing).

 

And I would bet that so many DCS user would have bought Leap Motion by now just for official support (easy to install, works on all VR systems).

[Kariyann71]

For years it has been no reaction from the Eagle Dynamics to support Leap Motion. While it was cheap (under 20€) to buy. When the company was bought, the device price rocketed to over 100€ price range.

 

Sure it wasn't amazing in the early time, but when in 2018 the "Orion" version came out, its tracking accuracy and latency capabilities became amazing.

The DCS was exactly what the Orion would have improved, if just officially supported.

 

I could almost bet that ED must have few of these devices laying around in their office, next to some VR systems.... But then a claptoglove gets implemented with a hand and finger tracking.... like, what the....?

 

Today we should be able go to DCS settings and tick box "Use Leap Motion in VR mode" and just attach sensor to HMD front and have a virtual hands working (even the skeleton model from Orion would be amazing).

 

And I would bet that so many DCS user would have bought Leap Motion by now just for official support (easy to install, works on all VR systems).

 

 

Totally agree.

But many users will complain about the fact that hands have to be in the field of view.

That’s why people want CaptoGlove and haptic feedback.

But how many of us know perfectly the position of each knobs and switches in the cockpit to not take a look at before using ?

...

Now we have a CaptoGlove implementation that seems to doesn’t work properly and no LeapMotion implementation while solutions exist...

[Snake122]

For years it has been no reaction from the Eagle Dynamics to support Leap Motion. While it was cheap (under 20€) to buy. When the company was bought, the device price rocketed to over 100€ price range.

 

Sure it wasn't amazing in the early time, but when in 2018 the "Orion" version came out, its tracking accuracy and latency capabilities became amazing.

The DCS was exactly what the Orion would have improved, if just officially supported.

 

I could almost bet that ED must have few of these devices laying around in their office, next to some VR systems.... But then a claptoglove gets implemented with a hand and finger tracking.... like, what the....?

 

Today we should be able go to DCS settings and tick box "Use Leap Motion in VR mode" and just attach sensor to HMD front and have a virtual hands working (even the skeleton model from Orion would be amazing).

 

And I would bet that so many DCS user would have bought Leap Motion by now just for official support (easy to install, works on all VR systems).

I sold my original Leap Motion for more than I paid for it a month or so after the original release, it was great in some demos but never worked well for actual application. So far, Orion seems to have been a big upgrade from what I remember. But ED seems to be done with VR controller support, even Oculus Touch/Vive Controller, despite not having a good knob manipulation solution. But I somehow don't think ED has one, it's an arcane enough controller that hasn't hit that threshold to be common enough were they would spend money to buy one. I think even the Captoglove support wasn't tested by ED. But I also agree between the two of them, Leap Motion seems to be the better candidate.

Totally agree.

But many users will complain about the fact that hands have to be in the field of view.

That’s why people want CaptoGlove and haptic feedback.

But how many of us know perfectly the position of each knobs and switches in the cockpit to not take a look at before using ?

...

Now we have a CaptoGlove implementation that seems to doesn’t work properly and no LeapMotion implementation while solutions exist...

But to be fair, Kariyann71, you have taken it further that ED would have. I find the Oculus Touch Controllers pretty usable until I need to turn a knob. You have at least added a way to do this plus the gestures. If ED added support it would just be the traditional VR controller "gloves "and that are still hard to use with knobs and wouldn't have gestures. You work is much appreciated. And I too can't understand the want to flip virtual switches outside of view, it is not practical. Leap Motion is a better solution that CaptoGlove, all reports is that it is even more of a mess than pre-Orion Leap Motion at a pretty steep price.

[Om]

I sold my original Leap Motion for more than I paid for it a month or so after the original release, it was great in some demos but never worked well for actual application. So far, Orion seems to have been a big upgrade from what I remember. But ED seems to be done with VR controller support, even Oculus Touch/Vive Controller, despite not having a good knob manipulation solution. But I somehow don't think ED has one, it's an arcane enough controller that hasn't hit that threshold to be common enough were they would spend money to buy one. I think even the Captoglove support wasn't tested by ED. But I also agree between the two of them, Leap Motion seems to be the better candidate.

But to be fair, Kariyann71, you have taken it further that ED would have. I find the Oculus Touch Controllers pretty usable until I need to turn a knob. You have at least added a way to do this plus the gestures. If ED added support it would just be the traditional VR controller "gloves "and that are still hard to use with knobs and wouldn't have gestures. You work is much appreciated. And I too can't understand the want to flip virtual switches outside of view, it is not practical. Leap Motion is a better solution that CaptoGlove, all reports is that it is even more of a mess than pre-Orion Leap Motion at a pretty steep price.

Man, can you make a guide how did you make it work with Rift S? I see no cursor, but I see ejection hand animation? What did you install? 32 or 64-bit drivers? But I also don't see mouse cursor if I move it, but Use mouse setting is on in the menu

[Snake122]

Man, can you make a guide how did you make it work with Rift S? I see no cursor, but I see ejection hand animation? What did you install? 32 or 64-bit drivers? But I also don't see mouse cursor if I move it, but Use mouse setting is on in the menu

 

I followed this completely:

https://github.com/kariyann/DCLeap/blob/master/DCLeap/LatestRelease/DCLeap_Manual.pdf

Be sure to install both the driverleap drivers and then DCLeap.

 

Then launched DCS with a shortcut that had --force_steam_vr added to the end of the target line

[Om]

I followed this completely:

https://github.com/kariyann/DCLeap/blob/master/DCLeap/LatestRelease/DCLeap_Manual.pdf

Be sure to install both the driverleap drivers and then DCLeap.

 

Then launched DCS with a shortcut that had --force_steam_vr added to the end of the target line

 

What method did you use - 1 or 2? What kind of driver did you install - 32 or 64 bit? Do you see tracking hands in SteamVR home, I mean can you make gestures there? Have you ever had issues with cursor displaying (I have no cursor in the cockpit but I have mouse cursor in the settings menu)?

 

And when you Alt-Enter your game it becomes stretched for you too on your monitor?

[Om]

I found how to show cursor in the cockpit, I had to disable controllers.

Ok, I see the cursor, I see the red and green square, I can on and off them, but I can't move the cursor with my hands. If I press ESC for menu, sometimes I can see the cursor moving at the bottom of the menu. And it seems like it moves with my head, it's like it's bound to my head. I'm trying to move hand here and there, nothing, no cursor.

I've tried 2560 x 1440 and 1280 x 1440 for HMD screen resolution in DCLeap... nothing. It doesn't work for me on Rift S.

[Fri13]

Totally agree.

But many users will complain about the fact that hands have to be in the field of view.

That’s why people want CaptoGlove and haptic feedback.

 

Well, as the pilot habit goes, you blindly set the instrument for the wanted setting and then you glance it did you make it right, and then back to looking between outside and instruments.

So sure you can't feel anything that where your hand is in the cockpit with hand tracking, but considering that we don't either have G-forces to worry nor any motion related stuff, we can already place hand in the instrument location and glance there to adjust it and back to look elsewhere.

 

And anyways I wouldn't want a haptic feedback for outside of view touching as I would be easily activating all kind things again without looking. While in reality we can lay hand on panel that has all kind safety elements to avoid accidental operation, and we can by feel search the correct knob or switch to operate even without activating wrong ones.

 

But how many of us know perfectly the position of each knobs and switches in the cockpit to not take a look at before using ?

 

When using mouse, none. Because you can't move your mouse cursor over the instrument without looking it first. So you can know "it is on right side panel at X position, next to red one" etc. But you can't do anything for it until you look at it.

 

With hand tracking (be it a controllers or actual hand tracking) we can already move hand in the correct position with very good accuracy (improves a lot longer you fly the one and the same aircraft) and be ready to operate it once you glance at it.

 

Now we have a CaptoGlove implementation that seems to doesn’t work properly and no LeapMotion implementation while solutions exist...

 

What is funny situation. And there has been even questions that how many has managed to purchase the captogloves? As are they just vaporware or very rare unique items with extreme demand?

If we could order a pair for a 90-120€ and get it in two weeks, we would definitely see far more users for those gloves.

 

But this is why the Oculus implementation is far more interesting for the Rift S, as it really has the blind spot only below the HMD where our joystick is located. While the sidepanels etc could be tracked even with a single camera, while not so accurately.

 

In that case the Rift S camera positioning is better as it has a better camera placement for the important area what is lower frontal hemisphere including sides.

 

The Leap Motion simply is more restricted, covering little larger area than the Rift S FOV is, what makes it work as anyways you want to touch something that you are looking. But it is not fun that you have something in your hand (ie. gun or any other object) that gets lost when you look elsewhere than your hand.

 

But in DCS purposes it works well as we anyways look what we do, as we don't need to be pointing hands elsewhere than our view is.

[Fri13]

But ED seems to be done with VR controller support, even Oculus Touch/Vive Controller, despite not having a good knob manipulation solution. But I somehow don't think ED has one, it's an arcane enough controller that hasn't hit that threshold to be common enough were they would spend money to buy one. I think even the Captoglove support wasn't tested by ED. But I also agree between the two of them, Leap Motion seems to be the better candidate.

 

ED changed the Oculus Touch controllers functionality not long time ago (maybe after Rift S). Where the mini-stick controls were changed so you do left/right click with moving stick forward/backward, and then you perform rotations by moving stick left/right, where you can start from front/rear and move toward the 9/3 clock position for maximum speed.

 

Before this we had the green/blue beam that we needed to change by twisting palm over 180 degree position so we could get left/right clicks. And it was awkward and difficult.

 

I would still prefer to be able have a A/B or X/Y buttons as left/right click and trigger/grip for grabbing knobs or levers if so wanted and use the hat for rotation as now.

 

But it is at least much better now than when CV1 was out.

 

And I as well believe that it was the ClaptoGlove that approached ED and made their support, as it is after all like a Tacview module among others, instead built-in feature.

 

But to be fair, Kariyann71, you have taken it further that ED would have. I find the Oculus Touch Controllers pretty usable until I need to turn a knob. You have at least added a way to do this plus the gestures.

 

With the computer mouse we can drag a knob up/down to make adjustment, but I find it difficult. Really prefer the touch controllers mini-stick rotation to left/right from forward/backward position to make a small accurate adjustments. And there is as well the grabbing possibility but moving hand up/down is as well awkward.

 

If ED added support it would just be the traditional VR controller "gloves "and that are still hard to use with knobs and wouldn't have gestures. You work is much appreciated. And I too can't understand the want to flip virtual switches outside of view, it is not practical. Leap Motion is a better solution that CaptoGlove, all reports is that it is even more of a mess than pre-Orion Leap Motion at a pretty steep price.

 

I as well appreciate these who does work to get Leap Motion working, sadly I use the Oculus and not having interest to launch DCS via SteamVR (even when I have many Steam VR games).

 

It is just sad that ED that should have good programmers and insights to their engine etc, that they just are not willing to implement it even at basic level. As I could think many of these third party developers would find more use to Leap Motion if it would be supported by ED itself. Be it a kneeboard or a new radio menus or what ever.

[Om]

What are you talking about? We got something on Oculus Rift S? You said something about Oculus Rift S implementation and cameras? So what actually we got? We got nothing but Oculus got multi bucks profit. We won't get any "hand tracking" for Rift S, it's only for Quest

[Om]

Anyway this guy just made something (even if it doesn't work for me), but he made it.. maybe even more than DCS devs made in this direction

[Kariyann71]

leapMotion implementation possibility

 

I found how to show cursor in the cockpit, I had to disable controllers.

Ok, I see the cursor, I see the red and green square, I can on and off them, but I can't move the cursor with my hands. If I press ESC for menu, sometimes I can see the cursor moving at the bottom of the menu. And it seems like it moves with my head, it's like it's bound to my head. I'm trying to move hand here and there, nothing, no cursor.

I've tried 2560 x 1440 and 1280 x 1440 for HMD screen resolution in DCLeap... nothing. It doesn't work for me on Rift S.

 

 

You’ve an old DCLeap, go to the GitHub depot and download the last One

 

https://github.com/kariyann/DCLeap

Edited May 27, 2020 by Kariyann71

[Om]

You’ve an old DCLeap, go to the GitHub depot and download the last One

 

https://github.com/kariyann/DCLeap

 

ok, I've downloaded and written about this https://www.digitalcombatsimulator.com/ru/files/3306533/?sphrase_id=1279529 version of DCLeap. And somehow I make it to work on 1280 x 1440, I disabled mouse AND disable VR controllers and It showed the cursor. And I managed even to use the pinch gesture to click BUT

1. I can't move the cursor with my hand, I only moved this by my head (cursor was like right in the center)

2. Pitch or whatever gesture it is was sooooo sensitive, even when I didn't make a pinch and just rotate my head it was clicking here and there all buttons and knobs.

3. Cursor never disappear as it happens with the regular mouse in several secs of idle. When I alt+tab the game and get back cursor disappeared and never came back and it all didn't work again.

I don't know WTF is this an how to manage that

BTW force VR enable seems like don't work for me cause I have Steam version of DCS.

 

So I have an old version of DCLeap?

[Fri13]

What are you talking about? We got something on Oculus Rift S? You said something about Oculus Rift S implementation and cameras? So what actually we got? We got nothing but Oculus got multi bucks profit. We won't get any "hand tracking" for Rift S, it's only for Quest

 

It is still open that what Oculus does with the Rift S. Their word was that they will observe the feedback from the hand tracking in Oculus Quest to get it stable. And now they have made the hand tracking in it official feature. So there is possibility that it is coming to Rift S in future.

 

“We started [hand-tracking] with the Quest platform first and we’re going to listen to the feedback from Quest as we roll out this feature, and will evaluate the technical challenges of bringing it to other devices [like Rift S],” an Oculus spokesperson tells Road to VR.

https://www.roadtovr.com/oculus-rift-s-hand-tracking-wait-for-feedback/

 

So unless you have a other source from Oculus saying that Rift S will never get the hand tracking, then it stays possibility to receive it.

 

And what I talked about cameras, is that in Rift S they are far better located for the flight simulators point of view by how aircraft cockpits are designed, compared to other VR games where you might need to raise hands above your head or pull something behind your head/back. As we do not need to do anything like that. And we anyways look widgets we set in the cockpit so hand tracking inside field of view is very usable.

[Kariyann71]

leapMotion implementation possibility

 

ok, I've downloaded and written about this https://www.digitalcombatsimulator.com/ru/files/3306533/?sphrase_id=1279529 version of DCLeap. And somehow I make it to work on 1280 x 1440, I disabled mouse AND disable VR controllers and It showed the cursor. And I managed even to use the pinch gesture to click BUT

1. I can't move the cursor with my hand, I only moved this by my head (cursor was like right in the center)

2. Pitch or whatever gesture it is was sooooo sensitive, even when I didn't make a pinch and just rotate my head it was clicking here and there all buttons and knobs.

3. Cursor never disappear as it happens with the regular mouse in several secs of idle. When I alt+tab the game and get back cursor disappeared and never came back and it all didn't work again.

I don't know WTF is this an how to manage that

BTW force VR enable seems like don't work for me cause I have Steam version of DCS.

 

So I have an old version of DCLeap?

 

 

Yes the version on DCS user’s files is old.

I published on GitHub early may a new version that introduce a debug file generation.

Download this version, try it and if it doesn’t work go to the Data folder of DCLeap, get the logfile.txt and attach it to tour post. I’ll take a look at it to identify the issue.

To avoid parasite behaviors (knob and clicks) modify the config file xml of LeapMotion for SteamVR. I explained the procedure in the manual.

[Fri13]

Is there possibility to have nothing more than a mouse cursor tracking the "active hand" index finger tip? So no gestures, nothing else than just get cursor follow index finger (that naturally becomes pointing things)?

[Kariyann71]

Is there possibility to have nothing more than a mouse cursor tracking the "active hand" index finger tip? So no gestures, nothing else than just get cursor follow index finger (that naturally becomes pointing things)?

 

 

Yes you can disable each hand gestures to use only cursor. Conversely you can disable mouse to use only hand gestures.

[havok2]

Yeah, I was playing with this driver all night recently and no success, it's so weird and clunky, I've tried Vive and Index controllers "mode" - the same, tried VR chat - it has offset, as you said, and it's impossible to use it there, cause sometimes it doesn't show right hand/controller. BTW I've never seen "hand/controller: in SteamVR home, only in settings menu or game (VRChat). But I didn't try to disable buttons. Maybe that will help. Anyway, it's sooo weird, but original LeapMotion Visualizer work perfectly, really strange thing.

 

Where did you disable buttons? In settings config file (where you type what kind of controller you wanna use?)?

 

Hey,

I have finally found the parts of the driver where I can anchor an appropriate offset of the hands. In DCS it works quite well. Now I have to do some fine tuning. Furthermore you have to press A or B for the left mouse button or the right mouse button in the beta. But you have to put your thumb and middle finger or your thumb and pinky finger together for a gesture for A or B. More intuitive will be the trigger gesture with the index finger and middle finger with thumb. When the whole thing is more suitable, I will open a separate thread for this. The advantage of this method is that it works direct in dcs.

[streakeagle]

I have my new Leap Motion installed. So far, all I have done is use the visualizer to see it map my hands. Amazing!

 

1) Why didn't the VR headsets with cameras have this capability built in?

2) Why hasn't Oculus made this available on the Rift S when they have it on the other lower capability Galaxy?

3) Where has this been all my life? If I can get this working reliably in games, this is as big as the VR headset itself. I would have been happy using this with a flat screen.

 

So, now I have to start running Steam VR and following the regular instructions for DCS Leap... but I am willing to work with whoever to make this work natively with Oculus software if independent third parties aren't locked out.

[streakeagle]

Leap Motion has my hands show up as controllers in SteamVR and I can show hands.

But once I start DCSLeap, the mouse cursor mainly reacts to my head position, not my hands. I have zero control. It overrides my real mouse, so the only way I can do anything is to use the oculus controller to kill steamvr to give me back mouse control.

 

Of course Oculus decides to send an update on the night I get my Leap Motion control.

[streakeagle]

I have the in-game cursor more-or-less following my hand, but I need to tune the settings to get it smooth and predictable. Right now, I struggle to move the switch/lever I am targeting, but bump all kinds of things accidently while looking around.

 

It is sad and disappointing to see how well my hands are tracked in the leap motion visualizer but all I can do in the game is mouse emulation... much like TrackIR in mouse emulation mode, close but no cigar. DCS needs native support for Leap Motion. It could work really well with what was intended for capto gloves.

[streakeagle]

I tried using the DCS World VR hands instead of DCLeap. I found it much easier to use, no tuning necessary. I could push buttons with my fingertips and use the palm-up/palm-down to switch between left and right mouse button. I was in an F-14 and was able to reach around and flip the ejection seat switch using my left finger to push it. The experience was very close to what I would have wanted with capto-gloves. If I could just connect the virtual pilot to my VR hands, it would really be cool. I also could use the mouse without having to <CTRL><ESC> to shut down DCSLeap, and then could transition back to virtual hands at my leisure.

[Om]

I tried using the DCS World VR hands instead of DCLeap. I found it much easier to use, no tuning necessary. I could push buttons with my fingertips and use the palm-up/palm-down to switch between left and right mouse button. I was in an F-14 and was able to reach around and flip the ejection seat switch using my left finger to push it. The experience was very close to what I would have wanted with capto-gloves. If I could just connect the virtual pilot to my VR hands, it would really be cool. I also could use the mouse without having to <CTRL><ESC> to shut down DCSLeap, and then could transition back to virtual hands at my leisure.

 

What do you mean "DCS World VR hands instead of DCLeap"?

[streakeagle]

In the VR settings for DCS World, you can enable VR hands for VR controllers.

 

I am forcing SteamVR and using the drivers I installed to support DCSLeap, so Leap Motion makes my hands show up as HTC Vive Controllers in SteamVR.

 

So: I have Leap Motion and SteamVR running, but not DCSLeap, then I enable hands in the DCS World VR settings, I see the same hands I would see if I was running Oculus with Touch Controllers. Only the hands follow my hands and the index finger follows my index finger. The VR hands do not quite match the orientation of my real hands, but it is more than close enough. The nice thing is if my hands go out of view of the leap motion sensor, they disappear from the screen. This makes it easier to manage: i.e. avoid bumping switches/levers while using the HOTAS or the mouse. It would be cool if the hands could be turned on and off at will, like a PTT button. So that when I don't want them floating around bumping things, they are invisible. Imagine a toggle function: when you have a pilot body visible, you press a key or button and the hands rest on the throttle and stick. Bump the key/button again, and the hands follow your real hands. That is what I would love to see. What I have right now is pretty close minus the connection to the virtual pilot's arms/hands.

 

What you lose by not running DCSLeap are the gestures and the simplified capability of the DCS hands that have trouble doing much more than toggling switches and pushing buttons. If I could have the virtual hands AND the gestures along with a single toggle function to turn both on/off, that would be awesome.

Edited May 30, 2020 by streakeagle