leapMotion implementation possibility

[frogger]

i tried the flyinside standalone flight simulator lastnight and its a step forward but i still think the leap motion is to limited in tracking ability to acuratly control a cockpit while flying. i could push the throttles and stuff like that alot better but then u go to grab your joystick and u hit about 5 switches on the way. i personally think you would be better off trying to implement leap motion hand models for the simple use of signals to wingmen in formations. good work though.

[Kariyann71]

Hi everybody,

 

as I said, I had a busy autumn at work.

I was able to get back to the project recently and today I have solved a major problem with the reset pilot's position function :drink:.

I was having this problem since the beginning of the project. Now that it's resolved, I'll be able to move faster.

 

Currently, I managed to:

- generate command exports to DCS;

- create simple buttons;

- create levers;

I now have to create the rotary knobs.

 

I hope to publish soon a video of my progress and why not create a deposit on Github so that those who are interested participate with me on this project.

 

Bye

[wormeaten]

Hi everybody,

 

as I said, I had a busy autumn at work.

I was able to get back to the project recently and today I have solved a major problem with the reset pilot's position function :drink:.

I was having this problem since the beginning of the project. Now that it's resolved, I'll be able to move faster.

 

Currently, I managed to:

- generate command exports to DCS;

- create simple buttons;

- create levers;

I now have to create the rotary knobs.

 

I hope to publish soon a video of my progress and why not create a deposit on Github so that those who are interested participate with me on this project.

 

Bye

 

 

:thumbup:

 

 

:pilotfly:

[mavrick1709]

Hello Guys, any news on this very interesting project ?

Thank you.

Bye.

[Willie Nelson]

Bumping back to the top, pretty keen to see if anyone can get something going with Leap Motion. All other options seem to be massive compromises for VR.

[TripRodriguez]

Bumping back to the top, pretty keen to see if anyone can get something going with Leap Motion. All other options seem to be massive compromises for VR.

 

Agreed!

 

But for me (cockpit builder) all I need is to be able to see my hands, I'll be operating real switches but I'm tired of having to feel around for them because my hands are invisible.

 

I'm chasing this down via multiple channels, now I'm about to check out the program this guy said he was using to do the overlay since an overlay is all I need. I'd like to be able to use Vive Trackers though, as well as Leap Motion.

[LhuanDula]

I'm chasing this down via multiple channels, now I'm about to check out the program this guy said he was using to do the overlay since an overlay is all I need. I'd like to be able to use Vive Trackers though, as well as Leap Motion.

 

Hi there,

just to add to the discussion, a possible solution would be to emulate valve's controller with the leap motion. This guy did it, and it's seems to work "nicely":

 

His implementation is available here.

 

According to the code, it should be possible to remove any controller inputs and just keep the finger tracking.

 

However, it still leaves you at the mercy of unwilling cockpit interaction because of hand/cockpit collision coded by ED. But if such problem were to be solved (I do hope it will) then this would remove the need to go through an overlay. Thus, you'll also have depth test on hands vs cockpit (such that a hand behind a cockpit feature will stay behind, which is not possible with an pure overlay).

Then, if ED ever attaches arms to the VR hands and add lighting interactions (shadows etc..), VR with a physical cockpit would truly shine :).

[TripRodriguez]

Well, perhaps I'd consider trying again but I fought with SDraw/Driver_Leap for hours the other day and it was a frustrating experience.

 

It was extremely unreliable as far as the virtual steam controller appearing, and to use it in DCS I had to hold my hands up in front of my face as soon as I click launch on the DCS.exe and keep them there until DCS loaded up or I'd have no controllers at all. That technique was recommended by the developer of the driver.

 

Then again, that video is only a month old so maybe worth another try.

 

Sometimes it had one controller active, sometimes six! And as you say, there's still the problem with DCS which hopefully they will solve but I don't want to wait without knowing if it might be years until they fix it. I will do a bug report on it and see if I get any response.

[LhuanDula]

That's good to know you already tried SDraw, there wasn't any mention of it in the thread.

 

It's a shame, I was hoping for it to be a good answer to hand tracking in VR.

 

Do you have any guess as to what is the root problem for these weird behavior?

[TripRodriguez]

I'd say it just needs to be debugged, and isn't getting enough dev time. I'd go for a payware version if it's easy to use and reliable.

 

Unfortunately Driver4VR Leap Motion support is also apparently a mess based on what I've read. I didn't buy it to try since lots of people were asking for refunds for that software.

[Willie Nelson]

Thanks for helping bring it back to the top guys, still seems like there’s a way to go till someone like me can use it but maybe it’s not out of the question one day.

[TripRodriguez]

What I find really really strange is that Leap Motion themselves don't just make it a native SteamVR device.

 

Seems like such an obvious thing to do....

[LhuanDula]

Well I guess LeapMotion is more interested in having their own implementation for hand/gesture interaction rather than having to fit into the OpenVR specification.

Ultimately, you can't blame them for that, mainly because the OpenVR specs are focused on interaction for physical devices (button pressed/released and so on) rather than modelling a virtual hand.

[TripRodriguez]

I use the BenOtterOVRlay http:// https://github.com/benotter/OVRLay.

Icebeat, you’re right, sharing this project is probably the best way to make it progress quickly and surely better. Indeed I’m very busy a work these past weeks and this fall I will be very busy.

Is there anyone comfortable with the github platform ??

 

Kariyann71 could you share your setup for the OVRLay?

 

I have no idea what I'm doing LOL, but I'm just trying to see my leap motion (and/or Vive Tracker) hands in DCS. I don't want my hands to operate anything in DCS, I just want to see them in 3D so I can find switches more easily in a real cockpit.

 

Any help would be very much appreciated, and I know there are others wanting the same thing.

[RealDCSpilot]

https://media.discordapp.net/attachments/303984261264965641/598173821069033512/JPEG_20190709_172855.jpg?width=658&height=878

 

 

Just worked on this, it still needs some minor polish, but it's nearly finished.

[TripRodriguez]

Awesome! I hope you can provide installation instructions "for dummies"!

90% of the time of I find something I need is on github I can't figure out how to use it.

 

I'd happily pay $20 usd for this of its easy to install and will be maintained so it keeps working when steam VR does updates.

 

Ideally (wish list here, not mandatory) with it would also have adjustable scale as well on case your home cockpit is a little later or smaller than in game.

 

You can adjust the cockpit size in DCS with force IPD but it would be better to scale the overlay up or down a bit.

Edited July 9, 2019 by TripRodriguez

[TacoGrease]

This looks amazing! Hope to see the finished project. Getting my Index Friday.

 

https://media.discordapp.net/attachments/303984261264965641/598173821069033512/JPEG_20190709_172855.jpg?width=658&height=878

 

 

Just worked on this, it still needs some minor polish, but it's nearly finished.

[TripRodriguez]

This looks amazing! Hope to see the finished project. Getting my Index Friday.

 

I've got mine mounted too. I got the cord coiled in the frunk

 

Alec Delorean were you just talking about mounting your Leap, or did you mean you are working on the software? I was on mobile at the time with almost no signal so I didn't see until now that it was just a pic of the Leap on the Index.

 

Software is the problem.....

Edited July 10, 2019 by TripRodriguez

[RealDCSpilot]

Sorry, i'm only working on designing the holder piece.

[RealDCSpilot]

https://www.thingiverse.com/thing:3739295

[Kariyann71]

Hi everybody,

 

I stopped this program cause I finally found it was not appropriate to DCS, especially considering World scaling.

I think a « mix » leap motion and hardware button would be more appropriate.

Nevertheless I could put my project in this thread if someone want to continue this one.

[TripRodriguez]

Hi everybody,

 

I stopped this program cause I finally found it was not appropriate to DCS, especially considering World scaling.

I think a « mix » leap motion and hardware button would be more appropriate.

Nevertheless I could put my project in this thread if someone want to continue this one.

 

Please do, "« mix » leap motion and hardware button" is exactly what I'm wanting to do but I don't know how to use Unity, most of the time I find something I need on Github I don't know how to use it. All I can do is hope for detailed instructions or someone who knows more and wants to help.

[aniron]

Please do, "« mix » leap motion and hardware button" is exactly what I'm wanting to do but I don't know how to use Unity, most of the time I find something I need on Github I don't know how to use it. All I can do is hope for detailed instructions or someone who knows more and wants to help.

 

Seconded. I have the leap motion and want to use it to help steer my hands towards hardware buttons.

[TripRodriguez]

Completed a proof of concept test tonight for my best sim cockpit solution so far which is using Vive Trackers via OVRIE. Here is a (REALLY CRAPPY and therefor unlisted!) video. Don't mind the mess around the sim, parts, sawdust, and tools everywhere.... For this video to really be worth anything I'd need to have captured the gameplay to put up side by side. I'll try to get that done later in the week. For the first bit of the video you can (just barely) see the monitor in the background so you can see my DCS hand.

 

Explanation: For several years on and off I've been trying to find a satisfactory solution to using VR in a full simulator cockpit. I find that my immersion is completely ruined by having to fumble blindly to find the buttons and switches that are right in front of me because I can't see my hands.

 

In this video, I have the beginning of my Huey sim cockpit, an (unfinished) complete forward console. I'm wearing the VR headset thorughout the video. I have used OVRIE to have a tracker attached to my hand emulate an Index controller, and I've used the OVRIE offsets functionality to get my hands orientation and position matched up pretty precisely.

 

The hand is not animated at all, but the 3D model is in a pretty standard relaxed hand pose so it's easy to just relax my hand so it matches. I have aligned the real switches and knobs so that when I put the middle knuckle of my VIRTUAL index finger on the right side of the switch or knob the same joint of the same REAL finger is in the same place.

 

In the video I demonstrate how quickly and accurately I can find my knobs and switches. It's very satisfactory. Now the only problem is I have to worry about OVRIE getting broken. =(

 

[Kariyann71]

Based on my previous work, I created a new solution.

This is not a full VR solution as the first one, this is more a VR mouse with some handsignals commands.

Currently mouse is working nicely, I’m continuing some tweaking but I expect a release in the next weeks. Clicks based on pinch and index movements.

About handsignals I succeeded in command catapult launch with salute gesture, I’m working on ejection command,power supply and catapult alignement.

 

Hope coming back soon