VR + Leap Motion + 1:1 homemade cockpit

[oleuzop]

Hi everybody,

I was wondering if it would be possible to use Leap Motion in DCS just to see the hands in the virtual world and then click real switches in a physical homemade cockpit.

Do you this is feasible?

Maybe too many problems with physical to virtual world precision?

Regards!

[speed-of-heat]

I’m sure you could… there may be issues  moving your hand through some switches …but … why not just use the switch…

[oleuzop]

I'm thinking about giving it a try with a front panel mockup.

The only problem I find ATM is how to disable the Leap Motion click with index finger so it doesn't interfere with clicking the real physical switch

[speed-of-heat]

So you want to see your hands move…

[winghunter]

I tried this but its not really accurate with leap motion. So you still need to fiddle around. Eventually you get better without leap motion and just muscle memory.

The best solution would be AR passthrough, which is something the Quest is developing.

[oleuzop]

Thank you very much for your reply, really helps me!

Wll research deeper about AR passthrough

 

Regards!

[JayRoc]

 

[oleuzop]

56 minutes ago, JayRoc said:

 

Hi JayRoc,

Yeap, I know that one, even if they are not using DCS (I think it's Prepar3d).

I know Varjo XR3 has been integrated already with DCS, there are also this two guys:

• One guy who is living in HK => https://www.instagram.com/p/CYGlRnfAy_n/
• One italian guy who is doing a 1:1 hornet pit => https://www.instagram.com/p/CYHFx5Wq2ex/

The big drawbacks I find in this implementation ATM are:

• Varjo XR3 is outside of my budget (and I think 99% of sim hobbyst), 6000€ + 1500€ yearly!!
• DCS, AFAIK, doesn't render MFDs or external screens when using VR. I hope ED changes in the near future, but I think it's a super big problem at the moment. The italian guy has fixed this masking the MFDs to show the virtual world, but I think it's kinda rough.

From my point of view, the best, would be to have some kind of way to place hands inside the simulation with high precision so it matches a 1:1 cockpit with the switches. No need to do weird mix of webcam (that you loose resolution when checking MFDs) and virtual world. I'm researching how to do it and maybe I give Leap Motion a try... it's not expensive at all, so I think it's worth the try!

Kind regards!

 

Edited January 15, 2022 by oleuzop

[oleuzop]

BTW, I don't know if it has been posted before here... and all the credits go to the author of the video but:

 

[JayRoc]

Yeah thats way past my budget, aswell. But give it a couple of years and it might be in Consumer range.

My brother just got the Leap Motion thing because booth of us were excited about it.
But its pretty much work in progress and due to positioning the device has a hard time tracking you hand properly when you reach to mfd´s.
The whole tracking is lagging a little bit. 
My guess is that the Ultra Motion 170 is way better suited...but its also a bit pricey.

I had the same idea as you do. But i switch airframes a lot, so a small fake cockpit wouldnt suit me.

[edmuss]

1 hour ago, JayRoc said:

Yeah thats way past my budget, aswell. But give it a couple of years and it might be in Consumer range.

My brother just got the Leap Motion thing because booth of us were excited about it.
But its pretty much work in progress and due to positioning the device has a hard time tracking you hand properly when you reach to mfd´s.
The whole tracking is lagging a little bit. 
My guess is that the Ultra Motion 170 is way better suited...but its also a bit pricey.

I had the same idea as you do. But i switch airframes a lot, so a small fake cockpit wouldnt suit me.

With the leap motion tracking, the trick is to have the IR cameras able to define the outline of your hands. The 170 unit has the same tracking (as that's handled by the software) just in a much wider area

I was having terrible problems with the MFDs, essentially (with help from ultraleap) I found out that there was too much IR bounce back from the surfaces below my hands; the wood of my desk was too reflective. I covered the surfaces with some 3mm thick self adhesive closed cell foam sheet and it's now just about perfect.

Open the ultraleap visualiser and put your hands in the positions that track badly, pay attention to the background behind your hands in these areas, if it's black you're good, it is white it's bad; ideally your hands should be a bright white shape against a black background.

Since I've sorted that, the tracking in DCS is actually rather good and most of the A10c cockpit is fully operable using real life hands to make pretend hands flick pretend switches

[oleuzop]

4 hours ago, JayRoc said:

Yeah thats way past my budget, aswell. But give it a couple of years and it might be in Consumer range.

My brother just got the Leap Motion thing because booth of us were excited about it.
But its pretty much work in progress and due to positioning the device has a hard time tracking you hand properly when you reach to mfd´s.
The whole tracking is lagging a little bit. 
My guess is that the Ultra Motion 170 is way better suited...but its also a bit pricey.

I had the same idea as you do. But i switch airframes a lot, so a small fake cockpit wouldnt suit me.

You are the second person that tells me that Leap Motion is not optimal... so I guess I'm droping the idea of using it and not even giving it a try.

I mainly fly F18 but yes, as you I switch airframes from time to time (and whenever AH-64D gets out I'll switch for a while, for sure!) so... In the end I think I'll go for PointCtrl (waiting to receive it).

I was thinking, probably as you, if I've understanded you right, doing fake cockpit (without switches, just structure and empty panels) to use either gloves or PointCtrl as another option.

Still evaluating the posibilities.

[edmuss]

Leap motion is getting better and is more than useable for most cockpit controls now, a good portion of it is getting the setup right, a bit like dcs vr in general

[JayRoc]

Dont get me wrong. What Leap Motion does is awesome...but aslong as i am way faster with looking at it and clicking my mouse i dont see a point in using it.
The Leap Motion Demo works also way better than the DCS integration. It will get there and it will be good. But its not there yet - its a nice gimic atm. imho.

[Bvv.Fr]

i want exactly the same thing @oleuzop, i just want to see my hands without interactions with the cockpit, disable click/touchs/interaction.

Someone find a solution?

we can enable and disable the Leap motion with command line with different config.json, but i see nothing for just disabling interaction and still show the hands, to use with 1:1 "real" cockpit in VR.

Edited February 23, 2022 by Bvv.Fr

[edmuss]

6 minutes ago, Bvv.Fr said:

i want exactly the same thing @oleuzop, i just want to see my hands without interactions with the cockpit, disable click/touchs/interaction.

Someone find a solution?

we can enable and disable the Leap motion with command line with different config.json, but i see nothing for just disabling interaction and still show the hands, to use with 1:1 "real" cockpit in VR.

 

As long as you turn pointers off and don't use your index finger to press buttons then it should work as you want.  I'm pretty sure that it's only the index fingertip that can actually interact with the controls, other fingers just pass right through them.

[Bvv.Fr]

i want to use my index IRL ^^

maybe we can use the Interaction_box parameter?

 

 

[GayCavalier]

On 2/23/2022 at 11:23 AM, Bvv.Fr said:

i want to use my index IRL ^^

maybe we can use the Interaction_box parameter?

 

 

Only 11 months to reply, however:

I've just set up a Winwing UFC/HUD and 3 MFDs in F/A-18C hornet geometry.  

I got a large piece of cardboard and taped it to the front of my desk where i was expecting the controls to be located.   I sat with my 8kx with hand-tracker installed and adjusted my seat and 'VR cockpit head cam' position until I had my RL hands sat upon my RL knees and VR knees.  Once done, I then moved my seat backward and forward until my index finger touched the RL cardboard as the same time as the VR UFC.  Dear Lady Wife then used a sharpy to circle my finger position on the cardboard as I called out tags such as, "UFC 1" or "RDDI gain down".  Once mapped, I precisely recorded the position of the cam, and the front of my chair from the cardboard.   I located the UFC and MFDs according to where I was touching.  I now have my seat about 1cm back from where it was, such that my index finger doesn't actually touch the VR button, but does touch the RL button.  If you turn off the green laser pointers, then the job is complete.  I also mapped button 12 on my F/A18-C thrustmaster joystick (the little one that is a push button and up/down rocker), such that I can turn on and off the left and right VR hands independently, through the UI-layer mapping.   I only 'need' the hands when working the radio or the MFDs.  I have all of my other buttons mapped to be toggled through voice-attack - so for startup i just look at the button in question and say, "Battery ON" or "Engine Crank LEFT", for example.

As EDMUSS stated, getting rid of obvious IR reflection really helps stop the ultraleap hands from disappearing or bouncing around on the screen.