leapMotion implementation possibility

[Kariyann71]

Hi everybody,

 

 

[2nd June EDIT]

New version 1.20 :

https://github.com/kariyann/DCLeap/tree/master/DCLeap/LatestRelease

Recoded knob function

Degug Log function

Kneeboard feature (advise you to download this https://github.com/rkusa/dcs-scratchpad)

 

Now compatible with Oculus, you’ve to force SteamVR with the --force_steam_vr command line

 

 

 

[19th January EDIT]

New version 1.11 :

https://github.com/kariyann/DCLeap/tree/master/DCLeap/LatestRelease

Better knob behaviour

Ability to show hands

New cursor offset setting

 

[13th December EDIT]

New version available 1.09:

https://github.com/kariyann/DCLeap/tree/master/DCLeap/LatestRelease

Modification of the sensitivity management of the "pinch - pinch" gesture

Also modified the management of the mouse control script, possibility of improvement for Rift and Pimax users.

I'm waiting for your feedback.

 

[23rd November EDIT]

Removed blue text "Right Click Available" and replaced by blue cubes.

Change ESCAPE key to RCTRL + ESCAPE to quit DCLeap, this will avoid parasite comportment when going to Pause Menu in DCS.

Changed Index trigger / click logic.

 

[17th November EDIT]

DCLeap is now available as first v1.0 release. Unfortunately, DCLeap is not compatible with Oculus and PiMAX, I'm waiting feedbacks from HTC users.

You'll get the ability to move your mouse cursor with your hands, set sensitivity on both axes X and Y.

Knob rotation is executed with pinch movement, then cursor is paused and you have to rotate your palm (maintain pinch) to turn the knob. You can set the sensitivity in the main menu.

You can also set the pinch sensitivity in the main menu.

Two new gesture have been added, Recenter (close both right and left fists) and automatic startup (index finger pointing to the sky during 2 seconds. Mind to map this two new command in DCS (see DCLeap Help Menu).

 

[27th Octobre EDIT]

Removed "RightClick avalaible" when no hand are designated to control the mouse.

Added catapult align gesture with thumb up.

Other minors modifications.

 

Be warned that if you change a toggle box, you will need to inform new resolutions before saving, otherwise your changes will not be taken into account (will be corrected in the future).

 

Here's a video showing carrier operation with the Hornet. I added hands in overlay in order to illustrate the gestures.

 

[14th October EDIT]

Pinch gesture and index trigger are now disabled when hand tracking is disabled.

"Right click available" text is now shown when palm is facing HMD

Introduction of catapult shoot gesture (based on the salute gesture), "Catapult Shoot !" is displayed on the right top of the HMD when triggered

Manual update

 

[11th October EDIT]

Updated installation instructions:

1) Be sure to have installed https://github.com/cbuchner1/driver_leap and https://github.com/SDraw/driver_leap

2) Open the 7zip file, uncompress the archive where you want and launch DCLeap.exe.

3) Double click DCLeap.exe

4) Choose windowed mode in the Default Unity window and press Play!

5) You will show the DCLeap's Main menu, and first of all you need to enter the width and height resolution of one of your HMD screen (WMR 1440*1440 - Oculus Rift 1080*1200 - etc...).

6) Press the Save Settings button, it is needed to register your resolution in the registry. DCLeap will use these registry keys.

 

After you can check the help menu to understand the basic gestures.

Click Start to start the application, after that remember to Alt Tab to get DCS window on top of the screen.

You will see 2 colored cubes on top of your HMD device screen, extend all fingers of one hand and face the palm in front of you to disable this hand and control the cursor with the other hand.

You can access the main menu to pause or quit by pressing ESCAPE on your keyboard.

 

Known issue :

sometimes, cursor goes out the DCS window and DCS goes background, so you cannot continue to control the game, just ALT TAB to go back to DCS.

 

Back in 2018

do you think it could be possible to get a cockpit model from blender, export it to unity, add interaction stuff (LeapMotion and InteractionEngine with InputEmulator), compile it to run in Steamvr background ?

The idea is to send keystroke when LeapMotion controler interact with this app, then interact with keymapping used in DCS.

Edited June 2, 2020 by Kariyann71
change post's name

[wormeaten]

Yes it is.

 

 

 

 

It is even possible with Il2 BoS which don't have clicablecockpit.

 

 

 

 

This is just prove of concept and for more information check there.

 

 

https://github.com/cbuchner1/driver_leap

https://github.com/BManx2000/VirtualButtonBox/wiki

[wormeaten]

[Kariyann71]

I know these Apps, that’s why I started wondering to include a 3D model instead 2D model like virtual button use. I succeed in importing f18 cockpit with blender, configuring button collision to send keystroke. All works nicely but now I would like get this unity app to run in background or to be shown as DCS overlay, like virtual button. But as my app is based on 3D model, I’m not sure this could be work. Any idea ?

[wormeaten]

This basically proves of concept and functionality if ED wants to implement VR hands in the cockpit. All this is already built in any module. The only thing we now need is 3D mouse cursors like in this videos.

DCS is a hi-end simulation with the even higher price but unfortunately without finish quality expected under this price range.

 

ED wake up and give us what you charging us. Our dedication is deserving that.

[Kariyann71]

I would like to know the who created VirtualButtonBox, because I don't undestand how to HeadlessOverlay ToolKit.

If he's looking this post...:music_whistling:

[Kariyann71]

Here's my very early dev on LeapMotion integration.

https://www.youtube.com/watch?v=A562fnJOmIg&feature=youtu.be

I've to deal with some camera settings now.

Next step, push button interaction

Edited October 16, 2019 by BIGNEWY
fixed youtube

[wormeaten]

I'm glad you are trying to do something. Wish you the best and waiting for your furder progress to share with us.

[jimiC]

Let me know if u want a tester

That's my il2 video above using the virtual button box (it has since stopped working for me with updates to vive firmware )

Here's my very early dev on LeapMotion integration.

https://www.youtube.com/watch?v=A562fnJOmIg&feature=youtu.be

I've to deal with some camera settings now.

Next step, push button interaction

 

Sent from my SM-G925I using Tapatalk

[Kariyann71]

I’m making small progress but I still have lot of work. But it’s not a problem cause I really want to do something useful with my Leap.

More than testers, it would be nice to get more hands to help me handle with unity and Leap.

If someone want to join I could push him the current project.

[jimiC]

Trying to match a 3d model of an f18 out of game with the in game 3d model seems like an exercise in extreme patience or even futility for both you and then the user when they try to recenter their view in game ...

Is the model you used in blender come from dcs files ? If not they may not even match dimensions

 

Sent from my SM-G925I using Tapatalk

Edited August 26, 2018 by jimiC

[Kariyann71]

leapMotion implementation possibility

 

That’s exactly what I was wondering about. I will give a test with panels like button box but I’m not sure that It will work as I intend.

I want to implement buttons and toggles switches in 3D to interact with like in real life and give a better experience tu users.

Edited August 26, 2018 by Kariyann71

[jimiC]

Yes sorry should have been more clear

 

Perhaps still 3d models (if it works in steam vr) but make separate panels that can then be moved about to adjust for user perspective rather than one rigid cockpit model that can't be adjusted ?

 

Sent from my SM-G925I using Tapatalk

[Kariyann71]

I didn’t have a lot of time past week to continue this project. I’ve got many issues solved recently and I hope have more time this week. I don’t want to promise too much but I expect to post a video of the current state of my progress by the end of this week.

[RudderButt]

I can't tell you how excited I am that someone is taking this project on! I just wanted to leave a message to provide one more voice of encouragement. I can't write code but despite that I am computer savvy. I have a lot of experience in Blender but minimal in Unity. I'm most experienced in Rhino 3d if that could be useful. I have created three Virtual Button Box profiles now and what a pain that is! Although the finished product works great, I would love an alternative.

[BlacleyCole]

Berserk what language are you working in I have visual studio from ms a couple of different versions I’m fluent in vb but after some wrk might be able to get with c I took a. Course in programming that used java about 15 years ago and got an A from Park college. If you want some help.i also have the leap motion device

[Kariyann71]

BlacleyCole,

I'm an impostor, I use Unity with assets, I manage these and only write some piece of C# to get virtual keyboard input.

I spent to many time last days trying to deal with SteamVR / DCS position reference but this drive me crazy. So I recenter my work on getting panels with push buttons.

 

See below :

https://www.youtube.com/watch?v=I6jED9v-8os&feature=youtu.be

 

Sorry for the quality, this is my first VR video.

As you can see, when I try to use UFC Bottom's buttons, nothing is happening.

This piece of code work

        public void PressButtonNumpad7()
       {
           //Press Numpad 7
           sim.Keyboard.KeyPress(VirtualKeyCode.NUMPAD7);
       }

This one doesn't

            public void PressButtonA()
       {
           //Press Enter
           sim.Keyboard.KeyPress(VirtualKeyCode.VK_A);
       }

 

Next step is to set a working knob button.

Edited September 16, 2018 by Kariyann71

[wormeaten]

Nice work Berserk.

 

 

Will be good if some more people join and help with this. It is obvious that this is possible to be done. We just need more support from the community.

I come from the ArmA community where modding is a way of using the game. People doing miracles but here on DCS comunity... I didn't see more inert community than here. Come one people move your self and join in the effort. This could be something beautiful.

[Icebeat]

Nice work, how affect the overlay the performance? If the performance is fine, this could be very interesting to all of us playing VR with a home cockpit (and trying to find the correct switcher).

[Kariyann71]

Currently, the drop I notice is 1 FPS.

[Icebeat]

could you tell us what exactly are you using to overlay the leap hands over the game please or just share your project if is it possible, thanks?

[Kariyann71]

leapMotion implementation possibility

 

I use the BenOtterOVRlay http:// https://github.com/benotter/OVRLay.

Icebeat, you’re right, sharing this project is probably the best way to make it progress quickly and surely better. Indeed I’m very busy a work these past weeks and this fall I will be very busy.

Is there anyone comfortable with the github platform ??

Edited October 7, 2018 by Kariyann71

[Shrike_AU]

How about using something like http://uwyn.com/gamewave/ or other leap motion mouse emulators for mouse control and using DCS's built in mouse pointer for the switch location problem.

[hannibal]

i have leap. they had that update like 3 years ago or something which i tried it again early this year and found that the leap works descent when you place the sensor on the desk, but i place it on my rift and move around, it is effin janky...

 

how is the leap functionality now a days, has there been any recent updates? i bought that when it was kickstarter 4 or 5 years, it has been a dssappointment ever since it came out. they want to put an app store, and it didnt work well to subsitute a mouse.. garbage

[Kariyann71]

How about using something like http://uwyn.com/gamewave/ or other leap motion mouse emulators for mouse control and using DCS's built in mouse pointer for the switch location problem.

 

 

 

I tried this way but I was not satisfied. Gamewave is not so bad but it is optimized to use LeapMotion in the desktop mode, that’s why accuracy was not perfect.

Now, with the Onion version of the leap motion, I’m sure we can do something very great with a head mounted type device.