IvyVR - Next-level gesture control in VR - Never use your mouse again

[Wyverex]

I've always wanted to feel more immersed when playing DCS, especially after I switched to VR (~2.5 years ago). Back then, we still had laser pointers to interact with switches and that just didn't feel good enough. I wanted to reach out and do it instead of just pointing at a switch from a distance. That's what motion controllers were always meant for, so I wanted to make use of that.

While hand controller support has gotten much better now, it still leaves a lot to be desired. It still doesn't feel good enough.

That's where IvyVR comes in.

It's an external program that projects two crosshairs into your cockpit and allows you to perform natural gestures on all available switches. Best watch the video to see how it works:

 

I had a first working prototype beginning of 2021 but then life got in the way, I stopped playing DCS etc. Now, IvyVR is finally ready for release. I hope you enjoy it as much as I do! I couldn't imagine ever going back to using the mouse in VR.

Note that IvyVR is not meant to replace your HOTAS, I still use mine and wouldn't give it away. But you can't possibly bind every cockpit function to HOTAS buttons and, frankly, I wouldn't even want to. There is an added level of immersion in physically reaching out to do something, even though VR currently can't give you the full haptic feedback of flicking a switch.

Download here: https://ivyvr.net

Now fully on OpenXR, you only need SteamVR if your headset requires it!

I'm committed to make IvyVR work on as wide of a range of headsets and controllers as possible, so if you have a combination that doesn't work, please let me know in this thread.

Edited July 18, 2023 by Wyverex

[VR Flight Guy in PJ Pants]

Looks promising. Following this thread to check progress.

[Habu_69]

Seems these devices would encumber using the HOTAS.

[Wyverex]

27 minutes ago, Habu_69 said:

Seems these devices would encumber using the HOTAS.

For the most part, your controllers and your HOTAS don't get in each others' way, especially if your stick is located where it should be in the cockpit. I have my stick mounted to the right side of my chair, so for most aircraft it could obstruct access to the right console. In practice I haven't found that to be an issue though. I also have a lateral offset configured (due to my armrests), which helps further with clearance. Sure, sometimes, you need to adjust your position a bit to reach a certain switch but I didn't find that to be a problem so far!

[WipeUout]

6 hours ago, Wyverex said:

currently only works with SteamVR

Looks interesting but giving up on native OpenXR is too much of a compromise for me!

[wormeaten]

This is nice. Also 
I think this should work without the problem with hand recognition in Quest, Pico or even Leapmotion.
Point is all this is have by default function of simulating controllers with just hands. It is also supported in OpenXR.
This could be game changer for using just hands in cockpit because controllers have same problems as hand recognition when you using it.

Just keep doing good work.

[Wyverex]

2 hours ago, wormeaten said:

I think this should work without the problem with hand recognition in Quest, Pico or even Leapmotion.

Possibly! However, I want this to work on as wide a range of headsets and controllers as possible. The point is that you don't have to invest in special hardware just to get a feature that you should already have access to using your motion controllers.

I agree that picking up your controllers whenever you need them is still a bit cumbersome. I've gotten quite used to rest them in my lap and picking them up when I need to but my endgame would currently be something like this: https://www.instructables.com/Etextile-VR-Gloves-for-Vive-Tracker/

It would be very easy to support in IvyVR. But before I start a project like that, I want to get OpenXR working first.

[Wyverex]

Quick update since I've been silent for a while. I've changed my mind and instead of first supporting the remaining aircraft, I've started the OpenXR port. Progress is good and it looks like everything will work just fine (with probably a few snags we'll have to live with). But it's still a lot of work, so it might take a few weeks until I can release it. Then IvyVR should work with just about any VR headset out there and will no longer require SteamVR to run (except if your headset requires it).

[NAM]

If Openxr works!..

Yes, Definitely 

[Swson]

Personally I stopped using my Ultraleap for the reasons you quote. Controllers use is not good because in VR you need to ret them in order to take physical controls.

so best solution I found is the trackball. Only one place to find, very accurate and fast even if it is a bit less immersive it is usable with a motioned cockpit while controllers are not.

[VR Flight Guy in PJ Pants]

I used to use trackball, but then I fix the mouse pointer to the center and use a small mouse attached to the stick as the mouse buttons and scroll wheel.

[jcenzano]

What I have found to be a compromise solution, is to have a 24"" touchscreen located where the instrument panel should be.

Before VR I used it to display instrument panels with Helios, but after switching to VR, I use a small program that converts the whole touchscreen into a big trackpad.

I can program finger gestures like one finger tap for left click, two finger tap for right click, or two finger swipe to turn rotaries left and right.

It requires a little bit of adaptation, but until a "standard" hand controlling method exists, it does the job for me.

Sorry for the off-topic

Sent from my SM-A528B using Tapatalk

[jnr4817]

So this looks promising. Kinda mimics what oculus already has going with native support. Would be nice to get that kinda support with reverb g2 controllers. 

[Wyverex]

13 hours ago, jcenzano said:

What I have found to be a compromise solution, is to have a 24"" touchscreen located where the instrument panel should be.

Before VR I used it to display instrument panels with Helios, but after switching to VR, I use a small program that converts the whole touchscreen into a big trackpad.

I can program finger gestures like one finger tap for left click, two finger tap for right click, or two finger swipe to turn rotaries left and right.

 

I actually started out with a similar concept. I bought one of these 3D touch pads and wanted to use that to control the mouse with finger gestures: https://www.microchip.com/en-us/development-tool/DM160225

But then I found out that you could render overlays into any VR game using OpenVR and that changed everything for me!

[jcenzano]

I actually started out with a similar concept. I bought one of these 3D touch pads and wanted to use that to control the mouse with finger gestures: https://www.microchip.com/en-us/development-tool/DM160225
But then I found out that you could render overlays into any VR game using OpenVR and that changed everything for me!

Looking forward to seeing the progress of your concept, especially if there are no VR controllers involved.

Sent from my SM-A528B using Tapatalk

[Wyverex]

Good news, IvyVR 2.0.0 now supports OpenXR natively!  SteamVR is no longer required unless your headset requires it.

Since my Pimax Crystal is still currently shipping, I could only test this on my Index yet (which does require SteamVR). However, it should work on all OpenXR runtimes and on a large variety of controllers. Give it a try and please let me know if anything goes wrong!

The website has been updated, you can download the new version here: https://ivyvr.net/download/

Some other nice benefits in this new version:

• Recentering now works using the same key as DCS uses (Ctrl+F12). Currently this is hardcoded but I'll add keybinding configuration later if there's any demand for it
• Better performance and reduced input lag. IvyVR now runs in lockstep with the DCS frame loop which feels more responsive
• Hint textures are only as big as they need to be

[bloop]

@jcenzano what software are you referring to? It seems like a good idea, especially as I also have some touchscreens from my Helios days!

 

On 7/1/2023 at 9:27 PM, jcenzano said:

What I have found to be a compromise solution, is to have a 24"" touchscreen located where the instrument panel should be.

Before VR I used it to display instrument panels with Helios, but after switching to VR, I use a small program that converts the whole touchscreen into a big trackpad.

I can program finger gestures like one finger tap for left click, two finger tap for right click, or two finger swipe to turn rotaries left and right.

It requires a little bit of adaptation, but until a "standard" hand controlling method exists, it does the job for me.

Sorry for the off-topic

Sent from my SM-A528B using Tapatalk
 

 

[jcenzano]

16 hours ago, bloop said:

@jcenzano what software are you referring to? It seems like a good idea, especially as I also have some touchscreens from my Helios days!

 

 

I am using this software,although I am sure that there are other similar alternatives.
https://tabletpro.com

In fact, I do not even have the video cable of the touchscreen connected. I onlz have the USB to emulate a trackpad.

I hope this helps

[Hartsblade]

On 7/16/2023 at 9:11 AM, Wyverex said:

Good news, IvyVR 2.0.0 now supports OpenXR natively!  SteamVR is no longer required unless your headset requires it.

Since my Pimax Crystal is still currently shipping, I could only test this on my Index yet (which does require SteamVR). However, it should work on all OpenXR runtimes and on a large variety of controllers. Give it a try and please let me know if anything goes wrong!

The website has been updated, you can download the new version here: https://ivyvr.net/download/

Some other nice benefits in this new version:

• Recentering now works using the same key as DCS uses (Ctrl+F12). Currently this is hardcoded but I'll add keybinding configuration later if there's any demand for it
• Better performance and reduced input lag. IvyVR now runs in lockstep with the DCS frame loop which feels more responsive
• Hint textures are only as big as they need to be

This looks great. I'm an Index user and just wondering if this is working with OpenXR in SteamVR or is it only working with OpenVR in SteamVR?

[Wyverex]

2 hours ago, Hartsblade said:

This looks great. I'm an Index user and just wondering if this is working with OpenXR in SteamVR or is it only working with OpenVR in SteamVR?

Previously it only worked using OpenVR (which implied using SteamVR). Now it only works using OpenXR (and as an Index user that means you're still using SteamVR, just using its OpenXR runtime)

[Hartsblade]

32 minutes ago, Wyverex said:

Previously it only worked using OpenVR (which implied using SteamVR). Now it only works using OpenXR (and as an Index user that means you're still using SteamVR, just using its OpenXR runtime)

Thanks.  That's what I figured, but wanted to be sure.

 

[Wyverex]

I've updated IvyVR to 2.1.0:

- Fixed IvyVR not starting up on various systems. Unfortunately I didn't know about this until very recently. Please let me know if you still can't start it!

- Pimax Crystal controllers should work correctly. There's a bug in the PimaxXR runtime which I had to work around until it's fixed properly in the runtime

 

[Dogmanbird]

I'm presently trying demo. It looks terrific in your video. 

I'm very happy using the DCS default lasers and hand tracking (with leap), but I wish there was a way to disable any interaction except upon a mouse press or turning the mouse scroll wheel. Is it possible to use the leap motion hand tracking with mouse buttons and scroll wheel, instead of the hand gestures, in your app? 

So far I've been unable to get the hand cursors to appear. The ivyvr exe is running and I can see the menu. I've included a screenshot below. I understand most of these should disappear if working correctly.

Does Ivyvr require the leapmotion option to be ticked within DCS? I have tested on and off

I'm running DCS with it's native OpenXR in single threaded mode.

I have the VR controller option disabled.

I'm running the steam version of DCS located in the path below. Am I right to be using the following as the ivyvr DCS install path? 

{
    "DCSPath": "D:\\SteamLibrary\\steamapps\\common\\DCSWorld\\"
} 

I believe mbucchia  was having issues with his toolkit hand tracking option in the latest version of DCS and using native openxr. Could the same bug be affecting ivyvr?

Thank you

Edited October 9, 2023 by Dogmanbird

[Wyverex]

On 10/9/2023 at 8:37 AM, Dogmanbird said:

Is it possible to use the leap motion hand tracking with mouse buttons and scroll wheel, instead of the hand gestures, in your app? 

So far I've been unable to get the hand cursors to appear. The ivyvr exe is running and I can see the menu. I've included a screenshot below. I understand most of these should disappear if working correctly.

Does Ivyvr require the leapmotion option to be ticked within DCS? I have tested on and off

 

Hi, thanks for the feedback!

There is no LeapMotion support in IvyVR as of now. That might be added later. Although it will always suffer from missing haptic feedback which is actually a huge deal. But I can see the desire for using LeapMotion. 

So, the reason why `L DEVICE` and `R DEVICE` are showing is because you presently need to use VR controllers with IvyVR. There are default bindings for a broad range of controllers included. 

Also, the presence of `MODULE` indicates that IvyVR isn't talking to DCS properly. This could be related to you using the Steam version of DCS which I've never used before. Where is the user data located in the Steam version? In Standalone, it's in `Users\<Username>\Saved Games\DCS.openbeta`. I probably need to update the installer to put in the correct path for Steam.

For now you should be able to fix that issue by copying the `Scripts` folder from that directory into the directory that Steam uses for its user data. It contains a LUA script that IvyVR uses to communicate with DCS.

Edited October 10, 2023 by Wyverex

[Dogmanbird]

Hi, thanks for taking the time to reply and provide those details.

I thought leap may be working because of this line one your page "Traditionally, VR flyers have been looking for external hardware solutions like LeapMotion or PointCTRL which solve these issues to varying degrees"  n...but I've obviously misinterpreted it.  

I'll try copying those files should I use the vr controllers in the future.

Thanks again