Is there any immersive way to interact with the cockpit?

[dlder]

Now that I have the Crystal, I've decided to tip my toe into the whole VR thing and although the visuals are not on par with 2D (which will probably always look sharper & clearer), the immersion is off the charts, especially with spotting and the... "closeness" to the controls and the cockpit, which is excellent!

But: I've tried using the controllers and even though the tracking especially to the side isn't good enough, the main problem is: switching from Hotas to a controller takes time, you have to find them on the desk and is just a major pain in the a$$.

Also, if you're using the controllers to directly manipulate the switches, it is just way too imprecise!? Like actuating a specific button is kinda luck? -> this means you'll kinda have to use it with the "pointer" and thus it works exactly as the mouse, which is easier and more precise to use.

So, I've mostly used the mouse so far and... it works pretty well. But it's only for my right hand, so I can't toggle anything with my left (sure, I could just attach another mouse, but have you tried using your "other hand" with a mouse? It's sooooo bad^^).

Here are my thoughts (I don't have any of that hardware, so couldn't try it as of yet) and would like to ask your opinions or even better your findings with hardware you've tried:

Handtracking

• would eliminate the need to grab a controller/mouse (which you first have to find when in VR^^)
• but has the same problem of being imprecise when directly manipulating controls?
• so has to be used with pointer?

which brings us to "ring mice"

• "button box" on every index finger to accentuate controls, which you point to via handtracking seems like a VERY good idea! The problem is there's practically nothing you can buy
• PoinCtrl has it's own tracking (which I guess doesn't need to be used) and is more or less unavailable
• there is a Bluetooth version you can make yourself (Hackster.io or Instructables.com), but I wouldn't know how, nor do I know how good it actually works. Looks like the best PoinCtrl alternative though!
• then there is Vanzy; which is only a Kickstarter product and seemingly unavailable. With JoyToKey I'm sure one could remap the buttons to what you need. I'd hope.
• the only other alternative I've found: Aibyks Multifunctional Bluetooth Ring on Amazon (or other, less trustworthy places^^). Don't know how usable that would be, but I'd try it first, seeing the price and all that
• haven't found any other so far...

Haptic Feedback Gloves

• no cheap options yet available
• only bulky gloves priced at 4.000€ and more
• would probably have the same issue as only Handtracking though, right? Like accidentally actuating controls in the cockpit and being imprecise when trying to toggle a switch?

So, I don't know how right I'm with my conclusion being:

• you either use the mouse or handtracking with pointer
• or even more precise: handtracking with ring-mouse

What do you use? What did you try and what's your conclusion? There are some sick setups out there, so there must be a "solution" to this problem (and I don't mean a 1:1 recreation of the cockpit^^).

 

Thanks for your time and hoping for some great tips! Cheers!

[A.F.]

Hi, I use Leap Motion because I don't have hand tracking available with the G2.
Leap Motion is unfortunately only supported by modules from ED.
It works very well (two-way switch and pressure switch and is really fun, plus it's very good for immersion.

In other modules I use PointCtrl. Luckily I was able to get another one. But either you can't buy one at the moment or you have to wait a year for it. I also use a trackball with my left hand. With a little practice it works very well.

My next HMD has handtracking and I will continue to ask ED for better Leap Motion support!

[dburne]

I absolutely love my Point CTRL and use it all the time. I do not fly without it. But yeah getting one currently is problematic, going to take a long while.

[Hempstead]

6 hours ago, dlder said:

Also, if you're using the controllers to directly manipulate the switches, it is just way too imprecise!? Like actuating a specific button is kinda luck? -> this means you'll kinda have to use it with the "pointer" and thus it works exactly as the mouse, which is easier and more precise to use.

I don't have Crystal, but my Quest 2, and now Quest Pro controllers are just as precise as a mouse, if not more so. Works great! So great that I never opened my PointCtl. I will sell you my unopened PointCtrl if you want (if I can find where I put it. ;-).

 

6 hours ago, dlder said:

the main problem is: switching from Hotas to a controller takes time, you have to find them on the desk and is just a major pain in the a$$.

A simple strap work solves the problem, although I have to admit, the right stick hand with a monkey on the back is very annoying particularly in a dog fight. I don't hang the right hand controller though. I only use the right hand controller when I am on the ground doing cold start anyway.

 

[MAThrasher]

@Hempstead I will buy your pointctrl if you are interested in selling it.

 

[Baldrick33]

After much experimentation including PointCtrl I have found a combination of simply looking at controls and using the HOTAS buttons for mouse control and physical button boxes and quadrants the perfect solution for me.

It is surprising how quickly muscle memory kicks into use physical controls you can’t see in VR.

I also use Joy2Key to provide a recentre mouse function with a HOTAS button.

edit to add: I don’t use the VR controllers, they are not even connected.

Edited September 9, 2023 by Baldrick33

[Qcumber]

1 hour ago, Baldrick33 said:

After much experimentation including PointCtrl I have found a combination of simply looking at controls and using the HOTAS buttons for mouse control and physical button boxes and quadrants the perfect solution for me.

It is surprising how quickly muscle memory kicks into use physical controls you can’t see in VR.

I also use Joy2Key to provide a recentre mouse function with a HOTAS button.

edit to add: I don’t use the VR controllers, they are not even connected.

 

This is what I do too. It works really well. The side control panels are a bit tricky but xr/vrnecksafer helps with this. 

[Dentedend10]

I use QPro controllers as well. Accuracy is great, it gives the extra function of zoom and opening the menu as well as can scroll and even imitate touch (with haptic feedback in some modules). 
I do agree it’s not ideal that I have to pick it up from my lap every time but the alternatives I tried (leap motion and HTCC) did not provide the ease of use controllers do. 
I was thinking one day to have some Velcro on the controllers and my shorts so I can just strap the controller to my legs (like a kneeboard) when I’m not using them. Maybe I’ll try it one day

[Hempstead]

9 hours ago, MAThrasher said:

@Hempstead I will buy your pointctrl if you are interested in selling it.

 

Sure... send me an offer. I reserve the right to cancel the deal and refund if the shipping cost is too crazy. I mean, if the shipping cost is like USD $100, then I might as well just keep it.

[Baldrick33]

The reason I don't like VR controllers or buttons on fingers is for me it feels very odd to be waving a controller or my hand around and pressing a button in mid air.

I find using either a stick hat or a switch on a button box just feels more immersive.

For instance adjusting the altimeter or gunsight by looking at the dial and having the mouse wheel mapped to a rotary switch in front of me works well. Of course if you have enough controls you can map the altimeter to a dedicated rotary but it rather depends how many planes you fly to get it in roughly the right place. The mouse option works well if you want a more generic solution.

You can map mouse buttons and wheel functions to multiple physical buttons and switches to cover various parts of the aircraft consoles, e.g left, centre, right or more.

It is oddly more satisfying (for me at least!) flicking a physical switch to turn say the battery on (even if it is just a mouse click mapped to it) then pressing a button on a controller. For aircraft which allow right click for on, left for off then the switch matches the animation perfectly.

I even have all 20 buttons on Thrustmaster MFDs mapped as left mouse clicks. I just look at the virtual MFD button and press an MFD button in roughly the right place, no need for fumbling around counting buttons.

It works for me but I am happy looking at the controls and find I can do this precisely. For everything I need to be operating whilst looking elsewhere during combat I have it mapped to physical buttons & switches.

Edited September 10, 2023 by Baldrick33

[markturner1960]

My solution is using a trackball, which I mount on my right thigh on a pilots kneeboard mount. I do have point control as well, but tend to use the trackball most, its just really easy and right next to my stick. To the OP, I would not advise complicating it with those things you mentioned, most dont work great either. Of course, most controls are on my HOTAS and othe rsim pit hardware, which I can easily access with muscle memory or an easy peek under my Quest pro.

 

[Ala12Rv-Tundra]

32 minutes ago, markturner1960 said:

My solution is using a trackball, which I mount on my right thigh on a pilots kneeboard mount. I do have point control as well, but tend to use the trackball most, its just really easy and right next to my stick. To the OP, I would not advise complicating it with those things you mentioned, most dont work great either. Of course, most controls are on my HOTAS and othe rsim pit hardware, which I can easily access with muscle memory or an easy peek under my Quest pro.

 

do you happen to have a link for the kneeboard?

[Rodeo]

19 hours ago, Baldrick33 said:

The reason I don't like VR controllers or buttons on fingers is for me it feels very odd to be waving a controller or my hand around and pressing a button in mid air.

I find using either a stick hat or a switch on a button box just feels more immersive.

For instance adjusting the altimeter or gunsight by looking at the dial and having the mouse wheel mapped to a rotary switch in front of me works well. Of course if you have enough controls you can map the altimeter to a dedicated rotary but it rather depends how many planes you fly to get it in roughly the right place. The mouse option works well if you want a more generic solution.

You can map mouse buttons and wheel functions to multiple physical buttons and switches to cover various parts of the aircraft consoles, e.g left, centre, right or more.

It is oddly more satisfying (for me at least!) flicking a physical switch to turn say the battery on (even if it is just a mouse click mapped to it) then pressing a button on a controller. For aircraft which allow right click for on, left for off then the switch matches the animation perfectly.

I even have all 20 buttons on Thrustmaster MFDs mapped as left mouse clicks. I just look at the virtual MFD button and press an MFD button in roughly the right place, no need for fumbling around counting buttons.

It works for me but I am happy looking at the controls and find I can do this precisely. For everything I need to be operating whilst looking elsewhere during combat I have it mapped to physical buttons & switches.

 

My summer “project” was going to be build button box for right, center, and left panels with a switch and rotary encoders binding them as mouse commands. Glad to hear that my idea will work now I just need to find time to build them because I for sure didn’t get it down last few months. 
 

Also great idea on the MFDs I am doing that ASAP. 

[markturner1960]

12 hours ago, Ala12Rv-Tundra said:

do you happen to have a link for the kneeboard?

Here you go  https://www.flightstore.co.uk/product/flyboys-kneeboard-with-clipboard-olive-drab-15468

With this trackball   https://www.kensington.com/en-gb/p/products/control/trackballs/orbit-trackball-with-scroll-ring/

Edited September 11, 2023 by markturner1960

[dlder]

On 9/9/2023 at 1:21 PM, A.F. said:

I also use a trackball with my left hand. With a little practice it works very well.

Yeah, maybe I just have to try it; would need a place for my mouse first though.

On 9/9/2023 at 6:15 PM, Hempstead said:

controllers are just as precise as a mouse

Yes, when used with the "pointer" right? With imprecise I didn't mean the tracking of the controllers themself, but the manipulating of switches "directly", without the pointer like with a mouse. Does that work well with you?

Concerning PointCtrl: I'll keep it in mind  (but I'm currently waiting for my leap motion, so wanna try that first)

On 9/9/2023 at 6:54 PM, Baldrick33 said:

they are not even connected.

Yeah, mine too^^

On 9/9/2023 at 8:20 PM, Qcumber said:

xr/vrnecksafer helps with this. 

This is probably the most useful addon for VR... holy moly, it's literally a pain in the neck to fly without^^

 

So... I can just look at the controls to then press a button and activate them? Hm... that's what I'm gonna try next. Sounds not too bad (but a bit imprecise? I mean, you'd need to keep your head perfectly still while fixing on a 5x5mm button^^)

 

22 hours ago, Baldrick33 said:

The reason I don't like VR controllers or buttons on fingers is for me it feels very odd to be waving a controller or my hand around and pressing a button in mid air.

I find using either a stick hat or a switch on a button box just feels more immersive.

For instance adjusting the altimeter or gunsight by looking at the dial and having the mouse wheel mapped to a rotary switch in front of me works well. Of course if you have enough controls you can map the altimeter to a dedicated rotary but it rather depends how many planes you fly to get it in roughly the right place. The mouse option works well if you want a more generic solution.

You can map mouse buttons and wheel functions to multiple physical buttons and switches to cover various parts of the aircraft consoles, e.g left, centre, right or more.

It is oddly more satisfying (for me at least!) flicking a physical switch to turn say the battery on (even if it is just a mouse click mapped to it) then pressing a button on a controller. For aircraft which allow right click for on, left for off then the switch matches the animation perfectly.

I even have all 20 buttons on Thrustmaster MFDs mapped as left mouse clicks. I just look at the virtual MFD button and press an MFD button in roughly the right place, no need for fumbling around counting buttons.

It works for me but I am happy looking at the controls and find I can do this precisely. For everything I need to be operating whilst looking elsewhere during combat I have it mapped to physical buttons & switches.

So, if I understand you correctly: you have physical buttons (like the MFDs) and (more or less) all bound to left mouse click, so you don't really have to find exactly the correct button  with your fingers, while looking at it in VR?

That's... not a bad idea. So you have like one rotary (probably bound as a mouse wheel?) and you look at the correct control in VR and manipulate basically any virtual rotary you want with one physical?

That's genius! Really does sound doable!

 

[Qcumber]

I had a brainwave after reading this thread and realised I have an old Wacom Bamboo tablet. I set it up and did a quick test this morning. It works great. I can keep it just next to the throttle and easily use my left hand to adjust the head cursor for easy access to the side panels. Also a single tap counts as a click and you can then adjust the roller settings by sliding your finger up and down. I need to do more testing but it could be the way forward for me.  

[A.F.]

vor 6 Stunden schrieb dlder:

Yeah, maybe I just have to try it; would need a place for my mouse first though.

 

I have attached my trackball with velcro to my left knee

[Baldrick33]

6 hours ago, dlder said:

So, if I understand you correctly: you have physical buttons (like the MFDs) and (more or less) all bound to left mouse click, so you don't really have to find exactly the correct button  with your fingers, while looking at it in VR?

 

I ended up with a hybrid solution. Physical buttons, switches and rotaries mapped to dedicated controls and some mapped to mouse functions for controls that aren't directly mapped, so I look at the virtual control and use the closest mouse control to the virtual one - be it a button switch or rotary - in the vicinity (e.g. left, centre or right).

Generally I use dedicated controls for things I need to access whilst not looking directly at the control or stuff that is hard to look at (e.g. seat arm, cockpit close, F18 obogs. Mossie fuel controls etc.)

[Hempstead]

7 hours ago, dlder said:

Yes, when used with the "pointer" right? With imprecise I didn't mean the tracking of the controllers themself, but the manipulating of switches "directly", without the pointer like with a mouse. Does that work well with you?

Well, it does have a pointer, and used to have a laser beam too but the MT release broke the laser beam... and I mean using the trigger, buttons, and thumbstick to manipulate the buttons, toggles, and dials. Directly using the virtual fingers to press/flip/rotate is just difficult. For instance, with the F-16 HSI HDG dial trying to set your heading... I use the thumb stick to do that. It's difficult to get within a degree precisely, but I can easily do two degree resolution.

 

[dlder]

8 hours ago, Qcumber said:

I had a brainwave after reading this thread and realised I have an old Wacom Bamboo tablet. I set it up and did a quick test this morning. It works great. I can keep it just next to the throttle and easily use my left hand to adjust the head cursor for easy access to the side panels. Also a single tap counts as a click and you can then adjust the roller settings by sliding your finger up and down. I need to do more testing but it could be the way forward for me.  

That sounds awesome, nice work!

 

2 hours ago, Baldrick33 said:

I ended up with a hybrid solution. Physical buttons, switches and rotaries mapped to dedicated controls and some mapped to mouse functions for controls that aren't directly mapped, so I look at the virtual control and use the closest mouse control to the virtual one - be it a button switch or rotary - in the vicinity (e.g. left, centre or right).

Generally I use dedicated controls for things I need to access whilst not looking directly at the control or stuff that is hard to look at

That is slick, I gotta say.

I'm not quite sure how I'd change my Hotas layout to accommodate for the 2 mouse buttons + wheel, but I'm gonna try that ASAP.

 

1 hour ago, Hempstead said:

Directly using the virtual fingers to press/flip/rotate is just difficult.

Yep, that's what I meant

 

I've just fooled around with the DFR which released on the Crystal today... this is a bit OT, but damn, that works great! Highest setting (like the ones I use for 2D) with my 4090 on Marianas I got ~30fps. With Dynamic Fov. Rendering I now have ~42. And with my default VR settings I get about 55-60 fps.

[Baldrick33]

8 minutes ago, dlder said:

I'm not quite sure how I'd change my Hotas layout to accommodate for the 2 mouse buttons + wheel, but I'm gonna try that ASAP.

I use the CMS hat on the stick (Virpil or TM Warthog).

Forward=right mouse
Aft=left mouse
Right=mouse wheel up *
Left=mouse wheel down *
Push=recentre mouse **

* I actually use Joy2Key to map these to keystrokes which I then map to to mouse wheel in UI layer. This allows the button to repeat, handy for "turning" knobs by holding it in.

** I also use Joy2Key to perform the mouse centring function

[Qcumber]

24 minutes ago, dlder said:

That sounds awesome, nice work!

Thanks. I had another idea. I wonder if it's possible to place multiple track pads in the areas which approximate the positions of the key control panels in VR so that when you reach out you can press a virtual button, similar to what Baldrick33 has. 

[dlder]

1 hour ago, Baldrick33 said:

Forward=right mouse
Aft=left mouse
Right=mouse wheel up *
Left=mouse wheel down *
Push=recentre mouse **

* I actually use Joy2Key to map these to keystrokes which I then map to to mouse wheel in UI layer. This allows the button to repeat, handy for "turning" knobs by holding it in.

** I also use Joy2Key to perform the mouse centring function

nice; I also use Joy2Key^^
So, one can add keyboard buttons in DCS to perform the mouse wheel function? Didn't know that...

 

[dlder]

16 hours ago, Baldrick33 said:

Push=recentre mouse **

I'm sorry, but I can't seem to find that function in DCS... what's it called? Where can I find that? (search in general and ui).

Also, what settings do you use for the mouse-wheel-keystrokes in JoyToKey?
I'm currently using Auto Repeat: 2 times per second.
But it seems not enough for some dials like altimeter... I'm gonna try 10 times next (ever 100ms)

[Baldrick33]

37 minutes ago, dlder said:

I'm sorry, but I can't seem to find that function in DCS... what's it called? Where can I find that? (search in general and ui).

Also, what settings do you use for the mouse-wheel-keystrokes in JoyToKey?
I'm currently using Auto Repeat: 2 times per second.
But it seems not enough for some dials like altimeter... I'm gonna try 10 times next (ever 100ms)

The mouse recentre is a Joy2Key function. Under Mouse (Advanced) set absolute position Target position to X=50 Y=50 and Jump cursor to target

I am using auto repeat 5 time per second for the mouse wheel keystrokes which seems to work well