PointCTRL - Finger Mounted VR Controller

[Harlikwin]

  Chic said:

Harlikwin, why no VoiceAttack?

 

Well, I'm definitely on the "simmer" side of DCS so I still use voice attack for the radio menu, and some FC3 plane features (grrr) and like switching on ejection seat switches that you can't see etc. But I found it kinda "dumb" to use it for actual cockpit controls that you should be able to interact with. Point control is generally great for what it is because it lets me "flip the switch" over there like a real pilot would. Its not a perfect solution, but so far its the best thing I've seen for DCS that actually works today. So I have to live with the same cockpit limitations that a real pilot does.

 

And not to detract from point control, which is great for what it is, but in the future for DCS I'd like to see a tracked hand in 3d space interact with the 3d pilot model, so when I move my hand off the throttle, and over to the MFD, I can see the whole arm move. Even more points for some sort of haptic feedback when my hand reaches the switch.

[MilesD]

  Harlikwin said:

And not to detract from point control, which is great for what it is, but in the future for DCS I'd like to see a tracked hand in 3d space interact with the 3d pilot model, so when I move my hand off the throttle, and over to the MFD, I can see the whole arm move. Even more points for some sort of haptic feedback when my hand reaches the switch.

 

I'm with you on that, PointCTRL was supposed to have been an interim device until a fully integrated solution came along.

 

I would like to continue to improve PointCTRL until something better comes along. I have been drinking from a firehouse for the last year with this stuff. I am still amazed how technology has made all this moving electron stuff so easy to conceptualize and effectively implement, for an old knuckle dragger like me.

 

I try to stay in my lane and keep things as simple as possible to effectively accomplish a specific task. With that being said, my current plan is to incrementally improve PointCTRL, and I would like some feedback and suggestions on what our priority of work should be.

 

PointCTRL was developed with and for this community, and I would like this community to improve it and have it move in the direction you want.

 

Here is a top 10 of what I am currently thinking, as long as there is continued interest and no better working alternatives available. They are listed in order of priority. I would love to hear your suggestions.

 

1) Improved cursor resolution.

 

2) Aux VR headset mounted Push-buttons

 

3) Radial Virtual Buttons just outside your FOV for admin task

 

4) Graphic user interface to simplify setup and calibration (VR Head Set specific pre-sets)

 

5) Reduce size of FCU's (2 buttons only, 2/3rds the current length and 1/2 the height )

 

6) Tactile touch panels.

 

7) Integrate VR headset movement so cursor is more fluid and does not move when you turn your head quickly.

 

8 ) New HMS cortex processor and footprint allowing flat mounting to the face of VR headset. (Index frunk, Rift S)

 

9) DCS support for having the arm follow the cursor.

 

10) 2 camera HMS version for 3d interaction

 

Maybe I should start a separate thread for future developments?

 

Thanks,

 

Miles

Edited October 1, 2019 by MilesD

[kaltokri]

I'm very happy with my PointCTRL. And it is so great that you will enhance is furthermore.

 

My personal wishlist and order: 8, 1, 2, 3, 4, 7

 

I'm unsure about 6

I don't need: 10, 9, 5

That are just my thoughts. :)

[FireCat]

IMO. I guess, what is practical and relatively easy to achieve now, in a comfortable time zone for you Miles? Also, benefit the most owner/users of let's call the Mark I PointCTRL.

 

I think firmware/software-based benefits all owner/users. So I think 4, 1, 3, 7, I like the idea of 9 but would that need input from ED DCS?

 

I suppose on a physical manipulation change 8 will be welcomed by many and I love the idea of two-button FCU's.

 

Is it possible as a tempory measure to apply this as a firmware solution by turning one of the buttons off? This type of firmware update is probably last on the list and maybe could be classed as an extra. Which I and I am sure many would not mind paying a small fee to cover the cost of your time. Those of us who want it pay for it.

 

I think Getting the firmware (4, 1, 3, 7) is probably the most important at present to complete the Mark I package.

[Greyman]

With the hand and finger tracking that Oculus is developing for the Quest, and possibly the Rift S, it will be interesting to see how PointCTRL works alongside that. I don't think that the finger tracking will be accurate enough for a DCS cockpit, but the hand images might be useful when used with PointCTRL.

 

Also, I wouldn't have expected that the Quest, with the Quest Link, would be useable for flying DCS, but it would appear that its performance might be better-than-expected, so I guess we will see.

[FireCat]

Considering there are some 32+ 3D cockpits of one type or another helo's/warbirds/fighters etc and more in the pipeline cockpits full of buttons/switches/knobs/levers etc all needing to be adapted with new code to allow hand tracked fingers to engage them, + all the 2D screens, would ED even consider taking on that task?

 

Maybe with new modules, but how many of the present modules would they adapt and how long would it take say, to write the code for the F18 to allow interaction this way?

 

I don't know if anyone does I'd truly appreciate being educated on this subject. Just food for thought.

[Greyman]

I'd guess that if ED did decide to allow finger tracking, they would probably implement it by simulating mouse input, which would mean that it wouldn't need to be implemented for each and every cockpit individually.

 

However, like I said above, I don't expect that this finger tracking will be accurate enough to cope with the complexity and proximity of the many controls in the cockpit. Apparently it may not be that much more powerful than Leap Motion already is but it might still work well alongside PointCTRL.

[kaltokri]

Another thing I would like to have is a FCU with only 4-5 joystick buttons and no IR led, which can be attached to my Warthog Stick and/or Throttle. Like a mini button box. For zooming, kneeboard etc.

 

But maybe number 3 is better for that kind of action?

[MilesD]

I like this feedback,

 

@Chic. I almost have a good lens combo completed for the Static Mount Non-VR PointCTRL

 

@Kaltokri, I had some challenges with the VRB (virtual radial buttons) and PointCTRL as a mouse and keyboard causing a conflict with some keyboards. I am going to try and make PointCTRL be seen as a mouse and joystick as you mentioned, and see how that works. I may have to upgrade the processor to implement the VRB's, as I am up against the speed and memory limits of the 32u4 processor,

 

What are VRBs? VRBs are 16 virtual buttons just outside your field of view. You press them by going slightly off screen at specific intuitive locations and pressing one of the three FCU buttons, like where the multi-player chat would be, or the Huey weapons status panel. With the three button FCU you can have a total of 48 VRBs.

 

@ FirCat, I agree that's it's best to focus on firmware first that benefits all current users first. Your new button mapping will be in the next firmware release. As soon as the GUI is complete, I will add the ability to configure as a two button FCU with the extra button as another function.

 

You can also use both FCUs simultaneously in a complimentary technique. Pressing button 1 on the Left FCU and pressing button 3 on the right FCU is the same as pressing buttons 1 and 3 on a single FCU.

 

Edit: Mariner suggested in the discord using a button to help with the 2d cursor of the comms menu. We could use the extra button for that. It can shift the cursor right to be more user friendly while in the comms.

 

@GreyMan, I would very much like to integrate hands that follow the FCUs for those that would like it.

 

Thanks for your input guys, its really great stuff!!

 

Miles

Edited October 2, 2019 by MilesD

[FoxDelta]

i would vote for 7 , and a wider field of view in regard to up and down bcause its hard to reach the lower back switches at the moment, plus smaller fcu s

[FireCat]

  MilesD said:

I like this feedback,

 

@ FirCat, I agree that's it's best to focus on firmware first that benefits all current users first. Your new button mapping will be in the next firmware release. As soon as the GUI is complete, I will add the ability to configure as a two button FCU with the extra button as another function.

 

You can also use both FCUs simultaneously in a complimentary technique. Pressing button 1 on the Left FCU and pressing button 3 on the right FCU is the same as pressing buttons 1 and 3 on a single FCU.

 

Edit: Mariner suggested in the discord using a button to help with the 2d cursor of the comms menu. We could use the extra button for that. It can shift the cursor right to be more user friendly while in the comms.

 

Thanks for your input guys, its really great stuff!!

 

Miles

 

I look forward to trying out the firmware. That's a good idea from Mariner which FCU right hand I presume? If so, is it possible to have the spare button on the left hand FCU as the zoom button just a thought?

[MilesD]

  FoxDelta said:

i would vote for 7 , and a wider field of view in regard to up and down bcause its hard to reach the lower back switches at the moment, plus smaller fcu s

 

Interesting... The cursor moevement should be the full width and height of your FOV in VR. Is it less in the Index? The cursor may be using the native vertical resolution of the desktop monitor instead of the Index's. Can you move your standard mouse to the FOV limits? Mariner experienced reduced cursor movement when he went from the CV1 to the Reverb before re-calibrated and expanded his cursor limits and targets.

 

Miles

[MilesD]

  FireCat said:

I look forward to trying out the firmware. That's a good idea from Mariner which FCU right hand I presume? If so, is it possible to have the spare button on the left hand FCU as the zoom button just a thought?

 

Yes it is a good idea from Mariner. The comms menu has been frustrating him for a while, he has made some changes to font that helps a bit.

 

Reference the separate functions per Left and Right FCU, I could probably add functions recognized by pressing both of the same buttons on each FCU simultaneously, but I can't recognize which FCU is being pressed.

 

Now that I think about, I probably have room in memory to add hot spots in the field of view for the third button, or if the same FCU button is being press on each FCU simultaneously. Similar to the VRBs, but on screen.

 

Examples would be, point to where the chat window would be and press the unused button and it toggles the chat menu, Esc could be upper left corner, Zoom the center area of the screen, Knee board the lower right, Frame rate counter where that is, ect...

 

I think the suggestions that can't be implemented are just as useful as the ones that are. They get us thinking in a different direction and outside our current box.

 

Thanks FireCat, I think you are onto something really useful here. Code-wise, its lighter to implement, and its more intuitive for the user than the VRB's.

 

Thanks again everyone, please keep the ideas coming because I am now working on the next firmware update.

 

Miles

Edited October 2, 2019 by MilesD

[Harlikwin]

  MilesD said:

I'm with you on that, PointCTRL was supposed to have been an interim device until a fully integrated solution came along.

 

I would like to continue to improve PointCTRL until something better comes along. I have been drinking from a firehouse for the last year with this stuff. I am still amazed how technology has made all this moving electron stuff so easy to conceptualize and effectively implement, for an old knuckle dragger like me.

 

I try to stay in my lane and keep things as simple as possible to effectively accomplish a specific task. With that being said, my current plan is to incrementally improve PointCTRL, and I would like some feedback and suggestions on what our priority of work should be.

 

PointCTRL was developed with and for this community, and I would like this community to improve it and have it move in the direction you want.

 

Here is a top 10 of what I am currently thinking, as long as there is continued interest and no better working alternatives available. They are listed in order of priority. I would love to hear your suggestions.

 

1) Improved cursor resolution.

 

2) Aux VR headset mounted Push-buttons

 

3) Radial Virtual Buttons just outside your FOV for admin task

 

4) Graphic user interface to simplify setup and calibration (VR Head Set specific pre-sets)

 

5) Reduce size of FCU's (2 buttons only, 2/3rds the current length and 1/2 the height )

 

6) Tactile touch panels.

 

7) Integrate VR headset movement so cursor is more fluid and does not move when you turn your head quickly.

 

8 ) New HMS cortex processor and footprint allowing flat mounting to the face of VR headset. (Index frunk, Rift S)

 

9) DCS support for having the arm follow the cursor.

 

10) 2 camera HMS version for 3d interaction

 

Maybe I should start a separate thread for future developments?

 

Thanks,

 

Miles

 

Miles

 

My comments

1. Yes please

2. Not sure what that means.

3. Maybe, not sure how well it would work though. I.e. how far out would it be?

4. Yes, I think this would be very helpful for new folks both on GUI and pre-sets

5. I think they are fine the way they are, but my hands are YUUUUGE I also like double stack 1911's cuz the fit better :)... But maybe as an option

6. Not sure what you mean? Like cougar MFD's?

7. That would be cool

8. I have a reverb

9. Double Yes but I Think you'd need some serious support from ED to do it.

10. Double yes, but I think it might cause some problems (i.e. I can't reach through my desk to get at some switches)

[Solo86]

Hi Miles.

I'm in the 'Now Serving' group at number 152. I'm quite happy to pay now and wait until you have one ready to send out. Please ask Carol to send me an invoice. I have PM'd you as per instructions.

Regards

Peter

[Gruman]

Hi Miles.

I received the package 2 weeks ago, but dint really have the time to test it out.

Yesterday, after the release of the F-16, I took PointCTRL on my first ride.

 

After some hassle to get the Mouse Pointer onto my Field of View (I think my start location in DCS is wierd) it worked great. Little bit of getting used to, but its great. I think I have to make the active area a bit larger, so I can actually press menu buttons :).

 

Thanks.

 

I like a lot of the points of your list.

 

1) Improved cursor resolution.

-> Yes please. :thumbup:

 

2) Aux VR headset mounted Push-buttons

-> That would be nice. To be able to attach a small PCB to your Headset to enable stuff like NVG would be good. I assume the connectors on the PointCTRL is already prepared to enable two extension boards? How many buttons will they each provide? Does this need a Firmware Update or could we already play arround with this if we DIY some buttons?

 

3) Radial Virtual Buttons just outside your FOV for admin task

->That would mean we need STEAM VR running, right? Oculus doesnt support something like this.

 

4) Graphic user interface to simplify setup and calibration (VR Head Set specific pre-sets)

Would be great. Largest hurdle I had to take. But would also mean the same as 3) (SteamVR)

 

5) Reduce size of FCU's (2 buttons only, 2/3rds the current length and 1/2 the height )

-> Especially important in regards to the Hornet Grip, which is quiet uncomfortable with the Finger Tracker. Maybe Touch Sensitive ones instead of clicky buttons?

What about a glove with contacts at the finger tipps? Similiar concept is arround already.

 

6) Tactile touch panels.

-> Whats that? :joystick:

 

7) Integrate VR headset movement so cursor is more fluid and does not move when you turn your head quickly.

-> Would be a gamechanger. Even non PointCTRL Users would like this feature for Mouse Operation :pilotfly:

 

8 ) New HMS cortex processor and footprint allowing flat mounting to the face of VR headset. (Index frunk, Rift S)

-> Yes, please. But that would mean we need to buy a new one:music_whistling:

This would free up the camera on top of my RiftS. :thumbup:

 

9) DCS support for having the arm follow the cursor.

-> IMHO not needed. Would just block the view. And without 3d tracking would look wierd.

 

10) 2 camera HMS version for 3d interaction

-> 3d Interaction would make clicking buttons a lot more difficult... I think the 2d variant is great.

 

Great work :)

[DutchCoolHand]

  MilesD said:

Yes Sir :surrender: , I asked Carol to send your invoice

 

 

Send me my invoice Miles, I want to make sure I get a unit from the next batch.

 

Peter

[MilesD]

  DutchCoolHand said:

Send me my invoice Miles, I want to make sure I get a unit from the next batch.

 

Peter

 

Hey Peter,

 

Carol has your info, We are still waiting on the correct micro-controllers to arrive. Thanks for the offer to pre-pay, but you certainly wont miss out. I really really promise.

 

I just did not realize the wrong boards were shipped to us when I posted the "Now Serving" 125 thru 150.

 

Thanks for understanding, hopefully it won't be much longer,

 

Miles

[MilesD]

Hey Gruman,

Thanks for your time with the feedback. I am guessing your using (alt/enter) full screen mode and your cursors were shifted to the right, I will have to check to see if thats documented.

 

Response to questions

 

  Gruman said:

2) Aux VR headset mounted Push-buttons

-> That would be nice. To be able to attach a small PCB to your Headset to enable stuff like NVG would be good. I assume the connectors on the PointCTRL is already prepared to enable two extension boards? How many buttons will they each provide? Does this need a Firmware Update or could we already play arround with this if we DIY some buttons?

You are correct, the ports on the top of the HMS are for the connector. Each button bank is 3 push-buttons, just like the FCUs.

They are not active yet and will need a firmware upgrade.

  Gruman said:

3) Radial Virtual Buttons just outside your FOV for admin task

->That would mean we need STEAM VR running, right? Oculus doesnt support something like this.

4) Graphic user interface to simplify setup and calibration (VR Head Set specific pre-sets)

Would be great. Largest hurdle I had to take. But would also mean the same as 3) (SteamVR)

The VRB prossesing can all be done on the HMS, without regard to which VR headset your using, but it looks like its going to require a HMS upgrade. The GUI should be compatabe with

the HMS.

Any upgraded components will be available to previous customers at a discount.

  Gruman said:

5) Reduce size of FCU's (2 buttons only, 2/3rds the current length and 1/2 the height )

-> Especially important in regards to the Hornet Grip, which is quiet uncomfortable with the Finger Tracker. Maybe Touch Sensitive ones instead of clicky buttons?

What about a glove with contacts at the finger tipps? Similiar concept is arround already.

Yea the F18 grip can be an issue depending on your hand size and stick location. Deano87 cants his right FCU on a angle to help with it. I went through what felt like hundreds of push buttons, and I am still on the lookout for a lower profile switch that with the same quality and precise feel of the current ones. I wish dome switches felt better. I believe a new two button FCU that is 1/2 the height will mitigate any interference issues.

I do like your glove idea, and as long as there is interest, I would like to add several different options that are all compatible with current PointCTRL hardware.

  Gruman said:

6) Tactile touch panels.

-> Whats that? :joystick:

I have experimented with a flat panel that mounts in front of you in the area of the UFC and MFDs. Basically a stiff lightweight panel with 1/4" of movement that activates the left mouse click when pressed. Its kind of redundant and a bit strange to have two different options to press a button. When I get caught up I will post some pics to gauge interest.

 

Thanks again for your feedback,and your idea for different style wearable controllers is going on the list.

 

Miles

Edited October 5, 2019 by MilesD

[javelina1]

  MilesD said:

Hey Jav,

This is a new one, sorry to hear your havining issues, but we will get it sorted. For starters, PointCTRL stores all calibrations and settings internally on non volatile memory, so there is no need to keep it on.

 

The order of power up or launching programs should not matter because PointCTRL is just recognized as a HID compliant mouse.

 

You can press the white reset button on the HMS at any time and PointCTRL will reboot in about 2 seconds without effecting the saved calibrations or settings.

 

Now on to your problem. It sounds like an issue between Full Screen (alt enter) and windowed mode. Are you using the same mode each time? You can calibrate for either, but you have to use whichever mode you are calibrated for, or it will shift the cursor left or right.

 

If your calibration settings are shifted right, it sounds like you calibrated in windowed mode and when you came back to fly you put it in full screen mode.

 

Its not clear in the crappy instructions. I have to stop saying I am going to work on them and just do it.

 

This may not be your problem, let me know if its not.

 

Also, you should join the PointCTRL discord in my sig that Clutter and Deano set up. I usually log into the voice channel when I am working, and you only have to yell my name and I should hear you. There is a bunch of guys like Deano that jump in and help pretty quick.

 

Miles

My apologies for not closing the loop on this sooner. I followed the guidance from Miles. Plus, I watched his calibration video three times over. All of which helped me sort this out. PointCNTRL is awesome. I've been using it while flying the Turkey and love it. If you fly full screen, perform your calibration in full screen. ;)

 

thanks Miles!

[sirrah]

Received my Pointctrl last Thursday!

 

It can't be said enough: What an awesome product Miles! And thank you, your wife and son for making this all possible for us!

 

 

One quick question, that might or might not have been asked before.

I walked through the calibration proces and the FCU's response and allround feeling is good.

One thing though. When for instance I point my finger at the Hornet UFC (finger a couple of inches in front of me) and move my head to left or right, my cursor drifts rapidly to the opposite side (so it won't stay more or less on the button I wish to press). While I understand why this happens I wonder if there's a way to recude this effect? Or did I do something wrong in the calibration proces?

 

Edit:

I did the pointing part of the calibration with my arms fully stretched. Would it help to recalibrate and do the pointing part with my arm not fully stretched?

(Sorry, I'd just try this out myself if I could, but my DCS time is very limited so I need to put the little time I do have, to good use :) )

Edited October 5, 2019 by sirrah

[MilesD]

  sirrah said:

Received my Pointctrl last Thursday!

 

It can't be said enough: What an awesome product Miles! And thank you, your wife and son for making this all possible for us!

 

 

One quick question, that might or might not have been asked before.

I walked through the calibration proces and the FCU's response and allround feeling is good.

One thing though. When for instance I point my finger at the Hornet UFC (finger a couple of inches in front of me) and move my head to left or right, my cursor drifts rapidly to the opposite side (so it won't stay more or less on the button I wish to press). While I understand why this happens I wonder if there's a way to recude this effect? Or did I do something wrong in the calibration proces?

 

Edit:

I did the pointing part of the calibration with my arms fully stretched. Would it help to recalibrate and do the pointing part with my arm not fully stretched?

(Sorry, I'd just try this out myself if I could, but my DCS time is very limited so I need to put the little time I do have, to good use :) )

 

Hey Sirrah,

You are correct, you should calibrate at the same distance where you will be operating your FCU's most of the time. Basically you want to extend your arm to where you feel your are touching the panels. If you put your finger a few inches in front of your face, you will experience what you mentioned with the cursor drift when you move your head.

 

With a properly calibrated HMS, the cursor should stay on the switch you are pointing at when you move your head and keep your hand still. However, if you move your head quickly the cursor will move off target and then settle back to where you are pointing about 1/4 of second later.

 

I cued up this video to the spot where it demonstrates an alternate method of calibration.

 

This is from the beginning, The sound gets better when I put the headset on. The calibration targets are at around 2:40.

Edited October 5, 2019 by MilesD

[sirrah]

I did see this video before, but now that I experienced it myself, I truely understand it. Will try this approach as soon as I get the time. Thanks for pointing it out again :thumbup:

[MilesD]

  sirrah said:

I did see this video before, but now that I experienced it myself, I truely understand it. Will try this approach as soon as I get the time. Thanks for pointing it out again :thumbup:

 

And, for everyone, be sure you are in the 3d cockpit without and menus active. The 2d cursor of the menus works differently, and will move opposite when you move your head. To use the 2d menus with PointCTRL you should look straight ahead and not move your head. Otherwise it's a bit like a dog chasing its tail.

 

Miles

[Goblin]

MilesD, or anyone...

I’d like to use the upper camera of my RiftS.

Has anyone tried mounting the sensor to the left or right, off center?

Does it mess with tracking? Could this offset be handled in software?