Fingers - Leap hand tracking and physical switches for VR and others

[frenzon]

13 hours ago, Sielu said:

Right now this whole solution doesn't work for beans in a certain other popular, WMR-based, non-combat flight simulator because it's cursor implementation doesn't like the winput() commands that Fingers sends to windows.  Outside of going down the rabbit hole of programming new OpenXR controllers entirely, I'm thinking of some way to get the Espruino's to send the PC a mouse change signal to kludge in support for other sims.  Just gotta figure out a way for the Espruino's to know where they are in space.

Before you do anything too drastic, this might be solveable inside Fingers - right now it's calling SetCursorPos in Winput.cs, and I can see why certain games may not receive that. But mouse_event (commented out in the same file) or SendInput might work instead as they generate events differently. I'd test it out, but my MSFS installation is stuffed.

[markbam]

8 hours ago, wju said:

meaning the "HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Holographic" trick or something else?

wired mouse version based on Arduino works fine, but the wire is annoying, maybe I am "the most ugly solution award" winner

anyway, espruinos ordered.

 

 

Allow me to present my current setup for the "most ugly solution" award. It's bulky, but man, does it work well. Leap Gemini tracking has handled this like a champ and I have zero tracking issues (outside the ordinary). I even have haptics buzz on button presses and joystick movements.

 

 

[wju]

8 hours ago, Sielu said:

Nope, this option in WMR settings:

 

ah, these I have also off

[wju]

well, my espruino based  "very ugly proof of concept" version is working! (left finger only so far, second espruino is on its way)

the only issue was caused by my own stupidity;  I have forgotten to plug bluetooth antena to motherboard and therefore spent two hours by thinking experimenting why espruino cannot be connected...

As you can see, I have decided to build "ring" version with lipo battery (230mAh), as I hate gloves, battery can be smaller in the future, but this one was on my shelf

the latching button swtitches on/off, where off can also be a charging mode; charging pins are not soldered yet.

the only weird thing is when mouse becomes  "white dot";  then it behaves oddly, i.e. moves with headset in oposit direction, (I understand why), but one can get use to it...

anyway, guys @frenzon,  @Sielu thanks for sharing!

[m0jo]

Hi, here is my take on it.

Instead of Bluetooth, this is using Nordic Gazell protocol with a small dedicated receiver running QMK. Only the receiver need to be flashed (via USB) if some key mapping need to be changed. This support keyboard buttons, mouse click, mouse wheel (with acceleration) and joystick buttons press. 

The CR2032 hidden on the top of the finger should last a few years since the microcontroller is always sleeping and don't need to stay in sync with the receiver. There is no delay after pressing a key, this is very similar to pre-Bluetooth wireless keyboard/mouse. 

I also added some extra wireless buttons on the VR headset with the same design (but different PCB) for things like centering the view and catapult salute

Edited September 14, 2022 by m0jo

[wju]

On 9/14/2022 at 3:14 AM, m0jo said:

Hi, here is my take on it.

Instead of Bluetooth, this is using Nordic Gazell protocol with a small dedicated receiver running QMK. Only the receiver need to be flashed (via USB) if some key mapping need to be changed. This support keyboard buttons, mouse click, mouse wheel (with acceleration) and joystick buttons press. 

The CR2032 hidden on the top of the finger should last a few years since the microcontroller is always sleeping and don't need to stay in sync with the receiver. There is no delay after pressing a key, this is very similar to pre-Bluetooth wireless keyboard/mouse. 

I also added some extra wireless buttons on the VR headset with the same design (but different PCB) for things like centering the view and catapult salute

 

would you be so kind and share more details? I am very curious...

[m0jo]

14 hours ago, wju said:

would you be so kind and share more details? I am very curious...

The electronic is based on the design of the Redox Wireless mechanical keyboard. The firmware was modified to support the keyboard, and 4 others devices with up to 7 buttons each on the same USB receiver. The receiver is exactly the same hardware, the custom PCBs are pretty much only connecting YJ-14015 boards with NRF51822 microcontroller to the buttons. The receiver is using a Arduino Pro running QMK to handle all the USB HID devices.

Here is the 3d model with the endcap removed (easier to see details than with a photo with the black plastic). The CR2032 battery+holder is on top on a thin PCB and the NRF51822 board is on the back of the PCB holding the buttons. Only 2 wires are used to connect the battery PCB to the main PCB.

Almost the same design is used for the buttons on the headset, but the battery is on the same PCB, and the thickness of the CR2032 make the entire thing a bit too thick to be finger mounted.

Here is the backside of the assembled PCB on the headset (buttons are on the other side): 

The design in the previous post is actually the 2nd version, here is the original version with the battery in the back:

Edited September 16, 2022 by m0jo

[wju]

thanks, very nice. 

[frenzon]

On 9/17/2022 at 9:57 AM, m0jo said:

The electronic is based on the design of the Redox Wireless mechanical keyboard. The firmware was modified to support the keyboard, and 4 others devices with up to 7 buttons each on the same USB receiver. The receiver is exactly the same hardware, the custom PCBs are pretty much only connecting YJ-14015 boards with NRF51822 microcontroller to the buttons. The receiver is using a Arduino Pro running QMK to handle all the USB HID devices.

This looks so awesome!

[j9murphy]

I'm trying to follow this thread, but I think I 'm missing the basic problem trying to be solved.  I *think* what you're trying to solve is that while the leap tracks your hands nicely, it's a little finicky for actually manipulating the controls?  then it looks like you're using leap hands to move the mouse cursor somehow and the finger ring to push the buttons?

[Tehrawk]

53 minutes ago, j9murphy said:

I'm trying to follow this thread, but I think I 'm missing the basic problem trying to be solved.  I *think* what you're trying to solve is that while the leap tracks your hands nicely, it's a little finicky for actually manipulating the controls?  then it looks like you're using leap hands to move the mouse cursor somehow and the finger ring to push the buttons?

Yeah, with the leap controls by themselves, you will end up pressing random buttons and switches. With this you get better control of clicks.

[j9murphy]

19 hours ago, Tehrawk said:

Yeah, with the leap controls by themselves, you will end up pressing random buttons and switches. With this you get better control of clicks.

Got it thanks, I have a pointctrl and a leap, I definitely get better control with the pointctrl but the immersion is better with the leap. I've wondered if there is a way to disable the pointctrl mouse cursor control and retain the button input, then use the leap for the mouse cursor... maybe best of both worlds?  

[markbam]

I really enjoy checking in and seeing what everyone else comes up with!

 

Here's my Version 2: Still not the prettiest, but much more compact and gave me a reason to play around with microcontrollers.

The board is a Seeed Studio XIAO BLE Sense nRF52840 running CircuitPython 8 and emulating both a keyboard and a mouse. (The chip has an IMU but I haven't integrated it for anything quite yet.)

The buttons are a 5-way hat switch: Up is Mouse Wheel Down: Down is Mouse Wheel Up: Left is Left Mouse Click: Right is Right Mouse Click: Press is ESC key.

I loved the feel of the haptics on my V1 (Switch Joycon) so I added a small motor that vibrates when I use the buttons. 

It charges a 50mAh battery via onboard USB-C that I connect with a magnetic cable to reduce wear. It lasts a full day of flying even with the haptics.

The ring was 3D printed out of TPU from a Ring Sizer on Thingiverse, It ended up being the perfect mounting system for my needs, so I didn't design anything further.

 

Edited November 25, 2022 by markbam

[frenzon]

Looking good! I love all these projects!

FWIW the reason I haven't made progress on the rings is that the chip shortage is really making it difficult - I'm now on my third type of nrf52 module (BMD-350), and that has also gone out of stock until sometime late next year so I keep having to redesign them, which, in combination with being slow and lazy, is making it take quite a while.

[slughead]

Here's a thought. Instead of using rings for mouse clicks, how about some foot switches. Left foot, left click, right foot, right click.

[slughead]

Like these USB foot switches that can be programmed as keyboard or mouse buttons. I have some on order. Will report back!

TenYua USB Foot Switch Keyboard Pedal for HID PC Computer USB Action Switch Control Pre-Program Key Functions Mouse PC Game : Amazon.co.uk: Computers & Accessories

[whatthecrappin]

I'm assuming theres a bit more to than buying the  Kokuyo presenter, downloading and running fingersapp.exe?

I know the presenter is working, as on a button press its actually controlling my browser etc, but no idea how to connect it to the fingers app?  its currently just sitting on searching?

 

Edited November 26, 2022 by whatthecrappin

[frenzon]

On 11/26/2022 at 5:15 PM, whatthecrappin said:

I'm assuming theres a bit more to than buying the  Kokuyo presenter, downloading and running fingersapp.exe?

If you have something like the Kokuyo, it will do the mouse clicks by itself and Fingers doesn't need to connect to it - any ring connection message is about the bluetooth rings I made.

Just to check though - you do have a Leap Motion device connected and running?

[whatthecrappin]

I do have Leap running, but it turns out the kokuyo is hard mapped to certain buttons.  turns out you cant remap them to mouse clicks  https://www.amazon.com/ask/questions/asin/B00D63BIAO/2/ref=ask_ql_psf_ql_hza?sort=SUBMIT_DATE#

BUT you've given me a couple of ideas.  I'll try looking for a remapper like joytokey

Or just remap the clicks in the game

Failing that I also have 2 BT rings on order so I'll fall back to them....Im in OZ too, so I'm tipping its going to take a good couple of weeks for ebay to deliver

https://www.ebay.com.au/itm/304702897603?var=603733578872

 

Thanks for the reply though as I suspect even with the new controllers I'll have the same problem anyway

 

 

[frenzon]

5 minutes ago, whatthecrappin said:

I do have Leap running, but it turns out the kokuyo is hard mapped to certain buttons.  turns out you cant remap them to mouse clicks

IIRC, you should be able to map its keyboard inputs to clicks inside DCS - settings > controls > UI Layer > "as left mouse button" etc

Small warning on those bluetooth rings - all the ones I've tried have a similar problem where they only really work with phones.

[slughead]

On 11/25/2022 at 8:33 AM, slughead said:

Like these USB foot switches that can be programmed as keyboard or mouse buttons. I have some on order. Will report back!

TenYua USB Foot Switch Keyboard Pedal for HID PC Computer USB Action Switch Control Pre-Program Key Functions Mouse PC Game : Amazon.co.uk: Computers & Accessories

Reporting back. These are working well. They can be programmed to keypresses, mouse button clicks, string, multimedia buttons, game controller buttons etc. So very flexible. Can be bought for a tenner each! You can get them as single pedals, double pedals or triple pedals.

[firefox121]

En 28/11/2022 a las 10:40, slughead dijo:

Reporting back. These are working well. They can be programmed to keypresses, mouse button clicks, string, multimedia buttons, game controller buttons etc. So very flexible. Can be bought for a tenner each! You can get them as single pedals, double pedals or triple pedals.

I am using this ring mouse for the last four years and with the new fingers software works well without pairing via bluetooth, it has a scroll wheel and two buttons and acts like a normal mouse with no interferences on the other system mouses.

 

https://es.aliexpress.com/item/1005004377392225.html?spm=a2g0o.productlist.0.0.6a082e23dnLRmW&algo_pvid=ac8570e8-24a2-4428-8363-0d6164c59167&algo_exp_id=ac8570e8-24a2-4428-8363-0d6164c59167-8&pdp_ext_f={"sku_id"%3A"12000028965503594"}&pdp_npi=2%40dis!EUR!15.0!10.5!!!!!%402100bde316707173977031513e7c1b!12000028965503594!sea&curPageLogUid=TU4NxCnjW2SP

Hope this helps.....

[R1ckJon3z]

On 12/10/2022 at 7:16 PM, firefox121 said:

I am using this ring mouse for the last four years and with the new fingers software works well without pairing via bluetooth, it has a scroll wheel and two buttons and acts like a normal mouse with no interferences on the other system mouses.

 

https://es.aliexpress.com/item/1005004377392225.html?spm=a2g0o.productlist.0.0.6a082e23dnLRmW&algo_pvid=ac8570e8-24a2-4428-8363-0d6164c59167&algo_exp_id=ac8570e8-24a2-4428-8363-0d6164c59167-8&pdp_ext_f={"sku_id"%3A"12000028965503594"}&pdp_npi=2%40dis!EUR!15.0!10.5!!!!!%402100bde316707173977031513e7c1b!12000028965503594!sea&curPageLogUid=TU4NxCnjW2SP

Hope this helps.....

So you are still using leap motion, but using this as the clicking mechanism instead of making your own device?

[firefox121]

hace 13 horas, R1ckJon3z dijo:

So you are still using leap motion, but using this as the clicking mechanism instead of making your own device?

Yep, more or less, 

sometimes I only use the mouse cursor (displayed when I move head in vr to point and click with this mouse)

Sometimes I use my trackball mouse attached  via magnets to my armchair 

And now, I have moved my Hotas  far from my desk and can use the leap motion without reaching my table or my monitor and only use the fingermouse to turn dials clock or counter wise.

But as you can see...for less than 10 bucks there is no need of DIY rings devices, I am a bit dumb and sure will end with my fingers burned or something worse 

[Indyplumber]

When I click on the Fingers.exe it opens for a few seconds then shuts down.  I must be missing something.