TrackIR Killer: Facial AI Tracking Using WebCam

[Ghiath]

Do you play DCS but you don’t have an IR Tracker or VR headset? have you thought about buying an IR Tracker or VR but you are hesitating because of their cost? Then we are on the same page. I’m in the same situation. I kept searching for alternative and found finally a very good one that is very comparable (if not even better than) IR Tracker. I presenting to you the Facial AI Tracking. I had my doubts at first until I tried it out myself on DCS. Yes, it is working just perfect. Here is how you need to set it up. it is totally free, yes, it is totally free and takes just about 5 minutes or even less to configure it. depending on how much do you want to reconfigure.

Let’s start:

• Download the opentrack 2022.3.1 from Releases · opentrack/opentrack (github.com). Note, I tried the latest version opentrack 2022.3.2 but it contained a trojan, don’t use that one. The 2022.3.1 works just fine.
• Download the AI Track v0.7.01 from Releases · AIRLegend/aitrack (github.com)
• Run Setup and install opentracker.
• Unzip the AITrack and run AITrack.exe
• Follow the configuration instruction shown on this video (382) aitrackinstallsetup - YouTube

Thant’s it, run DCS and enjoy. You might need to do some adjustment to that configuration/mapping in openTrack when you are in DCS until you are happy with it.

Note: some Anti-Viruses have a webcam protection feature. I need to disable it on my machine before I started the AITrack and OpenTrack. You can re-enable the webcam protection when you don’t use them.

[Mike_Romeo]

There are other solutions that might work better. Facial headtrackers have often a very hard time to recognise and to track a complex human face. I recommend to use Aruco headtracking instead where you mount a QR code on your head which is much eassier for AI's to recognise and to track.
 

 

 

Edited December 27, 2022 by Mike_Romeo

[Ghiath]

Thanks for the feedback. My intention is to get rid of anything that I need to mount on my head  

The intention behind using the AI Track in combination with opentrack is to get better facial recognition. I used for some hours so far and I had no issue with it what so ever. You can test the facial tracking and recognition with AI Tracker if you enable the preview option before you use it with opentrack. But you suggestion is also a good alternative for those who might have no issue with having a QR Code on their forehead while they are gaming.  

[MadKreator]

Just tried it out. Actually works quite well for just facial recognition. It is a bit delayed but definitely useable. My webcam says it will do 60 fps but I feel its only running at 30 so that could be it too..maybe its just the way open track works.   I do use Track ir and its camera is 120 fps so naturally its much smoother. I think the ps3 cam is 90 if I recall, I didn’t try that one. But all in all it does work and seems like a good step for anyone who wants to try head tracking. 

[EmJay22]

I'm new to DCS though for years wanted to get Track IR for IL2.  But since I have a spare webcam laying around (Logitech 9000 Pro) I decided to give this a try before going the Track IR route.

It seems to work REALLY well.  I don't notice really any lag. 

But I am not sure how to change some of the curves or other settings. 

When I look up, down, right and left, it is fine since I inverted pitch. 

But if I move my face closer to the camera, the cockpit moves back away from me and if I move back, it gets closer. Not sure how to reverse this.

And of course, looking very far away from the screen to right or left as you might in a real plane doesn't work if you can't see the screen.  And I think less input when looking down would be super helpful just not sure which setting for that either. 

If someone could share some settings it would be very helpful... or maybe point me to some. 

[MadKreator]

For the zoom in and out, invert the Z axis, just as you did the pitch. There is a place in opentrack to adjust the curves/ sensitivity and such.. A week ago I could have actually helped when I was playing around with it lol Now I deleted it so hopefully someone else will chime in. But it can be done for sure

[Nahen]

For $35, you can have a ready-made headtracker kit based on a PS3 camera and IR LEDs. It works perfectly, I dare say after about 3-4 years of use that it is as good as the original "Track-IR"
You have two versions of the power supply - a cable via the USB port or AA size batteries. Instead of tinkering with some inventions, it is better to buy something like this or find someone who will build something like that for similar or less money.
I tried some "face tracker" once, it's a waste of nerves and health for lamps/spotlights shining straight into the eyes - every change of lighting caused a problem. Not worth it.
I know that there will be a lot of people who will defend such inventions ... but probably only someone who has not used the original Track-IR or such as the self-made one I described, may claim that it is not worth spending such money. I wouldn't spend money on the original Track-IR either. I to this day spent it on Thrustmaster Warthog and Coguar sets, T-16000 set plus Oqulus goggles. Enought.

The original Track IR costs about $250, and this costs the same as the DCS module - $35.

[Hiob]

54 minutes ago, Nahen said:

For $35, you can have a ready-made headtracker kit based on a PS3 camera and IR LEDs. It works perfectly, I dare say after about 3-4 years of use that it is as good as the original "Track-IR"
You have two versions of the power supply - a cable via the USB port or AA size batteries. Instead of tinkering with some inventions, it is better to buy something like this or find someone who will build something like that for similar or less money.
I tried some "face tracker" once, it's a waste of nerves and health for lamps/spotlights shining straight into the eyes - every change of lighting caused a problem. Not worth it.
I know that there will be a lot of people who will defend such inventions ... but probably only someone who has not used the original Track-IR or such as the self-made one I described, may claim that it is not worth spending such money. I wouldn't spend money on the original Track-IR either. I to this day spent it on Thrustmaster Warthog and Coguar sets, T-16000 set plus Oqulus goggles. Enought.

The original Track IR costs about $250, and this costs the same as the DCS module - $35.

I'd beg to differ. I would love to get rid of the stuff "attached to my head" - wireless or not. Unfortunately I have yet to find a facetracking solution that works as good as trackir-kinds.
I will try this one probably, but I agree that from a "it just works"-pov, trackir-kinds are still the best.

[Nahen]

I assume that each of us uses headphones when flying in DCS... So we won't get rid of it anyway. And if so, devices weighing 100-200 grams, mounted on headphones, are not a problem. I gave up the battery in favor of power supply via the cable, because I have a headphone cable anyway, so I don't mind the other one "connected" to it. I'm wondering whether to mount the diodes themselves in the HGU-55 replica as well as the headphones and cables and all put in the pipe from the oxygen mask. But It's just that I prefer to fly in headphones than in a helmet

[Solemn-laugh]

If you have an iPhone you can use that to track your face for DCS and other games.

There are several apps that do this, but I've tried this one.

https://apps.apple.com/us/app/head-tracker/id1527710071

It's free to try, and if you like it, the full version is (I think) $1.99.

I use VR now, but this is useful for some games that don't support VR.

Not sure if there are apps like this for android phones too?

[Gasman6]

6 hours ago, Solemn-laugh said:

If you have an iPhone you can use that to track your face for DCS and other games.

There are several apps that do this, but I've tried this one.

https://apps.apple.com/us/app/head-tracker/id1527710071

It's free to try, and if you like it, the full version is (I think) $1.99.

I use VR now, but this is useful for some games that don't support VR.

Not sure if there are apps like this for android phones too?

Hello,

 

I found two similar apps for Android, one for $1.99 and the other for $9.99. I had a couple bucks in Google Play credit, so I opted for the $1.99 app (since it was technically free). I installed OpenTrack, followed the instructions in the app, and it seems to work quite well. However, I can not properly test it within the game environment yet, and tweak the settings, until I have a way to mount my phone over my monitor. I'm going to purchase a gooseneck-style clamping mount and affix it to either my desk or the spine of my monitor stand and give it a go.

 

I used to have a TrackIR 5, but it sadly did not survive one of the many moves I've had to make over the past few years. Thank you for the suggestion. I'm excited to give this a go.

[vr2viv]

I have tried TrackIR & Facetracknoir, I will say TrackIR is much better than Facetracknoir, I guess TrackIR will knock down all similar products.  But end up I gave up all similar devices/methods included TrackIR because i really hate it when i'm turning my head and still have to stare at the screen.

[Hiob]

18 hours ago, Nahen said:

I assume that each of us uses headphones when flying in DCS...

Why would you assume that? I'd rather fly without, unless I'm in Multiplayer and need a mic.

[Nahen]

2 hours ago, Hiob said:

Why would you assume that? I'd rather fly without, unless I'm in Multiplayer and need a mic.

I assume that the vast majority of people have families, do not live alone and this is reason enough to use headphones. And besides, when you play with headphones, you definitely "hear more".
Anyway, if someone doesn't use headphones, it's their right.

So, taking the correction - anyone who plays with headphones should not feel uncomfortable because of the head tracking devices attached to them.

If someone does not use headphones, let them get tired of "face track" inventions that will never work like any devices based on IR diodes or laser.

For obvious reasons, I omit virtual reality goggles.

9 hours ago, vr2viv said:

I have tried TrackIR & Facetracknoir, I will say TrackIR is much better than Facetracknoir, I guess TrackIR will knock down all similar products.  But end up I gave up all similar devices/methods included TrackIR because i really hate it when i'm turning my head and still have to stare at the screen.

So you had a misconfigured software for the device. That's the idea behind Track IR and its likes, to configure movements so that minimal movements allow for the full range of cockpit view. That's why, for example, when I fly PvP, I never fly in VR goggles, only on Track IR. This gives a huge advantage over goggles - but it's definitely not real. If you configure the software correctly, the range of head movements should be limited to about 10-15 degrees of movement to the left / right and up / down, which is quite comfortable and does not require a "turning the eyeballs"

[EmJay22]

22 hours ago, Solemn-laugh said:

If you have an iPhone you can use that to track your face for DCS and other games.

There are several apps that do this, but I've tried this one.

https://apps.apple.com/us/app/head-tracker/id1527710071

It's free to try, and if you like it, the full version is (I think) $1.99.

I use VR now, but this is useful for some games that don't support VR.

Not sure if there are apps like this for android phones too?

I just tried this with my iPhone 11 Pro and it works surprisingly well.  Need to play with placing it just right. 

Of course my "old" phone is a 6 so won't work with that one.  It also works on my new iPad though positioning it is a bit of a problem. 

I have ordered TrackIR as it looks to be a lot easier to use.

 

Edited January 6, 2023 by EmJay22

[EmJay22]

23 hours ago, EmJay22 said:

I just tried this with my iPhone 11 Pro and it works surprisingly well.  Need to play with placing it just right. 

Of course my "old" phone is a 6 so won't work with that one.  It also works on my new iPad though positioning it is a bit of a problem. 

I have ordered TrackIR as it looks to be a lot easier to use.

 

 

And since I needed to upgrade to a new iPhone, which is arriving today, I can use the older 11 Pro for this. 

It's a simple matter of using the old holster to position it correctly.  No hat or headset needed. Just power for the phone. 

[vr2viv]

On 1/6/2023 at 7:31 PM, Nahen said:

So you had a misconfigured software for the device. That's the idea behind Track IR and its likes, to configure movements so that minimal movements allow for the full range of cockpit view. That's why, for example, when I fly PvP, I never fly in VR goggles, only on Track IR. This gives a huge advantage over goggles - but it's definitely not real. If you configure the software correctly, the range of head movements should be limited to about 10-15 degrees of movement to the left / right and up / down, which is quite comfortable and does not require a "turning the eyeballs"

I understand what you mean and I tried it when I first got the TrackIR, but I got airsick.

[cordite]

OK, why does it have to be a Track IR “killer”? What’s wrong with competition?

[Gasman6]

14 minutes ago, cordite said:

OK, why does it have to be a Track IR “killer”? What’s wrong with competition?

Hello,

 

I don't think the OP meant any ill will towards NaturalPoint. By saying something is the "TrackIR killer," it is simply a casual use of marketing hyperbole to convey potential open source alternatives that do not have to cost ~$150. In a hobby that already digs deep at the wallets of its most avid enthusiasts, viable competition is obviously a good thing, otherwise this thread would not exist.

[Nahen]

There is no chance that it will be a competition Unless, as I wrote, someone likes to fly in a fully lit room, or with a lamp pointed straight at the face as in an interrogation ...
All "facetrakers" need a very well lit face, any shadow causes jumps, skips, positioning problems in general.
All devices based on IR diodes or laser will have the advantage that after proper setting, nothing will disturb them, regardless of the lighting. Yes, extreme cases, when, for example, bright sunlight falls through the window and is reflected on some smooth / shiny surface will interfere with work, but these are some extreme individual cases.

[Gomez Adams]

I'm using AITrack and OpenTrack with a simple Logitech C920 webcam. It works amazingly well. I'm in a completely dark room. I was so suprised with how well it worked out of the box, I made a video of it to show a friend.

Here it is:

 

Keep in mind this was the first time I started up the program just yesterday. I was adjusting it as I went, which is why it jerks around quite a bit from time to time. I've since played around with the mapping a bit and made it even better.

No, it's not Track IR quality I'm sure, but for free I don't think anybody can complain about the performance.

Edited April 22, 2023 by Gomez Adams

[vladimusmaximus]

It's a shame this project was abandoned. Tracking algo's have gotten better but the project hasn't changed.

[Gomez Adams]

UPDATE:

I finally got TrackIR 5 with the infrared headset attachment bundle. It is light years better than the free OpenTrack / AITrack combo. It's not even in the same class.

So for free, sure. The OpenTrack / AITrack combo is cool. It'll do in a pinch. But once you get the TrackIR 5 bundle, you'll kick yourself in the butt for having not done it in the first place. Far, far, far superior in every way.

[SharpeXB]

On 1/5/2023 at 6:34 AM, Nahen said:

The original Track IR costs about $250

Not sure which currency you’re referring to but in the US TrackIR costs $149.95, the TrackClip reflector is $9.95. 

[cj43g3r]

Wanted to mention that opentrack has built in Kinect Face support for Kinect V2. But only if you can get it at a reasonable price from a reputable seller.

I originally used AITrack v0.6.5 which worked well with a PS3 Eye but is sensitive to differences in ambient lighting conditions (backlighting in particular and works best with overhead lighting) and required adjustment in gain and exposure at different times of day. Its a good low cost solution and I did not want to wear a hat/headset for an extended session.

AI Track v0.7.0 and later uses a different face model and attempts to optimize performance by reducing the detection rectangle surrounding the face. This is ok when sitting at a desk but not at a distance.  I am using it on a sofa approximately 2 meters away from a tv and this causes detection problems.

I was intending to use kinect to do IR face tracking without wearing any clips. But Kinect Face does video face tracking with auto gain/sensitivity which works without any adjustment. From a completely dark room only lit by the tv to a brightly lit sunny room.

The caveat is the Kinect V2 is a discontinued product and you should only consider it if you can get one at a reasonable price from a reputable seller. I bought my Kinect V2 for $15 and its power adapter for $25 and have been using it for around 2 years. Since the Kinect is much larger than a webcam, you will need to mount an above tv shelf to support it.