Kinect PC is coming. Bye bye TrackIR?

[BigBANGtheory]

I have used an MS Kinect (xbox version obviously) and a PS3Eye with FaceTrackNoIR and let me tell you there is a huge difference in accuracy with these and a TIR5_Pro.

 

Don't get me wrong I love what Kinect promises, but I'll believe it when I see it and as for integration with DCS well that not a trivial task I can only see that happening if it really proves popular with the SIM community.

[Alterscape]

I work with Computer Vision as a graduate researcher. I'm more an end-user of CV tools than a developer of novel techniques, but my impression is: Computer vision is painfully slow. TrackIR and other marker-based systems make it faster by giving the computer a much simpler task ("find the 3 or 4 brightest objects in this scene, ignoring everything else, and then recover their orientation given known relationships between them" vs "Find a thing that looks like a face in this video image, and figure out what orientation it is in based on subtle cues that vary from person to person"). I am skeptical that Kinect-based approaches can match the speed/accuracy of marker-based solutions in the near future, although the depth buffer should allow better speed/accuracy than pure CV solutions like the PS3 GT5 setup, or FaceTrackNoIR.

[walker450]

I am curious what the CPU usage differenecss are between TIR and Kinect will be......

[sobek]

The PS3 Eye-toy already supports 60hz at a PC - and I hope that they will use a similar fast camera in the Kinect for PC.

 

The problem with the eye toy being that it taxes the CPU quite a bit, AFAIK.

 

I'm curious to see what API microsoft uses and if developers pick it up quickly.

[DayGlow]

Wait for the PC version! Will be more responsive and definitely will work above 30 Hz . Requirements deffer on PC than console. I know will include efferent Hardware.

 

How do you conclude that it is laggish. Have you seen the configuration settings this man uses?

 

In TIR5 you improve smoothness at the cost of responsiveness also.

 

I mean, no big mouth, just wait to see.

 

What source do you have that they are replacing the 30Hz camera in Kinect for PC? Why would they 2 different inventory and production trails and double ( or more because a faster camera would be more expensive) their costs?

 

In the video it's an Xbox 360 and Kinect as its used in Fornza. It is laggy. You can see the fraction of a second difference from head movement and movement on screen. It is like when smoothness is cranked right up in TIR and your chasing the camera with your head.

 

That's why you turn smoothing down to 0 in TIR. As long as you can get god fps in a game the fact that TIR5 is sampling at 120fps you will never get ahead of the camera and don't need smoothness set at all.

 

I'd love for a good head tracking without any lights, but as it stands Kinect is a downgrade in responsiveness. I'll wait and see what people do with it, but I don't want any lag.

[sobek]

That's why you turn smoothing down to 0 in TIR. As long as you can get god fps in a game the fact that TIR5 is sampling at 120fps you will never get ahead of the camera and don't need smoothness set at all.

 

The smoothing filters are there because headtracking units amplify your movements by a margin of ~10, which means that the natural shiver your head goes through will be amplified as well, which causes the view to go absolute nuts. This in turn results in you not being able to press any buttons or see well, for that matter.

 

Smoothing has hardly anything to do with temporal resolution.

 

If the unit doesn't amplify your movement, as in the racing game displayed, then smoothing is not an issue, however, in a combat flight sim, you need to be able to turn your head +- 180°, which is not feasible without some sort of smoothing algorithm.

Edited November 26, 2011 by sobek

[Revvin]

What source do you have that they are replacing the 30Hz camera in Kinect for PC? Why would they 2 different inventory and production trails and double ( or more because a faster camera would be more expensive) their costs?

 

In the video it's an Xbox 360 and Kinect as its used in Fornza. It is laggy. You can see the fraction of a second difference from head movement and movement on screen. It is like when smoothness is cranked right up in TIR and your chasing the camera with your head.

 

That's why you turn smoothing down to 0 in TIR. As long as you can get god fps in a game the fact that TIR5 is sampling at 120fps you will never get ahead of the camera and don't need smoothness set at all.

 

I'd love for a good head tracking without any lights, but as it stands Kinect is a downgrade in responsiveness. I'll wait and see what people do with it, but I don't want any lag.

 

I'm guessing you've never used Kinect of Forza 4. Forza 4 uses a lot of head sway and bobbing to give the sensation of speed, having played a lot of Forza 4 with the Kinect (the developer hasn't done a good job on implementing it either) that video doesn't really show lag at all in my opinion as you can't reallly tell how much the players head is turning and how much of the view is actually the head bobbing and tilting as part of the game itself giving that sensation of speed. Even on the 360 I find the Kinect fine for small movement, very good alll round but its large quick movements where iit can lag but for head tracking on a PC with the new firmware they are adding to the Kinect then I am cautiously optimistic.

 

It seems you've already made up your mind though that its going to fail, scepticism is fine but I'd hate to think there was any of the old PC snobbery element coming out just because this has come from the console platform.

[DayGlow]

Building your curves properly remove any small head 'shivers' and speed up relative movement the farther you look from center. The smoothness slider dampens the whole curve.

[DayGlow]

I'm guessing you've never used Kinect of Forza 4. Forza 4 uses a lot of head sway and bobbing to give the sensation of speed, having played a lot of Forza 4 with the Kinect (the developer hasn't done a good job on implementing it either) that video doesn't really show lag at all in my opinion as you can't reallly tell how much the players head is turning and how much of the view is actually the head bobbing and tilting as part of the game itself giving that sensation of speed. Even on the 360 I find the Kinect fine for small movement, very good alll round but its large quick movements where iit can lag but for head tracking on a PC with the new firmware they are adding to the Kinect then I am cautiously optimistic.

 

It seems you've already made up your mind though that its going to fail, scepticism is fine but I'd hate to think there was any of the old PC snobbery element coming out just because this has come from the console platform.

 

No, I've simply have gone from TIR1 to 3 to 5 and even though TIR1 ran at 60hz I remember the lag it had compared to the 120hz of 5. I've also seen reviews of people using Kinect in Forza that have used TIR and they all report the lag between moving your head and it happening on screen. It isn't much, but it is noticeable and it's equated to the 30hz of the camera.

 

A firmware upgrade can't increase the hardware specs. I'm too cautiously optimistic and hope its everything that people hope it is, but I do see a serious shortcoming pointed out by other users and my own experience going from a 60hz to 120hz device.

Edited November 27, 2011 by DayGlow

[tjhowse]

Smoothing has hardly anything to do with temporal resolution.

 

Actually, I'm reasonably sure it does. Smoothing works by taking an average of the last n position samples. For exmaple if you have a very large n, say 120 (matching a 120Hz sample rate), your detected averaged position will be about half a second behind reality.

 

In order to make it less jittery they have to stop it responding to small movements, however the software doesn't know if any given small movement is just a jitter or the start of a bona-fide movement. The trick is finding an acceptable middle ground.

 

As for whether Kinect will replace TIR/Freetrack, time will tell. I'm not sure if the 30FPS is a limit of the IR camera, the visible camera or the software. If it's sampling a simple, 6-DOF model (A head), rather than a large, complex 120-DOF (Two people) model it may be able to achieve higher throughput. If it can only manage 30FPS I'm very skeptical it will be of any use to us.

[Revvin]

No, I've simply have gone from TIR1 to 3 to 5 and even though TIR1 ran at 60hz I remember the lag it had compared to the 120hz of 5. I've also seen reviews of people using Kinect in Forza that have used TIR and they all report the lag between moving your head and it happening on screen. It isn't much, but it is noticeable and it's equated to the 30hz of the camera.

 

A firmware upgrade can't increase the hardware specs. I'm too cautiously optimistic and hope its everything that people hope it is, but I do see a serious shortcoming pointed out by other users and my own experience going from a 60hz to 120hz device.

 

I've been a TrackIR user from the fiirst model and the performance for me has come from the increased resolution with each new model. The firmware unlocks a lot of new features that are now available to developers for the first time with this new SDK and firmware but I know you're not big on reading so I'll leave it at that.

[crazysundog]

The kinect also has a mic, so there is a possibility of voice control as well.

 

I would much prefer to NOT have to wear a clip to have my head tracked, I'm anxious to see if this will work as good as TIR4.

[sobek]

Actually, I'm reasonably sure it does. Smoothing works by taking an average of the last n position samples. For exmaple if you have a very large n, say 120 (matching a 120Hz sample rate), your detected averaged position will be about half a second behind reality.

 

A moving average filter is a very simple technique, but there are other options available that work way better under certain parameters. E.g. in picture denoising, median filters are very common.

 

What i meant with the statement you quoted is that better temporal resolution does in no way deal away with the need for smoothing, i wasn't talking about the implementation. The problem is the movement amplification.

 

In order to make it less jittery they have to stop it responding to small movements, however the software doesn't know if any given small movement is just a jitter or the start of a bona-fide movement. The trick is finding an acceptable middle ground.

 

It's not just a simple tradeoff between making the moving average filter longer or having faster response time, as i said, there are other filters available than just a moving average, each of which works best under certain parameters. I don't know what is used exactly for TIR, i doubt it's just a simple moving average, however.

[DayGlow]

I've been a TrackIR user from the fiirst model and the performance for me has come from the increased resolution with each new model. The firmware unlocks a lot of new features that are now available to developers for the first time with this new SDK and firmware but I know you're not big on reading so I'll leave it at that.

 

Wow, don't you have your panties in a bunch. Can you point me where the firmware increases the performance of the camera? From what I've read it makes it able to track movement closer to the device so it works on a desktop vs living room environment. If it does increase the performance of the camera, halejulah! I too look forward to ditching having to wear the pro clip and having natural head tracking with no gear.

 

You don't have to be condescending because we have different viewpoints.

[molleh]

This would be really cool if it pans out. However, I do remain cautiously optimistic. Not only would the Kinect have to prove that it can process input data for a nice lag-free movement in game, but we would need some official support in some form to get good use out of it in game with out hours and hours of configuration and troubleshooting headaches. I can pretty much guarantee MS will have no interest in developing a Trackir-type program for interfacing with obscure hardcore PC flight sims, so that leaves it up to either each individual game developer - and who has time to dedicate to such a project when they have the game itself to worry about? - or end users doing stuff in their spare time, which is cool but leads to buggy and troublesome software.

 

Honestly I'd just be thrilled if Naturalpoint would make a wireless (battery powered) Trackclip Pro and a way to wear it on my head without having to use a headset. Maybe a slim, wireless design on a lightwight and comfy headband would be just the thing.

[BTTW-DratsaB]

Honestly I'd just be thrilled if Naturalpoint would make a wireless (battery powered) Trackclip Pro and a way to wear it on my head without having to use a headset. Maybe a slim, wireless design on a lightwight and comfy headband would be just the thing.

 

 

I agree with the battery thing. This has been possible for a while now. If a tiny rechargeable battery can be put in Nvidia's 3D vision glasses, then NP could do it easily for few small LEDs.

 

I believe the reason they don't is monopoly, but now thanks to Microsoft they will have some competition so they will have to innovate.

 

It can only be a good thing for us customers.

[cichlidfan]

Honestly I'd just be thrilled if Naturalpoint would make a wireless (battery powered) Trackclip Pro and a way to wear it on my head without having to use a headset. Maybe a slim, wireless design on a lightwight and comfy headband would be just the thing.

 

I have not gotten rid of the wire but I have mine clipped to a womans head band and it works like a charm.

 

As for the MS competing with TIR, well...they are still trying to sell phones too!

[Wolf Rider]

 

I can pretty much guarantee MS will have no interest in developing a Trackir-type program for interfacing with obscure hardcore PC flight sims, so that leaves it up to either each individual game developer - and who has time to dedicate to such a project when they have the game itself to worry about? - or end users doing stuff in their spare time, which is cool but leads to buggy and troublesome software.

 

 

It would be amiss of Microsoft to go and relese the Kinect for PC and not incorporate something into DirectX for it for developers to use in their games/ sims.

 

 

 

+1 for the battery thing

[HydroToad]

It would be amiss of Microsoft to go and relese the Kinect for PC and not incorporate something into DirectX for it for developers to use in their games/ sims.

 

That something would be DirectInput, circa 1997.

[CyBerkut]

Well, for anyone interested in getting "wireless" with their Track Clip Pro... Here's a thread on how somebody made the clip wireless on their Logitech G930 (wireless) headset.

 

http://www.sdsquad.com/index.php?topic=2403.0

 

That appears to have been inspired by what Teej had posted in this thread:

 

http://forums.eagle.ru/showthread.php?t=67081&page=6

 

This post over on SimHQ also has a modified Track Clip Pro / G930 combo:

 

http://simhq.com/forum/ubbthreads.php/topics/3389109/Update_on_Obutto_Pit.html

[agrasyuk]

what i really grown to hate about TIR is the cheap LED clip. the thing broke on second day of use, i glued it together several times already. and last night it broke yet again! arrr.

35$ for replacement made from the same ultra fragile plastic that will get broken tommorow? no thank you.

 

when kinect is ready for simming natural point can close its doors.

[sobek]

when kinect is ready for simming natural point can close its doors.

 

Natural Point is one of the biggest suppliers of professional infrared tracking solutions, i think they are going to be fine. :)

 

Target based tracking is going to slowly go out of fashion as performance of hardware and feature tracking algorithms increases, but we are not there yet.

[hassata]

Kinect 2 will be able to lip read, apparently.

 

http://www.eurogamer.net/articles/2011-11-25-kinect-2-so-accurate-it-can-lip-read

[DayGlow]

Kinect 2 will be able to lip read, apparently.

 

http://www.eurogamer.net/articles/2011-11-25-kinect-2-so-accurate-it-can-lip-read

 

lol maybe it will be advantageous to let people beta test Kinect v1 on PC. And wait for v2 :D

[sungsam]

Updated,

 

Kinect for Windows

 

http://kinectforwindows.org

 

Bye-Bye Stone Age of Track-IR.

 

I think ED must look forward to this technology. It will be great for sims.

 

Of course let's see how Natural Point will react in the market. :smilewink:

Edited December 22, 2011 by sungsam