Full cockpit + VR?

[TripRodriguez]

Hey guys. I know that there is an odd kind of conflict between VR and full cockpits, but I've got a different approach that I'm really surprised not to see other people talking about.

 

As a home pit builder myself what I consider the ultimate solution is to be able to fly in VR, but still have 100% use of the cockpit without any fumbling around.

 

There are two solutions for this:

 

1) Quick switch to pass-through camera, operate cockpit via that.

 

2) Accurate hand tracking with a cockpit perfectly matched up to the VR cockpit. This is my preferred method and the one I'm trying to get set up myself.

 

I think it's the perfect solution. You reach for that switch or knob in VR, able to see your hands exactly, and when your hand gets there, the real thing is there under your fingers! VR haptics can't get any better than that!

 

Has anyone else been working on this? The obvious options are Leap Motion and various tracked gloves available as "dev kits". I purchased Vive Trackers for my hands for this purpose. My early tests were promising, but the big drawback is that there is no animation for the hands so you can't actually see where your fingers are. Only solution for that so far is to always keep my hand in the same position as the static pose of the VR hands LOL. Also a side note, it is a pain when you first load into the aircraft to carefully adjust the vr view to match the cockpit. Not a deal breaker by any means, but an automated solution would be awesome. Put a Vive Tracker on your sim cockpit somewhere and in-game match up the same location on the virtual cockpit and attach the two? That's definitely getting ahead of ourselves though, one step at a time Trip!

 

There was a guy who had a working Leap Motion hand tracking plugin for SteamVR over a year ago. In the next few days I'm going to test and see if it still works with the current version. I think he gave up because what he was really after was "FlyInside" style cockpit interaction, which is an excellent goal, but all I need for this project is to see my hands! There's also the Leap Motion plugin for SteamVR to try. That might be the better option anyway, and I'll be testing that in the next few days as well. My biggest concern with Leap Motion is that the positional tracking may not be consistent enough.

 

So let me know what you guys think about this. I'll report back on whether or not the leap motion thing works for me. If it does, I'll be working on finishing my Huey cockpit. =) I'll also

in the future be building a pit for one of the allied WWII aircraft, and at least one jet. Probably the 'hawg but maybe the F-14 or the viper. I'd have a hard time talking myself into a Viper pit without getting my hands on a force sensing stick though...

 

Building pits for this setup will be less work because you don't need all the fancy appearance stuff, just plain panels with all the knobs and switches will basically do the job.

 

I personally have an additional complication in keeping the real cockpit and the virtual one matched up because I'm also on a motion simulator! With proper motion compensation it's still totally possible but I do worry about things getting a bit out of alignment with all the variables in play.

Edited June 14, 2019 by TripRodriguez

[matbog]

Building pits for this setup will be less work

 

Not necesserily. I guess it is not so easy to build a pit which is 100% the same as the virtual one. Think about the perspectives and the fact that the virtual cockpit is not always at the same dimension that the real one. I see lot of trial and errors in the conception.

 

Another solution could be a geodesic dome covered with green fabric all over your cockpit. You can add a camera (2 for 3D perception) on you VR goggles which has an optic with the same field of view than the field of view you have in the simulator. And you can use a software which insert the virtual view wherever you have green color on the camera image. The final image is pushed in your VR goggles.

Just another idea...

[Gladman]

I bought 1 panel to give it a try. After a couple short flights the memory muscle klcks in and you can easily navigate the switches. I'll be finishing the entire pit over time. I am however unsure of the MFDs but time will tell.

[TripRodriguez]

I bought 1 panel to give it a try. After a couple short flights the memory muscle klcks in and you can easily navigate the switches. I'll be finishing the entire pit over time. I am however unsure of the MFDs but time will tell.

 

I've done the same and I mostly agree, but after many hours of playing around I decided it really does break my immersion to not be able to look at a switch and see my hand move to it.

 

Also, I'm not very good with dexterity. Aspie.

[TripRodriguez]

Not necesserily. I guess it is not so easy to build a pit which is 100% the same as the virtual one. Think about the perspectives and the fact that the virtual cockpit is not always at the same dimension that the real one. I see lot of trial and errors in the conception.

 

Another solution could be a geodesic dome covered with green fabric all over your cockpit. You can add a camera (2 for 3D perception) on you VR goggles which has an optic with the same field of view than the field of view you have in the simulator. And you can use a software which insert the virtual view wherever you have green color on the camera image. The final image is pushed in your VR goggles.

Just another idea...

 

I've already done some pit building specifically for this. The things you mention can be an issue for certain, but you can load into the aircraft and use a VR hand controller to verify locations on the panels before making holes. Same thing goes for placing the panels. I've spent hours at it already LOL.

 

Also in some cases I was able to get panels extracted directly from DCS.

[Icebeat]

The only thing we need is Leap Motion integration in DCS, it will be fast and easier for everyone, the only problem as always is ED (limited resources). The resources required shouldn't be more than one week one man.

[TripRodriguez]

The only thing we need is Leap Motion integration in DCS, it will be fast and easier for everyone, the only problem as always is ED (limited resources). The resources required shouldn't be more than one week one man.

 

Theoretically the SteamVR plugin for Leap Motion to emulate a standard VR controller should be able to give us what we need. Last I heard DCS wasn't playing well with it but I'll be testing in the next few days and will let you know.

[hollywoodvillain]

I think augmented reality will be the solution to this wish, similar to what matbog said, where it would basically render the sim around the real cockpit that you see through the camera. And i doubt you would need a green screen to accomplish it

[TripRodriguez]

I think augmented reality will be the solution to this wish, similar to what matbog said, where it would basically render the sim around the real cockpit that you see through the camera. And i doubt you would need a green screen to accomplish it

 

That would definitely be the ultimate solution but as niche a crowd as pit builders are I doubt we'll see anything like that on a consumer level any time soon, if ever.

 

I'm looking for a more plausible solution that I can make work now with current tech.

 

Step 1 is Leap Motion testing. I'm worried about the accuracy of the positional component of tracking.

 

Step 2 is back to the Vive Tracker experiments. Captogloves WITH Vive Trackers would be perfect for me if they ever implement that. Except for the cost. =( I really hope the Leap Motion tracking is good enough to do the job.

[hollywoodvillain]

That would definitely be the ultimate solution but as niche a crowd as pit builders are I doubt we'll see anything like that on a consumer level any time soon, if ever.

 

I'm looking for a more plausible solution that I can make work now with current tech.

 

Step 1 is Leap Motion testing. I'm worried about the accuracy of the positional component of tracking.

 

Step 2 is back to the Vive Tracker experiments. Captogloves WITH Vive Trackers would be perfect for me if they ever implement that. Except for the cost. =( I really hope the Leap Motion tracking is good enough to do the job.

 

You think so? Seems like to me its only a matter of software (and a good vr cam). Similar to how you can define virtual walls now, you could define virtual walls around your cockpit where it would show camera passthrough instead of sim. Or maybe im way off there lol idk but i share the same dream!

[agathorn]

Is there any way to get the in game cockpit models out of the game if you wanted to build a 1:1 pit to match?

[frogger]

ive tried it with a leap motion , press a button to switch to camera view for taking notes. however the cameras are IR filtered so not practical. leap motion itself is not good for interacting with small items like a cockpit as it only tracks from 1 directions so if your finger is blocked by another finger it flips out on you and there are to many switches is dcs for that to be ok.

[TWC_Alamo]

Have you been following the PointCTRL thread? Not the ultimate VR solution, but definitely better than anything available today for flipping switches and rotating knobs in VR while not interfering with the HOTAS.

 

 

 

https://forums.eagle.ru/showthread.php?t=218861&page=121

[retai]

If you want to see your cockpit in vr you should try reality blender on steam. I am not sure if it's allowed to post a link but a quick search on Google will help.

[hollywoodvillain]

If you want to see your cockpit in vr you should try reality blender on steam. I am not sure if it's allowed to post a link but a quick search on Google will help.

 

Holy smokes this is exactly what i was envisioning! Thx for this link, really want to try this out now!

[TripRodriguez]

I'll have to check it out too. My main concern though is the performance hit it might cause. This is a solution I've had in mind since the beginning too but I didn't expect it to become available!

 

Many thanks for the info @retai !

 

PS- I wonder if it will support stereoscopic real world image with the dual cameras on the Index? =O

Edited June 14, 2019 by TripRodriguez

[TripRodriguez]

Something just occurred to me though, gauges!

 

I do not have any working instruments, and I'm sure that is the case for a lot of other people as well.

 

And you can put screens behind MFD's, but then the screen is in the way of installing other switches where they should be.

 

Putting green screen inserts in gauges would require an awful lot of accuracy from the tracking to keep them displaying properly. I don't think there's any way I can get my motion sim compensation accurate enough to keep the gauges centered in the gauge holes all the time. D=

 

Re: PointCTRL That looks like an excellent project, but not what I'm looking for. I looking for a full 6DOF tracking solution where you can see your hands in VR.

[hollywoodvillain]

Something just occurred to me though, gauges!

 

I do not have any working instruments, and I'm sure that is the case for a lot of other people as well.

 

And you can put screens behind MFD's, but then the screen is in the way of installing other switches where they should be.

 

Putting green screen inserts in gauges would require an awful lot of accuracy from the tracking to keep them displaying properly. I don't think there's any way I can get my motion sim compensation accurate enough to keep the gauges centered in the gauge holes all the time. D=

 

Re: PointCTRL That looks like an excellent project, but not what I'm looking for. I looking for a full 6DOF tracking solution where you can see your hands in VR.

 

Im confused, why would putting screens behind the mfd's prevent the installation of other switches? You're talking about a 1:1 pit right?

[TripRodriguez]

Well, you can set the screen back far enough to not be a problem, but that looks weird and I don't like it.

 

See my "A" Grade MS Paint art attached!

 

[hollywoodvillain]

Ah i gotcha. Dogfight boss did a great job on their pit, but on close inspection you can see its not exact, probably for that reason. Guess you have to make compromises, but i think i could die happy looking at that pit through a vr

[TripRodriguez]

Ah i gotcha. Dogfight boss did a great job on their pit, but on close inspection you can see its not exact, probably for that reason. Guess you have to make compromises, but i think i could die happy looking at that pit through a vr

 

It is true, if the cockpit doesn't have to match the game exactly you can get around this and the method we are discussing at the moment does not require an exact match.

[jimiC]

Couldn't get blender working . It would shut down dcs in order to open itself

Might require a steam install if dcs to work

 

Sent from my SM-G960F using Tapatalk

[TripRodriguez]

Were you using SteamVR though, with your non-steam DCS? I assume yes, but you know the whole "are you sure it's plugged in" concept could apply! =P

[Bullant]

Interesting thread, I've been thinking through the same things myself. I have always planned on building a full close to 100% accurate Hornet pit, but after buying a rift S a couple of weeks ago, I just can't give up on VR

 

The ultimate solution is something along the lines of blender, where you can mark out/green screen the edge of your cockpit and just have the sim rendering the outside view.

 

However I am pretty certain that as long as the cockpit is close to 1:1 that there will be no problem with reaching out and flipping the correct switch. Don't forget that the switches/knobs all have different tops on them for a reason. I've done some practice with a few practice panels and was able to get the correct switch most of the time. The hard bit is getting the panel in the correct 3d position, as I don't have my pit shell yet.

 

What a great time to be alive!

[jimiC]

Were you using SteamVR though, with your non-steam DCS? I assume yes, but you know the whole "are you sure it's plugged in" concept could apply! =P

Yeah I have vive so steam vr

was on and dcs open. Start blender and it shuts down dcs

Vice versa open blender then start dcs . Blender shuts down

 

Sent from my SM-G960F using Tapatalk