A possibility to synchronize VR with a Sim-Pit

[Meathawk]

It would be great for VR to have a way of synchronizing the location of the Sim-Pit with the VR-Cockpit, so buttons would be in the correct position if you reach for them in VR.

[cfrag]

I'm not sure how that can work. Most VR implementations simply track the HMD's position relative to when it was last reset/turned on; when you press re-center, DCS simply repositions the game's camera to coincide with your HMD's current position and forward direction, and then stores that as the new reference location. All subsequent trackings are relative to that location.

Some VR systems do have a notion of where the play area's center is, and where the HMD is located in space (e.g. Valve/Vive Lighthouse). These systems are much better at tracking (they have higher tracking precision). The problem: synching to that 'world' center location would make little sense: if you don't sit exactly there and just shift your own position slightly, or if there is a small tracking error, the entire world is off: the game camera does not reflect the HMD's location. And again, once you press 're-center head', all spatial information is forgotten and DCS resets so that your current HMD location/orientation is the new reference. The only way i can imagine this works is if you have some kind of system in your sim pit that you can fix your HMD exactly to (position and forward vector) - and by exact I man 1/10th of an inch correct - to re-set head to. Doing that in-game will probably be highly annoying if not uncomfortable.

Edited December 22, 2021 by cfrag

[Dangerzone]

I can't see it being easy to actually have the buttons synced with DCS as it is now. I think that would take even more technology and cost - and the simpit would need to be perfect, along with reference markers that would communicate with the headset so it auto-aligns itself. 

The better way I can see this happening is to have an augmented reality headset. Something where you have the simpit, and then a green backdrop for everythough outside of the aircraft, and where the VR display has camera's on the outside that 'passes through' video footage of the simpit, and wherever it is green it would render in external view.

In order for this to work though I imagine you would need:

1) A headset that supports such features

2) DCS to have an option where it allows for rendering of external graphics only, and

3) Software that would take DCS's external graphics and 'paint' it onto the green screen parts.

There may be other ways as well - but from what I can tell - at the moment we're a fair distance away from having something available on consumer market. Wouldn't surprise me though if such setups were already available commercially.

I dream of such things, but in reality - just VR alone at the moment is darn incredible. yes - it's a pitty we have to choose between either simpits or VR - but I'm amazed at what we already have. My next wish is for VR graphics to be as good as 2D graphics, and for us to be able to record high quality content in VR easily. 

[Meathawk]

Thanks for the comments.
Yes - the Simpit would have to be "relatively perfect" in terms of the placement and size of the controls - that is a major challenge but IMHO it is doable if you take the DCS 3D Data as a reference - as a template. I guess the professional/military VR simulators are doing something like that already.
For the alignment there are basically two options:

1. place the VR headset

2. the inside-out cameras of the headset have some kind of visual marker to sync

2. b) the inside-out cameras of the headset use object tracking (analog to some AR applications) to sync

The AR or indirect AR (VR with camera) approach is also interesting, will IMHO though not be as immersive. BUT I totally agree - would eliminate the "perfect" requirement and thus be much more flexible.

Interesting discussion - tnx guys!

Edited December 23, 2021 by Meathawk

[A.F.]

The problem is to have a simpit for each type of aircraft.
For example, I fly F-14, F-16, Bf-109 and others.
Leap Motion is a great thing!
I am currently building a frontpanel for the Viper. It is made of very light material. 
I have glued important and often used control surfaces (e.g. ICP) with soft foam, so that I feel a feedback when pressing.
The virtual key does not necessarily have to be 100% in the correct position.
But I have a relatively good reference where I have to tap.

Unfortunately, leap motion is not yet finished.
Let's see where journey goes.

[winghunter]

Viperwing and Varjo can do this with a greenscreen

 

Edited December 23, 2021 by winghunter

[cfrag]

Great find!!! That should be possible with any HMD that has high-precision tracking and two front-facing cameras (e.g. Vive 2). Doesn't neccessarily look like a simple greenscreen-effect to me, but a depth-sorted spherical re-projection of the simulated world: any pixel that is outside the cockpit is replaced with the sim's output, anything closer with what the HMD cameras see. That way the pilot can even use a handy with a green image inside. Very nice indeed.  Synching the cockpit model for correct HMD depth clipping may still be a b***, though - even if it requires only a rough approximation (if the camera can measure distance of the eye's focus point, that might be easier since the software can much easier determine if the focus is inside or outside the cockpit -- those Varjo HMD are really something).

Only drawback I see: just one cockpit per 20k+ USD. Ok, time to play the lottery again  

 

[Tank50us]

Has anyone here considered the possibility of augmented reality for their sim-pits?

Obviously militaries do this already with systems like IHADS, JHMCS, and other HMDs, but if one could set it up where the system only tracks where the head is looking in relation to the cockpit, and your own HMD is feeding you the required data, I'd see that as a more viable option for a proper sim pit.

[winghunter]

The Quest 2 also has some AR apps. You can define a 3D area for camera passthrough.

If AR support was implemented in DCS, one could define the cockpit geometry in 3D similar to how room setup works in the Quest 2.

DCS would then render everything beyond the simpit.

As an advanced feature, one could also setup displays within the cockpit that would be rendered by DCS. Because one thing that doesn't make sense is wiring up your own displays for a simpit. Considering we are approaching retina resolution for VR headsets next year.

 

 

Edited December 25, 2021 by winghunter

[bjornc]

Should be relatively easy to make. 

Use the Leap Motion addon and add a x and z axis (got only y axis now). That way you can align the digital hands on your physical pit using left and right (y), up and down (x) and forward and backward (z).

Another addition should be to turn clicks off (now we only get indexfinger and thumb, but no "off" setting). This makes your digital hands a reference to your actual physical simpit.  

Should be easy fix. Unfortunately DCS doesn't do a lot to help vr users. 

Edited September 27, 2023 by bjornc