Mixed reality in Industry

[Merlins51]

I attended I/ITSEC this week, the annual international industry trade-show for simulation and training, primarily aimed at military training.

 

 

A number of the booths were showing off mixed reality flight setups, which I'd not seen before. In this context, mixed reality means that in the same VR headset, I can see the external visuals, the HUD and some of the controls in VR but there is a mask in the aircraft model through which I see the output of the passthrough camera instead.

 

 

What this means is that as I was flying, I could see my own hands (not representations of my hands), my legs, the real stick, real throttle etc. but everything else was VR.

 

 

It essentially enables someone to use real MFDs, ICPs, switches, buttons etc. in a VR visual environment. And write notes if you so wished.

 

 

Here's a video of an example from Flight Safety International: https://www.facebook.com/watch/?v=959882581036137

 

 

 

I think they were using the Varjo XR-1 headset. No idea if/when this will make it to entertainment simulators (and more affordable headsets) but it can only be a matter of time. Thought someone may find that interesting.

Edited December 7, 2019 by Merlins51

[Supmua]

Those headsets look like the Vive Pro to me. Anyway, this seems like an ideal approach to fly in VR.

[Montes]

apparently it creates a window where the monitor is, will it be so difficult to program something like that?

[Montes]

A nadie parece interesarle mucho pero sería fantástico.

[Yuriks]

It seems that all it takes is to map area that would take stream from passthrough cameras to show physical cockpit, while the rest is VR.

 

Sounds easy, but I have not seen soft that would do that. May be VR owners know better how difficult is mixed reality?

[Harlikwin]

Yeah, can't really comment on mixed reality like that. I get what they are doing though, only showing the outside world while if you are looking in the physical cockpit you see the real pit.

 

Current VR doesn't let you do that, the biggest issue is syncing a phyiscal pit with the VR one in terms of control locations. I only use a physical HOTAS and can get that about right and it works, then interact with the virtual switches using point control.

[Echo179]

I think this solution would be the best of all worlds for those that like the tactile feel of having all the switches and controls physically in the pit, but also want the immersive view outside the cockpit. You could still build your 1:1 cockpit, and even put screens in your MFD's, and then that would be visible in your view, while everything outside the glass is virtual. Maybe have it so you have a mix of real and virtual instruments. Have the MFD buttons visible, but have a virtual screen overlaid. Save on time and money of having all the guages and instruments by having them represented via VR, but still have all the controls visible.

 

It would make using an actual kneeboard much easier too.

[Icebeat]

You can do it right now with the index's cameras and enabling room view with transparent passthrough and lowing the room illumination. There is a software called reality blender that supports a green mask aswell.