DCS VR + AR interim solution

[Beau1969]

Hi all,

I recently set up a 2D hardware cockpit solution, which prompted me to develop this file (Keep DCS (Digital Combat Simulator) Window Active) as I was getting a few DCS "out of focus"-like issues (due to other software for hardware, such as Buttkicker, Winwing, AimXYZ.) After getting the 2D solution working well, I integrated my Meta Quest 2 Pro with the new setup.

I needed to determine how to allow the MFDs to be rendered when in VR. Fortunately, after reading through a few DCS forum streams, I found that adding this line to the monitor setup LUA file used to configure your displays is the solution.

VR_allow_MFD_out_of_HMD = true

I also came across this file, which is very helpful and straightforward (assuming you are comfortable with Windows CMD). It helps to align monitors precisely instead of "eyeballing" them in the Windows Display settings. This is great for defining exactly where the Windows monitor positions are for configuring the DCS monitor LUA file.

Link: GitHub - programmer2514/DPEdit: A simple Windows command line utility to accurately set the relative position of displays in a dual- or multi-monitor setup.

Thus far, I have not found an optimal AR solution (such as a lower 1/3 passthrough, 2/3 DCS view in the headset configurable/initiated by a single key press), so I am simply looking out the bottom of the VR headset, which is easy enough to see the winwing MIP cockpit (with some learned action memory on switches on the throttle and PTO2 panel.) There is a promising "around the hands" solution from a Japanese programmer, but only for SteamVR, which I avoid due to better performance in OpenXR.

I will post here if I find solutions (outside Virtual Desktop and OpenKneeboard) that seem optimal/helpful. Otherwise, I hope the above is helpful to anyone looking for similar solutions. I also appreciate it if anyone has found a solution that might help accomplish the goal mentioned in the paragraph directly above. I aim to minimize additional software interfaces between DCS and the sim/VR hardware (such as Virtual Desktop or SteamVR.)

Ventor (csbeau)

Edited January 26, 2024 by Beau1969

[firefox121]

20 hours ago, Beau1969 said:

Hi all,

I recently set up a 2D hardware cockpit solution, which prompted me to develop this file (Keep DCS (Digital Combat Simulator) Window Active) as I was getting a few DCS "out of focus"-like issues (due to other software for hardware, such as Buttkicker, Winwing, AimXYZ.) After getting the 2D solution working well, I integrated my Meta Quest 2 Pro with the new setup.

I needed to determine how to allow the MFDs to be rendered when in VR. Fortunately, after reading through a few DCS forum streams, I found that adding this line to the monitor setup LUA file used to configure your displays is the solution.

VR_allow_MFD_out_of_HMD = true

I also came across this file, which is very helpful and straightforward (assuming you are comfortable with Windows CMD). It helps to align monitors precisely instead of "eyeballing" them in the Windows Display settings. This is great for defining exactly where the Windows monitor positions are for configuring the DCS monitor LUA file.

Link: GitHub - programmer2514/DPEdit: A simple Windows command line utility to accurately set the relative position of displays in a dual- or multi-monitor setup.

Thus far, I have not found an optimal AR solution (such as a lower 1/3 passthrough, 2/3 DCS view in the headset configurable/initiated by a single key press), so I am simply looking out the bottom of the VR headset, which is easy enough to see the winwing MIP cockpit (with some learned action memory on switches on the throttle and PTO2 panel.) There is a promising "around the hands" solution from a Japanese programmer, but only for SteamVR, which I avoid due to better performance in OpenXR.

I will post here if I find solutions (outside Virtual Desktop and OpenKneeboard) that seem optimal/helpful. Otherwise, I hope the above is helpful to anyone looking for similar solutions. I also appreciate it if anyone has found a solution that might help accomplish the goal mentioned in the paragraph directly above. I aim to minimize additional software interfaces between DCS and the sim/VR hardware (such as Virtual Desktop or SteamVR.)

Ventor (csbeau)

 

I found recentñy an app that seems could solve part of your issues, It works trough steam an allows you to create mixed reality windows to see trough them, the name is "Reality Mixer - Mixed Reality for VR headsets" I bought it at low price at steam. And works fine, sadly I don't use steam anymore for flying DCS , but now I am Thinking to start again because recently bought two trustmaster's MFDs. I will try this afternoon

Here are a two examples 

 

Sadly only works with steam Vr, the good news is that the developer is diggin to make openXR compatible, maybe next release

Edited January 27, 2024 by firefox121

[Phantom711]

But as far as I know, you need Virtual Desktop additionally to make it work.

I could not get both (VD and steamVR) to work with DCS. 
I had brought up that matter a few weeks ago already. If you get it to work, let us know.

 

[firefox121]

2 hours ago, Phantom711 said:

But as far as I know, you need Virtual Desktop additionally to make it work.

I could not get both (VD and steamVR) to work with DCS. 
I had brought up that matter a few weeks ago already. If you get it to work, let us know.

 

Yep, triyed this evening and steam must stay backwards and also mst run through openVR not openXR

[Phantom711]

So can you kind of talk us through the process?

how do you set up VD? Do you tell it to execute SteamVR?

Why does it have to be OpenVR? That also limits you to the ST version of DCS…

 

[cordite]

Future possibility:
https://www.infusevr.com

 

edit: Still uses SteamVR, no mention of OpenXR

Edited January 28, 2024 by cordite

[nikoel]

Here is something I would consider as better - hand tracked passthrough, Works for Quest 3 and Pico I believe

https://eurekaworks.booth.pm/items/5174163

[Beau1969]

Thank you, all. The recent post by nikoel is the promising program I referenced above for SteamVR by a Japanese developer. However, I am trying to keep the solution to OpenXR… I may reconsider…

[Beau1969]

Up late tonight, as I was (and still somewhat) frustrated by the menu challenges when exporting displays in VR. However, after hours of investigation, the best solution appears to put the MFDs/other being exported to the left of your primary display. The displays somehow claim real estate in the screen, which can overlay with important menu selections (such as accepting option changes, starting missions, etc.)

Separately, I am flying and testing in the F-18, and I am not able to see the kneeboard either as it would be in 2D, nor on the pilots leg. It is simply transparent or blank. Is this normal, or another byproduct of trying to export the displays?

 Thank you!

[Phantom711]

vor 56 Minuten schrieb nikoel:

Here is something I would consider as better - hand tracked passthrough, Works for Quest 3 and Pico I believe

https://eurekaworks.booth.pm/items/5174163

I thought, that was actually the same developer like the Reality Mixer. But I could be wrong though.

 

vor 27 Minuten schrieb Beau1969:

the promising program I referenced above for SteamVR by a Japanese developer. However, I am trying to keep the solution to OpenXR

But steamVR can be selected as an OpenXR runtime. That‘s why I also had the thought, that it could be combined with DCS (in OpenXR).

So I thought I would simply use VD, start up SteamVR with the respective tab in VD and have SteamVR selected as the OpenXR runtime. But even just getting into the main menu of DCS was in the single digit FPS. So I didn‘t try any further.

[cordite]

On 1/28/2024 at 5:33 AM, Phantom711 said:

I thought, that was actually the same developer like the Reality Mixer. But I could be wrong though.

 

But steamVR can be selected as an OpenXR runtime. That‘s why I also had the thought, that it could be combined with DCS (in OpenXR).

So I thought I would simply use VD, start up SteamVR with the respective tab in VD and have SteamVR selected as the OpenXR runtime. But even just getting into the main menu of DCS was in the single digit FPS. So I didn‘t try any further.

Definitely not the same dev.

 

Then it appears the best option is waiting for the “Reality Mixer” dev to implement OpenXR support.

 

edit: another option

https://flightsim.to/file/64739/color-panel-for-vr-passthrough

 

 

Edited January 29, 2024 by cordite

[whitav8]

Color-panel as mentioned just above works very well (BUT JUST FOR MSFS) - just set the passthrough color in Virtual Desktop (Streaming at the bottom center ) to purple with a small amount of threshhold in VD.  Then when you are flying in VR mode, the Toolbar will then have a tab just to the left of center to enable it. Then you just move the panel to where you want it and size it. When you click on it to minimize it so you can see on the other side of it, it leaves a small outline and a small circle in the center of the passthrough panel to reenable it. I found that if I selected the panel by holding down the left mouse button, I could "bring" the panel toward me which helped my eye distance focus.

I asked in the review comments if the addon author could make it work for DCS but no answer yet

[Beau1969]

Journey continued via this forum post,

I am able to export to a remote PC and now looking to see how best to integrate with VR and whether it helps with performance.

[TED]

I feel the optimal solution to this would be a at the modeler/developer level, whereby they can produce an option to have a mask layer over certain components in the modeled cockpit. Or indeed simply change the existing color completely of the various cockpit panels/controls to one single unique color. For example a magenta mask over the ufc, the edges of the MFD’s and any other panels we wish to have a physical view of. If they were to do this we could then simply use the VD pass through to exclude those masks, thus giving us the view of the physical panels. Unfortunately we do not have access to the various module design models, but if sufficient demand and pressure was put on the developers, I’m pretty sure this would not be a particularly difficult or time consuming task, requiring them to render some extra options for MR ability. I’d expect in theory a community 3rd party free mod aircraft would be a place to start and would require someone with a bit of blendr experience to create the masks. These could then be presented as “options”. Surely we can’t be that far off actually achieving this. Perhaps one of the better community mods would be prepared to give it a go? 

Edited January 31, 2024 by TED

[Dogmanbird]

the app i saw being developed about 18 months ago (qpro) ran within the headset and the masking was drawn, manipulated and processed inside the headset. It was getting applied to the game being played over link cable. unfortunately i haven't heard anything of it since. Has anyone come across any news lately that is similar to this?

[Beau1969]

Hi all,

Now that I have gotten my setup to a place I am comfortable with (for both QPro and Q3), I plan to see what I can do about cockpit passthrough. Interestingly enough, almost exactly what we want is available in the headset, which outlines items in your immediate vicinity. Perhaps we can figure out how to have it only show the area within a specific area. The outline of the TOPGUN MIP is perfect, unfortunately, it also picks up the other items; so will explore a few options.

Anyone with suggestions on where to start is welcome and appreciated.

Thank you,

csbeau

[TED]

I actually just got something working last night.

Using openkneeboard with a pink page. Then adjusting the size of the box to cover the ufc. I can then bind a control to switch it on and off if I need to input into the ufc. Stages were -

1. Make a pink png file. I used 255 red, 0 green, 255 blue.

2. Enable passthough in vd. Unfortunately this is only possible in high settings (not ultra or godlike)

3. Configure passthrough with 5% similarity.

4. Open dcs and use the openkneeboars settings to move it into the right position and set size accordingly.

This is not an ideal solution but it does work. Of course graphics settings are lower. Theres a lottle reflection from the desktop, but setting similarity as low as possible reduces this. Alignment can be a bit tricky and with the pico 4 there is a bit of parallax error so it's not really possible to align 100%. If u can love with the downsides though its useful to be able to see the ufc when punching in coordinates and other ufc info.

[Beau1969]

I found an interesting solution using the latest updates to the Quest platform. You can set up the space and identify "furniture" to create a box around your cockpit hardware. I added a "box" around the center column of my cockpit (AMPCD, and UFC), so that when I login to DCS and start-up the F-18, I can align the VR central column with the hardware central column. It works very well. You start to ignore the blue box around it, which shows up when you look down or reach toward it. It is not perfect, but very helpful. My hope is that Quest will allow the furniture to show in passthrough at some point in the future.

[Ditch]

Solution that worked for me: