Off the Wall...Mirror 2D version to VR Headset?

[Gunthrek]

Ok, so I was gifted a Pimax 5k VR headset, but my PC just isn't powerful enough to run DCS in VR at any decent quality.  I'm able to run it at max settings in 2D, though, so the obvious question is:  does a software exist that allows you to use your VR headset as a 2D monitor?

The reason I ask is that I'd love to get the immersion and precision head tracking that comes from my headset, and I'm fine if I have to sacrifice the true 3d experience if it means I can get much better visual quality by not forcing my computer to render the scene twice for VR.

I've seen programs out there like Virtual Desktop, but they all appear to run in a VR space, which I'd like to avoid because it's just unnecessary overhead...and it also doesn't list my headset as supported anyway.  I'd love something that just takes my desktop and mirrors it straight to my headset.

Has anyone heard of something like this?

[Dangerzone]

3 hours ago, Gunthrek said:

Ok, so I was gifted a Pimax 5k VR headset, but my PC just isn't powerful enough to run DCS in VR at any decent quality.  I'm able to run it at max settings in 2D, though, so the obvious question is:  does a software exist that allows you to use your VR headset as a 2D monitor?

The reason I ask is that I'd love to get the immersion and precision head tracking that comes from my headset, and I'm fine if I have to sacrifice the true 3d experience if it means I can get much better visual quality by not forcing my computer to render the scene twice for VR.

I've seen programs out there like Virtual Desktop, but they all appear to run in a VR space, which I'd like to avoid because it's just unnecessary overhead...and it also doesn't list my headset as supported anyway.  I'd love something that just takes my desktop and mirrors it straight to my headset.

Has anyone heard of something like this?

I don't have an answer for you but I too am very interested in this too. I'm guessing it would give you more a feeling of being in a 'Simpit' with a curved projector wall around.

I can also imagine it would be great for working on video recordings too (given that in 3D VR your eyes will be left and right of the HUD and the mirror only mirrors left or right so will be offset - whereas with this approach 100% of the GPU is dedicated to the 2D rendering giving better recording plus it would be easier to work the 'camera position'. 

Basically Track-IR on steroids. 

[Gunthrek]

1 minute ago, Dangerzone said:

I don't have an answer for you but I too am very interested in this too. I'm guessing it would give you more a feeling of being in a 'Simpit' with a curved projector wall around.

I can also imagine it would be great for working on video recordings too (given that in 3D VR your eyes will be left and right of the HUD and the mirror only mirrors left or right so will be offset - whereas with this approach 100% of the GPU is dedicated to the 2D rendering giving better recording plus it would be easier to work the 'camera position'. 

Basically Track-IR on steroids. 

My thoughts exactly.  I've been doing DCS videos on my channel, and the thought of even trying to run OBS while doing VR on my not so updated rig makes me shudder.

[draconus]

4 hours ago, Gunthrek said:

Ok, so I was gifted a Pimax 5k VR headset, but my PC just isn't powerful enough to run DCS in VR at any decent quality. 

Can you share your PC specs? Have you tried MT version and Low VR preset already?

https://www.howtogeek.com/270017/how-to-play-any-game-in-vr-with-steamvrs-desktop-theater-mode/

[Gunthrek]

43 minutes ago, draconus said:

Can you share your PC specs? Have you tried MT version and Low VR preset already?

https://www.howtogeek.com/270017/how-to-play-any-game-in-vr-with-steamvrs-desktop-theater-mode/

i7 8700K, 2060 Super, 32 GB RAM.  Low VR preset still gives me less than 30 fps even on MT.  Did the whole OpenXR thing, played with settings, but I'm pretty sure I just don't have the hardware to turn the settings up to a level that makes the visuals acceptable to me.

Edited May 31, 2023 by Gunthrek

[Eugel]

I don´t think this would work. 
I think, displaying the same 2D image for both eyes would look weird and probably confuse the brain.

[draconus]

7 minutes ago, Eugel said:

I think, displaying the same 2D image for both eyes would look weird and probably confuse the brain.

Please, read again what is the OP about. Have you ever been to a theatre? Like that, but interactive with a game on a screen.

@Gunthrek I used to play on GTX970 and still prefered VR over 2D but it's highly subjective of course. If you're not going for 3D VR then a monitor is far better imho than virtual screen.

[Eugel]

30 minutes ago, draconus said:

Please, read again what is the OP about. Have you ever been to a theatre? Like that, but interactive with a game on a screen.

He wants to display a 2D image within the headset without it having to calculate 2 different images for 3D display, no ? 

Stuff like Desktop theatre mode in VR displays a 2D image on a virtual "theatre" screen, but the theater itself is a 3D image, and the tracking would only work within this theatre, not on the 2D picture displayed within, so you would be unable to use VR tracking within the cockpit if you put DCS on that theatre screen.

But, yeah, english is not my first language, so I might not have understood the OPs post.

[desolunatic]

7 hours ago, Gunthrek said:

Ok, so I was gifted a Pimax 5k VR headset, but my PC just isn't powerful enough to run DCS in VR at any decent quality.  I'm able to run it at max settings in 2D, though, so the obvious question is:  does a software exist that allows you to use your VR headset as a 2D monitor?

The reason I ask is that I'd love to get the immersion and precision head tracking that comes from my headset, and I'm fine if I have to sacrifice the true 3d experience if it means I can get much better visual quality by not forcing my computer to render the scene twice for VR.

I've seen programs out there like Virtual Desktop, but they all appear to run in a VR space, which I'd like to avoid because it's just unnecessary overhead...and it also doesn't list my headset as supported anyway.  I'd love something that just takes my desktop and mirrors it straight to my headset.

Has anyone heard of something like this?

You can do that via SteamVR but mind you the performance will be suboptimal still compared to simply playing 2D - you need some resources on the GPU to run the SteamVR in reasonably high resolution to warrant readability of the game mirror and the game itself on top of it - if you get Virtual Desktop on steam you can also try doing a "fake VR" - run the game in 2D, tie the game mirror to follow your head movements and then you can run OpenTrack in the background with SteamVR as source of the head movement with 1:1 curves. Regular 2D curves need to be applied for regular "VR theatre" viewing. @Eugel

Edited May 31, 2023 by desolunatic

[Gunthrek]

@Eugel It would have to be adjusted for the lenses and aligned so that the screen is split up in a way that makes it look like one big monitor, but I imagine that's just a problem with configuration rather than the headset.  Not sure, though.

@draconus No, I'm definitely not interested in a theater mode.  I'm looking to completely cut out the VR environment and just have the image displayed exactly as it would on my monitor.  You look around and the screen stays the same and the camera moves where you move.

@Eugel Yes, you had my original intent correct.  I want to basically just turn my VR headset into a straight monitor.  I don't want any kind of 3D environment hogging up resources, just some kind of program that mirrors the image to my headset and corrects for the lenses and splits the image in a way that makes it look like a single monitor, sort of the way you would adjust a projector to show a flat screen on a curved wall or something.

@desolunatic I'd rather just cut out the VR environment completely.  I looked at Virtual Desktop, but they don't list my headset as being compatible.  I do use Opentrack right now for head tracking, but I have this expensive headset sitting right there not being used that has a super precise accelerometer in it that works great except the for the fact that it makes you blind while using it.  I tried putting it on backwards, and it works perfectly, but the weight of it being on my head that way makes my neck hurt.  If I could just find a way to get my screen mirrored to the headset, I could get precise headtracking and pretty decent graphics quality.

To all, I'm not sure a solution exists, though.  I've done a bunch of searching and have only come up with solutions that use a VR environment to either make a desktop window in virtual space or, like Virtual Desktop, provides what I'm looking for but still uses up resources for their virtual environment and do not list my headset as supported in their software.

I'm just kind of hoping that maybe someone out there knows of some unknown developer who came up with something that bypasses all that and just treats the headset like a weird monitor.  I'd still need OpenTrack to make the head tracking work; that's fine.  I just need to be able to see what I'm looking at through the headset.

Edited May 31, 2023 by Gunthrek

[cfrag]

9 minutes ago, Gunthrek said:

I'd rather just cut out the VR environment completely.  I looked at Virtual Desktop, but they don't list my headset as being compatible

I did this with my HMD very early, when the SDK for VR glasses wheren't that great. It's essentially as if you would glue your monitor 10 inches in front of your eyes. No tracking, no lateral movement - just that image in front of your eyes. Now recall that you only can really focus on the sweet spot. Moving your eyes left and right can only get you +/- 20 degrees, and the image won't be sharp. It's unpleasant, you only have a  circle in center of the screen making about 30% of the entire screen estate to work with. And since you can't see sharp or focus on the outer parts of the screen, including edges, you can't even click or read any menus. A VR HMD without tracking is useless. You can't use it for anything. 

[Gunthrek]

1 hour ago, cfrag said:

I did this with my HMD very early, when the SDK for VR glasses wheren't that great. It's essentially as if you would glue your monitor 10 inches in front of your eyes. No tracking, no lateral movement - just that image in front of your eyes. Now recall that you only can really focus on the sweet spot. Moving your eyes left and right can only get you +/- 20 degrees, and the image won't be sharp. It's unpleasant, you only have a  circle in center of the screen making about 30% of the entire screen estate to work with. And since you can't see sharp or focus on the outer parts of the screen, including edges, you can't even click or read any menus. A VR HMD without tracking is useless. You can't use it for anything. 

Agreed.  That's why it would be used in conjunction with OpenTrack to get the head tracking.  You can use the SteamVR option in that software to use the head tracking in the headset to move your view around.  I just need something that will put the image on a flat plane in the middle of my view, removing the strain of having to render the scene twice for parallax, then I can use the OpenTrack head tracking to focus on whatever it is I'm trying to look at.  Peripheral vision is only for noticing movement anyway, so having a reduced clarity outside the sweet spot is no worse than in VR except that hopefully the spot I'm looking at is getting close to the detail I see on my monitor rather than just having to put everything on minimum just to get close to 30 FPS.

Edited May 31, 2023 by Gunthrek

[draconus]

10 hours ago, Gunthrek said:

If I could just find a way to get my screen mirrored to the headset, I could get precise headtracking and pretty decent graphics quality.

22 hours ago, Gunthrek said:

Did the whole OpenXR thing, played with settings, but I'm pretty sure I just don't have the hardware to turn the settings up to a level that makes the visuals acceptable to me.

Aren't you contradicting yourself? If you can't get good quality in VR you won't get it in 2D mode on the same HMD.

[Dangerzone]

16 minutes ago, draconus said:

Aren't you contradicting yourself? If you can't get good quality in VR you won't get it in 2D mode on the same HMD.

Not seeing the contradiction.

VR requires rendering twice for the same frame. What I see he's saying is that if he can go to 2D where the same render is used in both eyes and you've got to be cutting down a significant portion of the processing, thus allowing better graphics on older hardware.  The logic appears sound to me.

[draconus]

34 minutes ago, Dangerzone said:

The logic appears sound to me.

I understand the logic, I just doubt the better settings will satisfy him while still keeping the performance, mind that reaching current monitor resolution will be far from enough.

[Gunthrek]

11 hours ago, draconus said:

I understand the logic, I just doubt the better settings will satisfy him while still keeping the performance, mind that reaching current monitor resolution will be far from enough.

We can speculate all we want, but until I'm able to test it, that's all it is.  Without a program that does what I'm suggesting, there's no way to find out for sure.

[cfrag]

On 5/31/2023 at 11:17 PM, Gunthrek said:

Agreed.  That's why it would be used in conjunction with OpenTrack to get the head tracking.

Apologies, how on earth would that work? As I understood, you want to blit the entire contents of the screen buffer onto both left and right screens, nothing in-between. That's it. No modification, no scaling, no transformation, nothing. Just 1:1 blit. That can be done, with good performance. In the HMD, you always look at the pixmap's center, and maybe swivel your eyes to be able to see to the map's edges. How does OpenTrack figure in there without any 3D and parallax transformation? If you want to use that, you need a reference point, 3D transformation of the projection screen, and you must create separate views for left and right eye. In that case you indeed might as well use SteamVR's theater mode, or Virtual Desktop which do exactly that (and hey, the latter even has decent support for pr0n, so not all is lost).

What am I missing? Do you perhaps want to use the control values that you get from HMD movement and feed that as control input into DCS to move the view output to control what is displayed on-screen as if it was an expensive TrackIR? That might work - if you can convince DCS to accept that input, but I would not hold my breath. And it would only work with turning information, not spatial movement.

Edited June 2, 2023 by cfrag

[Gunthrek]

9 hours ago, cfrag said:

Apologies, how on earth would that work? As I understood, you want to blit the entire contents of the screen buffer onto both left and right screens, nothing in-between. That's it. No modification, no scaling, no transformation, nothing. Just 1:1 blit. That can be done, with good performance. In the HMD, you always look at the pixmap's center, and maybe swivel your eyes to be able to see to the map's edges. How does OpenTrack figure in there without any 3D and parallax transformation? If you want to use that, you need a reference point, 3D transformation of the projection screen, and you must create separate views for left and right eye. In that case you indeed might as well use SteamVR's theater mode, or Virtual Desktop which do exactly that (and hey, the latter even has decent support for pr0n, so not all is lost).

What am I missing? Do you perhaps want to use the control values that you get from HMD movement and feed that as control input into DCS to move the view output to control what is displayed on-screen as if it was an expensive TrackIR? That might work - if you can convince DCS to accept that input, but I would not hold my breath. And it would only work with turning information, not spatial movement.

 

The OpenTrack software would be solely used for the headtracking part, making it like I just have a 2D monitor strapped to my face.  As far as I'm aware, playing DCS in non-VR mode requires an external program to do headtracking, and OpenTrack has an option to use the SteamVR protocol to take the accelerometer data from the headset and convert it into a TrackIR type signal, which can then be fed into DCS for headtracking.  I've used this successfully to make my headset move the camera around in game on my monitor, but it's very awkward and uncomfortable if you can't wear the headset properly, and you can't because it makes you blind if the image isn't being mirrored to the headset.

Now, if you're suggesting there's a way to do VR mode in a way that removes the VR component and just renders everything flat like it does on my monitor, then I'd be very interested in learning how to do that.  As I said before, I can run the game at max settings with no problems on my ultrawide monitor (granted it's only 1080p), so I have to imagine I could keep my settings turned up pretty high if it's not having to render the frames twice.