muelch

Link to MS ticket if anyone wants to upvote it: https://aka.ms/AAh4zmi

Okay, I think what's going on here is VRAM pressure is causing some critical VR-related resource(s) to get moved to shared GPU memory (i.e. system memory), which then tanks framerates since system memory latency is so much higher than VRAM latency. You can see when this issue occurs after returning to the main menu, the GPU bus usage is incredibly high (top right, circled in red as it drops due to alt-tabbing): I circled in red some relevant metrics when alt-tabbing to fix the issue while being in the main menu when this issue is occurring. Before alt-tabbing, GPU bus usage is roughly 40% (40% of what I'm not sure). After alt-tabbing, it goes down to 0%. Which makes sense, there's nothing on the main menu that should need to constantly transfer data between the CPU and GPU, better yet at the insane rate of 40% of the bus. However, the reason this bug is happening is clearly because something is stuck in shared GPU memory that shouldn't be, even after returning to the main menu when we have plenty of dedicated GPU memory available, and it's probably needing to be transferred between the CPU and GPU every frame, perhaps frame buffers used for VR or something. You can also see circled in red an increase in dedicated GPU memory usage when I alt-tab, and a decrease in shared GPU memory usage, as the issue fixes itself. Something is getting moved from shared to dedicated GPU memory when alt-tabbing, which is why the issue fixes itself. I'm guessing alt-tabbing causes windows to tell the process to trim GPU resources, since it's no longer in the foreground and therefore shouldn't have as high of priority. I haven't been able to track down what exactly the resource is. I'm going to give ED the benefit of the doubt and assume it's something related to WMR/openxr, especially since I was able to reproduce this issue in MSFS. And I think it makes sense that it would be a large VR-related resource. I think part of the issue is that there are two competing things going on here, DCS and the WMR/VR stuff. DCS runs in DX11, but even within DCS.exe, winxrruntime.dll is being used for VR, and it uses DX12. In DX11, I believe it would semi-intelligently manage these resources, and ensure that whatever this high priority thing is that's ending up in shared GPU memory would instead be in dedicated, or at least get moved there when returning to the main menu. In DX12, resources are managed manually, and it's the responsibility of applications to stay within their given budget to avoid stepping on other simultaneously running applications. I'm not sure how this works with both DX11 DCS stuff and DX12 VR stuff running in the same process. I'm also not sure if this is something that needs to be fixed by ED, Microsoft, or even possibly nvidia. E.g. maybe DCS needs to obey a stricter vram limit somewhere when in VR, or maybe Microsoft needs to pin these VR-related sources so they stay in dedicated gpu memory, or maybe this is the responsibility of the GPU driver. Either way, I think the one thing that isn't a solution here is buying a video card with more vram. An RTX 3080 is more than capable of running this in VR, this is just an unfortunate software problem getting in the way. The issue should be relatively easy to fix, but unfortunately requires someone to take ownership of it between ED/Microsoft/Nvidia.

Yeah I was finally able to repro without reprojection. At least that rules one thing out. I still have no luck reproducing with HAGS. Probably just masking whatever the issue really is on my machine.

In a lot of ways, but if you're seeing these same symptoms on Oculus then it sort of rules it out being an issue with WMR's motion reprojection.

You're seeing this issue with motion reprojection disabled? That's a bit different, since I was only able to repro with motion reprojection enabled. It also seems that enabling Hardware Accelerated GPU Scheduling makes the issue go away. However, I saw a noticeable GPU frame time hit when enabling that, but only if motion reprojection wasn't enabled. So if you aren't running with motion reprojection, you might try enabling Hardware Accelerated GPU Scheduling if it's not already and seeing if that makes the issue go away.

Stuck with just SLAM/SLAMER (and guns). I eventually tried attacking from the non-coastal side and for some reason a SLAMER could make it through on that side without getting shot down. Who knows, thanks anyways.

Any tips on how you're supposed to destroy the ammo depot, after destroying the submarine? I tried to use SLAM/SLAMER, but they reliably get shot down. Trying to dive in with guns, it seems pretty difficult to avoid the plethora of SAMs and AA. At least, I've been shot down over a dozen times before making it, which is super fun after the enthralling 20 minute flight there.

A thread for MSFS with the same issue (there are two issues in the thread, but one topic specifically mentions this same issue over there): https://forums.flightsimulator.com/t/vr-reverb-g2-wmr-performance-motion-reprojection-cpu-issues-overclocking/497696

I was able to confirm the same issue in MSFS. Triggering low fps in-game then carried back over to the main menu. Alt-tabbing fixed it. Disabling motion reprojection also caused the issue to go away. So seems like a really unfortunate bug with WMR's motion reprojection, possibly related to the RTX 3080, Reverb G2, and/or DX12.

That could really only help if you're low on physical RAM (and even then I'm skeptical), which should be easy to spot (and probably best solved by fixing whatever is using so much). I don't think anything that fixes could be consistent with the symptoms here (VR-specific, sometimes fixed by alt-tabbing, returning to main menu still seeing low FPS).

It seems like this has something to do with Motion Reprojection. If I disable motion reprojection in OpenXR tools for WMR, the issue seems to go away (albeit, my fps is higher, but it's much less smooth without motion reprojection kicking in). It seems like when I was testing through SteamVR earlier, it was still using WMR's motion reprojection, hence why it had the same issue. As a workaround, I am using SteamVR as the OpenXR runtime, that way I can use SteamVR's motion smoothing as they call it. This seems to fix any prolonged FPS drops, with the downside that motion smoothing only works at 45 fps and not 30 fps, as well as the general terribleness that is SteamVR with DCS (i.e. seizure inducing loading screens). I also need to set "Force DX11 Mode" in WMR for SteamVR settings (along side where the motion projection setting is). My understanding is this has nothing to do with DCS (as DCS is already DX11), but affects how SteamVR is doing the motion reprojection (i.e. it will use DX11 instead of DX12). It may affect more than just the motion projection as well, things did seem a tad more blurry. I'm assuming there's some kind of issue between DCS and the motion reprojection under DX12 (as the issue seems to occur with both SteamVR's motion reprojection under DX12, and WMR's motion reprojection, which is pretty curious). It seems like something is leaking, causing the motion reprojection on the GPU to have to do a lot more work, which is why framerate remains low in the main menu when this issue occurs. It could also be totally unrelated to motion reprojection, and it just so happens that switching to SteamVR's motion reprojection, and forcing it to DX11, freed up enough vram to not trip the same issue. The capture I posted earlier where DCS was seemingly able to run at 30fps but chose not to may still be a problem, but I don't think it's the main issue here.

There have been many reports of unexplained prolonged FPS drops in VR. I have provided what I hope is a consistent repro along with a GPU/perf capture later on below, but common symptoms of this issue seem to include: Happens randomly, but common triggers seem to be actions that cause temporary performance blips (that then turn into sustained FPS drops) FPS goes from something stable and normal (e.g. 30/45fps) to significantly lower and unplayable (e.g. 15-25fps) FPS drop lasts for many minutes, sometimes requiring exiting the game to recover VR-specific Alt-tabbing away from and back to DCS sometimes resolves the issue Returning to the main menu still exhibits abnormally low FPS I think those last three points are important, as they point to a software bug, not a hardware bottleneck. FPS would return to normal at the main menu, and alt-tabbing wouldn't fix the issue, if this were simply a problem with users' machines being incapable of sustaining adequate framerates. Furthermore, the issue seems to be VR-specific, pointing to a VR-specific bug as opposed to a hardware capacity issue. Here's a collection of forum posts that I believe exhibit this issue (there are more): The most consistent repro that I can find is to: Load into main menu Observe 90fps Load into Cold Start instant action on Marianas map in F/A-18C sim (this is a good starting point, since you're on a carrier, and can see the island in the distance) Observe stable 45fps Slowly turn your head to the left, until you're looking behind your left shoulder Slowly turn your head to face back forward, then right, until you're looking behind your right shoulder this time Turn your head back to face forward Observe ~14 fps Quit to main menu Observe ~25 fps This is on current stable Steam DCS (also on beta), Windows 10, a stock RTX 3080 (driver 512.95, doesn't seem to matter), stock Ryzen 5950x, HP Reverb G2, WMR 2000.21051.1282.0 (version doesn't seem to matter), OpenXR 110.2204.7007 (also reproduces on SteamVR), 100% render scale (3168x3096 pixels per eye), motion reprojection always on, and the following in-game settings: 2022-06-02 08:30:19.529 INFO APP: options.graphics = { ['messagesFontScale'] = 1; ['forestDetailsFactor'] = 0.75; ['rainDroplets'] = true; ['LensEffects'] = 0; ['box_mouse_cursor'] = true; ['anisotropy'] = 4; ['water'] = 1; ['motionBlur'] = 0; ['outputGamma'] = 2; ['aspect'] = 1.6666666666667; ['lights'] = 2; ['shadows'] = 4; ['MSAA'] = 1; ['SSAA'] = 0; ['civTraffic'] = ''; ['clutterMaxDistance'] = 0; ['cockpitGI'] = 0; ['terrainTextures'] = 'max'; ['multiMonitorSetup'] = '1camera'; ['shadowTree'] = false; ['chimneySmokeDensity'] = 0; ['fullScreen'] = false; ['forestDistanceFactor'] = 0.75; ['useDeferredShading'] = 1; ['DOF'] = 0; ['clouds'] = 1; ['sceneryDetailsFactor'] = 1; ['width'] = 1280; ['flatTerrainShadows'] = 0; ['visibRange'] = 'Ultra'; ['textures'] = 2; ['SSLR'] = 0; ['effects'] = 3; ['SSAO'] = 0; ['secondaryShadows'] = 0; ['sync'] = false; ['scaleGui'] = 1; ['preloadRadius'] = 150000; ['height'] = 768; ['heatBlr'] = 0; ['terrainMapMFD'] = { ['distance'] = { ['mapLodDistance3'] = 200000; ['mapLodDistance2'] = 100000; ['mapLodDistance0'] = 25000; ['mapLodDistance1'] = 50000; }; }; }; This behavior doesn't really make sense. I'm sure that the act of looking right on the Marianas map is exceeding the VRAM on my 3080, causing a temporary blip as stuff has to be paged out of VRAM for it, but when I look back forward, everything should get paged back in automatically (at least on DX11) and return to normal. And indeed, in the captures below, there isn't any persistent VRAM paging going on that would explain the severe drop in FPS. Similarly, when returning to the main menu, instead of getting 90fps like I was before, I'm at 25fps, and nothing strenuous is going on (I haven't grabbed a capture of this yet as it's much more difficult to get a capture of due to needing to alt-tab to start the capture, and alt-tab typically clears the issue in the main menu); I took a GPUView capture of the issue on the Marianas map (happy to provide to private support), and it clearly shows some kind of problem with how DCS is scheduling it's main thread. In summary, it appears to be sleeping too long and doing nothing. This isn't a problem that requires advanced multithreading, DX12, better VRAM management, etc... It simply appears to be an issue with DCS waiting too long to start rendering the next time. Here is a capture of a normal, stable 30 fps for me after loading into Marianas (higher settings than I mentioned previously, but still a decent 30fps): What I've gathered from this is that: In VR, DCS has a thread started from WinXrRuntime.dll that runs at 90Hz (or whatever the refresh rate of your headset is) You can see the row for this in the bottom left (labeled WinXrRuntime.dll) The blue boxes are when this thread is running on a CPU core. You can see a total of 14 of them, because this capture window covers about 160 milliseconds, so at 90hz we'd expect about 14 occurrences in a 160 millisecond time period I believe this is due to xrWaitFrame (https://www.khronos.org/registry/OpenXR/specs/1.0/man/html/xrWaitFrame.html), which must be called by the application to synchronize frame submissions with my headset. So this thread is probably calling xrWaitFrame on a loop to do that, which is why it's running at 90Hz The DCS.exe thread is running at 30Hz, every 3rd time that the WinXrRuntime.dll thread is running The WinXrRuntime.dll thread is seemed to be used to (among other things) schedule the main DCS.exe thread to run (i.e. this is how DCS ensures it synchronizes its frame submissions with your VR headset) You can see the row for this a bit above WinXrRuntime.dll, labeled as DCS.exe Similarly, the blue boxes are when this thread is running on a CPU core, which aligns with every 3rd time that WinXrRuntime.dll runs In this capture, when the DCS.exe thread is signaled to run, it executes for about 16ms This isn't fast enough to run at 90Hz As noted above, it's running at every 3rd time that the WinXrRuntime.dll thread is running I.e. this is running at 30Hz, or 30fps. Which with motion reprojection, is perfectly stable and playable This could run at 45Hz, however my GPU wouldn't be able to keep up, so it would be a bottleneck and my FPS wouldn't improve. You can tell this from the Device Context and RTX 3080/Hardware Queue 3D rows, which show that the GPU is being kept busy ~85% of the time. The GPU work scheduled to the Device Context queue also takes about 28ms to drain, so it's limited to about 30fps, and there's no point to the DCS.exe thread running more often (at least, not for scheduling GPU work) In summary, the main DCS.exe thread is running at the ideal 30Hz for how my hardware is performing, and I'm getting a reasonably smooth 30fps with motion reprojection Now, let's compare this with a capture of an abnormal, unstable ~22fps for me, on the same Marianas map, after briefly turning to the right and then looking forward in the same exact direction (i.e. should be rendering the same stuff as earlier in the 30fps capture): There are a few notable differences from the earlier capture that was stable at 30fps: The WinXrRuntime.dll thread is still running at the expected 90Hz The DCS.exe thread is running at 22.5Hz, every 4th time that the WinXrRuntime.dll thread is running (instead of every 3rd) In this capture, when the DCS.exe thread is signaled to run, it executes for about 23ms This is higher than in the good capture (23ms vs. 16ms) As noted above, it's running at every 4th time that the WinXrRuntime.dll thread is running I.e. this is running at 22.5Hz, or 22.5fps. Even with motion reprojection, this isn't very playable (it also varies a lot and isn't stable) This could run at 30Hz, but for some reason it isn't. My GPU isn't a bottleneck. The GPU work queued to the Device Context is still taking 28ms to drain, which is the same as the good capture. The GPU is also only being kept busy about 60% of the time, and there's no reason that the DCS.exe thread couldn't run more often (at least, probably not) In summary, the main DCS.exe thread could be running at 30Hz, and my GPU would be able to keep up, but for some reason it isn't, resulting in a very poor framerate My CPU also certainly isn't a bottleneck. I don't have an explanation as to why each frame of the DCS.exe thread went from 16ms to 23ms, but that's still fast enough to run at 30fps. I also have plenty of spare cores to which the DCS.exe thread could be scheduled to and run during this, so that isn't a problem either: In summary, it should be evident from these captures that DCS is more than capable of running this at a stable 30fps here, but isn't due to what appears to be a bug with the main DCS.exe thread literally deciding to do nothing and wait (lazy thread!). There is no CPU/GPU bottleneck. DCS must have some way of throttling how frequently the main DCS.exe thread runs. If it ran as frequently as it could, then my GPU wouldn't be able to keep up. This is why it only runs at 30Hz instead of 45Hz in the first capture, because even though it could run at 45Hz, the GPU would fall behind. So it must be doing something intelligent to throttle how often it runs. It's my theory that these framerate drops in VR (consistent with the symptoms mentioned here) are due to a bug with how the DCS main thread is scheduled to run, that causes it to run less frequently than it should in certain scenarios, and thus may be a relatively simple fix and a huge stability boost to anyone experiencing this issue (including myself). I'm happy to provide more info. I also have logs available, but I'll note that they look the same whether or not the issue is occurring, so I don't believe they'll do much good. I also have some experience in this field and would be happy to sign an NDA and look into the issue myself, but I'm sure ED has people better suited to fixing this than me. If only they're able to reproduce and look into the issue, I'm sure a lot of VR customers would be very grateful, as this issue is extremely disruptive, to the point of making the game unplayable in VR without excessive hardware purchases (like a 3090) or turning settings down to very low levels and avoiding maps like Marianas or even MP entirely.