I was wondering myself what overlay VR uses.
FSE = Fullscreen Exclusive - was used on Win7 and is most robust mode, it decreases tearing and app runs in its own exclusive window (was always prefered to windowed modes) also lower input lag!!!
borderless = windowed mode without borders from windows (causes latency)
windowed = works same but has borders
Now even if you disable FSO (Fullscreen optimizations) in compatibility tab of a game shortcut, it does not work! You can test it by changing game volume in-game (using Fullscreen mode), if there is a pop out window, it does not work and FSO is on and game is running in Fullscreen Hybrid mode!
You have to go to registry and change these values... save as .reg file, but first right-click given registry key and click export to backup registry!
Copy this into note pad and save as all files, appendix .reg and double click it, ok:
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\System\GameConfigStore]
"GameDVR_DXGIHonorFSEWindowsCompatible"=dword:00000001
"GameDVR_FSEBehavior"=dword:00000002
"GameDVR_FSEBehaviorMode"=dword:00000002
"GameDVR_HonorUserFSEBehaviorMode"=dword:00000001
Now check again in your game in Fullscreen mode! Whether or not you can see sound menu pop out!!! When changing volume using media keys on your keyboard (make sure HID service is on) You will be able probably to tell based on input lag, or stuttering too!
Now this does work only until DX11.
Since DX12/Vulkan, devs can choose to not include gold old fashioned Fullscreen exclusive mode, so game defaults to using Fullscreen hybrid mode (FSO), because there is nothing else to use. https://devblogs.microsoft.com/directx/demystifying-full-screen-optimizations/
Now there is new MPO overlay: https://nvidia.custhelp.com/app/answers/detail/a_id/5159/~/v-sync-off-not-recommended-as-a-global-setting-starting-with-driver-version
Which should work in borderless and has low input lag!
But I did not see in VR games option to choose overlay (which fullscreen mode, or windowed mode). So I am not sure, how to change this. Maybe via config. I will keep looking!