XeSS implementation into DCS World (and the better alternative to both DLSS and FSR?)

[LucShep]

I'd like for Eagle Dynamics to have a good look at this. This might be it.

 

It's already causing a lot of interest in gaming tech and, for DCS, it could be the additional solution for problems that most of us have with extremely intensive GPU usage in-game, especially at a time when the current market is horrible for anyone looking to upgrade their GPUs.

PS: for those of you with the attention span of a 12 year old, you can skip down to the videos to get the idea.

 

We've heard about Nvidia DLSS (Deep Learning Super Sampling).
This is a proprietary temporal image upscaling technology developed by Nvidia. It's AI based, machine deep learning mechanism/algorithm and, after revision 2.0, it produces fantastic results, with great image quality.

But there is a problem - it's exclusive to Nvidia hardware only, it does not and will not work on anything other than Nvidia GPUs.
That leaves out a very large chunk of users with current and future AMD GPUs, as well as upcoming Intel GPUs (coming next year).

 

Then we have AMD FSR (FidelityFX Super Resolution).

A bit like Nvidia's DLSS, AMD's FSR goes through a process called downsampling. It does have the greater benefit of being open-source, working with any recent and future GPU, whatever the brand.
But, it's not a machine learning mechanism/algorithm, instead it's a much simplified process, and here's a pretty noticeable problem... it does not produce the same image quality as DLSS (it simply does not look good at resolutions lower than 4K), which is disappointing to realize in the various analysis. 

 

And here comes Intel XeSS.
I think most have seen that Intel GPUs are coming early next year. Where it gets relevant for the matter in topic, is that Intel has also new upscalling tech, called XeSS.
XeSS is AI based, machine deep learning mechanism/algorithm (like Nvidia's DLSS) and, here comes the interesting part, it is to be open-sourced to competitors and game developers.
More than a very strong direct competitor to Nvidia's DLSS, Intel's XeSS will work on all GPUs, current and future, whatever the brand (AMD, Nvidia and Intel's own GPUs). 

Also, Intel revealed that they are already working with Unreal Engine, Microsoft (for DirectX 12 Ultimate), Vulkan (Kronos Foundation) and Unity, for API and implementation.
Demonstrations and analysis show already as good results, if not better ones, than Nvidia's DLSS. The excelent quality even when in motion (and not just in still images) is trully remarkable.

 

My own take on this is that XeSS is a stroke of genious from Intel, to get into a market where they have zero share. Have no doubts that this tech will disrupt things quite a bit.

Being the giant they are, far bigger than AMD, maybe even than Nvidia, they wouldn't get into such big announcements if they weren't very serious about this.

And, for what is worth, they've now also hired DLSS' pionner (Anton Kaplanyan).
 

The way I see it, it's all benefits with no downsides.

Everyone will be able to to use it, so long as games adopt it. Being associated with this technology seems to me the right way to go, for any game developer.

More to the point, for E.D. and DCS, it may turn out to be a valid solution to get (much needed!) performance from their graphics with no visual impact on image quality.

 

Following is Intel's demonstration of XeSS: 

 

(please watch in fullscreen at your monitor's native resolution) 

 

 

The demos that they showed off speak for themselves, and seem to show just how good XeSS is at upscaling.

 

EDIT: More technical information, analysis and dissertations on XeSS:

 

 

 

 

 

Edited August 22, 2021 by LucShep

[upyr1]

I don't care what they use, just as long as it won't run like crap on a particular cpu or graphics card

[cfrag]

This is really exciting news. Now I'm waiting for demonstration videos on how all these technologies works with in-game footage that changes a lot, for example VR footage of a head moving inside a cockpit. Up until now, all I have seen are slow-moving demos where the background scaling can happen piecemeal until after 10 (or so) frames the entire frame is at full quality. But what happens when the view changes rapidly, and continually? From my (amateurish) understanding of this technology, that pretty much represents the worst case scenario. When the worst case scenario is the norm (in VR your head constantly moves, in flight sim combat even excessively so), the added value may be minimal. Don't get me wrong, I really want this technology to succeed. I merely want more information and some more dynamic (dramatic, constant view changes) demos to see how these technologies stack up, and what improvements we can realistically expect.

Edited August 21, 2021 by cfrag

[Mars Exulte]

12 hours ago, LucShep said:

I'd like for Eagle Dynamics to have a good look at this. This might be it.

  Oi, it's literally every two or three days somebody says this @@

[5ephir0th]

We already have DLSS and FSR here and we know how it works.

We still dont have here XeSS and we dont know how ti WILL work so i think is a bit early, neither the Intel GPUs have been released...

[SkateZilla]

Intel has a history of making their proprietary code run like blah on other products.

[LucShep]

Extracted from The Intel XeSS Interview with Principal Engineer Karthik Vaidyanathan (source wccftech.com):

 

 

Usman: In terms of competitors, we have FSR, DLSS 1.0, DLSS 2.0 and even Unreal Engine's temporal upscaling, right? It's my understanding that XeSS is more like DLSS 1.0 and 2.0 because it's also based on neural super sampling while FSR does not use neural networks, and it's not ML or AI based and it also doesn't have motion vector support. So how would you classify FSR, DLSS 1 and 2, and XeSS just so readers can sort of place you on a map in terms of rendering technique.

 

Karthik: Yes, so the super sampling techniques need motion vectors because you're trying to use information from previous frames and you need to know how they move and how objects move.

So FSR to my knowledge is spatial upscaling, and we already discussed spatial upscaling and some of its limitations. DLSS 1.0, again I am not aware of the internals of DLSS because its not open, but from my understanding, it was not something that generalized across games, DLSS 2.0 plus was neural network based and it generalized very well. XeSS from day one, our objective has to be a generalized technique.

 

Usman: So it's most comparable to DLSS 2.0

 

Karthik: You could say that. Again, the technology is likely very different. To the extent that yes, it's neural network based. Yes. It is a super sampling technique. And yes, it generalizes to that extent. It's similar to the other system [DLSS 2.0], but the underlying technology is likely very different. Because when you have two independent groups, trying to solve a problem in their own way they will likely end up with very creative solutions to the problem.

You mentioned Unreal Engine 5 and it produces some of the highest quality shadow geometric lighting fidelity and when you're investing so much in your render, you really don't want to lose any of that quality when you scale from that - those pixels to your target resolution - and that's been our objective from day one. And also you don't want to have a solution that's fragile that requires training for every game that someone ships that's also been our objective from day one.

 

(...)

 

Usman: and it's my understanding is that it will initially be released as closed source, but eventually move to open source. Is this understanding, correct?

 

Karthik: I am not fully aware of the timelines involved – but it will be eventually open sourced, that I can confirm.

 

Usman: Fair enough. Are you guys working with any major game developers right now, to integrate XeSS or any ecosystem partners for XeSS specifically, like I know you guys are working with a lot of people as far as Xe HPG goes but what about XeSS or has it not reached that stage yet where you would sort of partner up with the ecosystem partners or other game developers?

 

Karthik: There are several partners that we are working with. I cannot comment more at this point.

 

Usman: but these are game developers right or other partners.

 

Karthik: This includes game developers. I wish I could share more haha.

 

(...)

Usman: Will XeSS be applied at the driver level in all games or need to be implemented natively in game engines? We are hearing of a tool which allows FSR to be implemented on pretty much all steam games?

 

Karthik: Just like DLSS, it would have to be integrated into the game engine. It's not something that can be hidden from the game engine, and that's why we need to work with ISVs to get this into the game and are trying our best to make it as easy as possible integrate this.

It requires developer support but having said that, generally, super sampling technologies that are implemented at the tail end of the pipeline, closer to the display, will always have more challenges. I can give you a clear example. Let's say you had film grain noise that was introduced as a post process - trying to apply an upscaling or super sampling solution after that fact becomes very challenging.

So even if one were to implement something like this as an upscaling solution, for example, just close to the display, there's always going to be scenarios like this when you know the game engine does some kind of post processing that just breaks it. So being closer to the render gives you, as we discussed the last time, the highest fidelity information with the amount of controllability that you need to be able to produce the best result.
 

(...)

 

Usman: How hard will XeSS be to implement compared to DLSS?
 

Karthik: It should be similar and there's another way to look at it. So for a game that implements TAA already, integrating something like XeSS should only be a small amount of effort because you already have all the pieces that we need with any TAA Implementation. Like you have the motion vectors, you have the jitter. So you have all the pieces for any kind of super sampling technique to be integrated if the game already has a TAA implementation. So that's a pretty large set of games right now. TAA has almost become like a de facto, you know, standard for antialiasing. So for any game that already has TAA, it already has the pieces that you would need to integrate XeSS or any super sampling technique with a few modifications, of course, but those are small modifications.

 

 

Edited August 27, 2021 by LucShep