Jump to content

32 bit normal maps


Taz1004

Recommended Posts

36 minutes ago, LucShep said:

 

 

Perhaps make a separate DLC with those overkill textures if so inclined, for the single digit percentage(?) of users who are unworried about the noticeable VRAM hit and appreciate such kind of (really) minute details.

 

 

We're getting a free map so I don't see why not? It'd be as perfect of a solution as possible. 

 

Either way, big thanks to Big_Newy and Pikey for bringing this up to devs.

  • Like 3

Reformers hate him! This one weird trick found by a bush pilot will make gunfighter obsessed old farts angry at your multi-role carrier deck line up!

Link to comment
Share on other sites

Lots of references to us in here, so I'll jump in-

 

The standard compression for normals, both in our products and EDs is BC5. (https://docs.microsoft.com/en-us/windows/win32/direct3d10/d3d10-graphics-programming-guide-resources-block-compression#bc5) This is a two-channel format that drops B in favour of better compression in R and G. The B channel is subsequently reconstructed in the shader for a correct unit vector representation of the surface. It looks like the A-10 files are not in BC5; which is odd, but likely an oversight. It would be more productive to report this as a bug. 

A couple of additional points:

  1. 8-bit normals are terrible, unfortunately I'm too busy to provide you with a set of examples - but this rings true for anything with sleek, rounded shapes and significant broad stroke normal detail. Not just shiny surfaces. Perhaps I'll get to producing some of these examples, but the assertion that one should use DXT1 for normals is silly. They have their place for something like landing gear, where roughness and albedo detail hide poor normals, but certainly not for major airframe parts. 
  2. BC5 compression for normals is industry standard at this point. The biggest case in point on this is probably UE4 that defaults to BC5 under the "Normals" texture group/sampler type. It's not an artist fever dream. 🙂

Edited by Cobra847
  • Like 6
  • Thanks 6

Nicholas Dackard

 

Founder & Lead Artist

Heatblur Simulations

 

https://www.facebook.com/heatblur/

Link to comment
Share on other sites

Here's comparison once again as how this affects DCS is only thing that really matters.

 

The mod I installed through OvGME in the screenshot is nothing more than cockpit textures in \Mods\aircraft\A-10C_2\Cockpit\Textures\ with 32bit normal maps converted to 8bit.  No other resizing of texture resolutions.  Again, this is only on A10C-II cockpit textures.  Not including my other optimizations on trees and vehicles.

 

Default 32 bit normals

VR_Normal.jpg

 

Converted 8bit normals

VR_8bit.jpg

The two numbers per screenshot, the first number is when the game started.  Second number when it drops is when I alt-tabbed out to take screenshot of the desktop.  And both numbers were affected by the optimized texture.

However, here's strange thing I didn't see when I tested this before.  Once I alt-tabbed out which lowered the VRAM, then go back into the game, that low number stayed.  I wonder if this is Afterburner issue.

 

And for those who are convinced there's quality difference, here's zoomed in shot of gauge with 8bit and 32bit normals as used in DCS.  The normal map would be applied to the bevels, edges and screws.

 

Normal_Quality.gif

 

Once again, this really doesn't matter to me personally as I already optimized them and everything running smoothly on my age old PC.  I really don't need to convince anyone.


Edited by Taz1004
  • Like 8
  • Thanks 2
Link to comment
Share on other sites

On 2/22/2021 at 4:25 PM, Cobra847 said:

8-bit normals are terrible, unfortunately I'm too busy to provide you with a set of examples - but this rings true for anything with sleek, rounded shapes and significant broad stroke normal detail. Not just shiny surfaces.

 

Is there any sleek rounded shapes with significant broad stroke in the cockpits?  The point I keep making is that where higher bit normals can actually make slight improvement, they are not used.  Instead, as I illustrated and mentioned many times, where it DOESN'T make ANY difference is where they are used.

 

Quote

Perhaps I'll get to producing some of these examples

 

I'd love to see them.  But I'm willing to wager that even if that is the purpose... on broad shapes, you may see slight improvement from 8bit to 16bit but you will not see any difference between 16bit and 32bit.

 

And even from 8bit to 16bit, on large rounded surface, once you put diffuse map over it, it'll be hardly noticeable.  Definitely not worth the VRAM which is why it's generally not "practiced" in game industry.  Unless it is highly polished shiny surface.

 

But then again, you don't see 32bit normals on racing sims where it's predominantly "highly polished", "large rounded" surfaces.  In fact, Assetto Corsa doesn't use any normal maps on car body surfaces.  That's because modern graphic cards are more efficient at processing polygons than large texture files.  LOD is much easier with polygons than with textures and doesn't affect draw calls.  And it depends on the engine but typically, normal maps can also do some "strange" things under certain lighting which are not noticeable on small details but definitely on large surfaces.  So in "general", normal maps are reserved for fine details such as cut lines and rivets and screws.


Edited by Taz1004
  • Like 6
Link to comment
Share on other sites

And if the Dev team is looking into this, please don't forget these are everywhere.  Not just in A10C-II.  174MB 32bit normal maps for road markings in Caucasus dating back to 2016.  I've also seen many in ground units (where surfaces are flat) dating back to 2014 and some in weapons too if I remember correctly. 

Does it really take an expert to see that this is an oversight?

 

Normal_32bit.jpg

RoadNormal_01.jpg

 

 

 


Edited by Taz1004
  • Like 11
Link to comment
Share on other sites

@Cobra847 thanks for the reply. i understand your points (well, as far as i understand the subject matter; game engine technicalities and specific compressions methods go over my head unfortunately) and have no reason to doubt you, but i think the more pressing issue is what @Taz1004 is so eager to point out thanksfully: it's not so much the problem of a specific texture being a specific way. if an artist thought that this was the appropriate texture size for a given object, than i would trust his judgement. you could of course argue, that he was biased too much towards fidelity instead of performance, but i think this is not the main point of most of the discussion here: to me it is very obvious that those judgements very simply not made on an per object basis. for me it is evident when you look at the ground vehicles and their texture set sizes. at this point i still refuse to believe that multiple hundreds megabyte of textures for a ground unit are appropriate. realtime texture streaming simply does not work well enough in the current engine on midrange hardware.

you don't have to comment on this, if your comemnt would critizise your collegues, but i just wanted to point out that -at least for me - the problem is less about the specific decissions to use a certain compression/size for a task, but simply the reality that dcs has absurd texture sizes for less important objects while at the same time struggling with vram management on lower and midrange hardware.

  • Like 9
Link to comment
Share on other sites

Even the beautiful (270 MB) B-17 has 8 files that are over 65537 KB: \DCS World OpenBeta\Mods\tech\WWII Units\Textures\B-17G.zip

This is a great "Low Hanging Fruit" optimization solution that will get addressed sooner or later.  Thanks Taz!

  • Like 3
  • Do not own:  | F-15E | JF-17 | Fw 190 A-8 | Bf 109 |
  • Hardware:  [ - Ryzen7-5800X - 32GB - RX 6800 - X56 HOTAS Throttle -  WINWING Orion 2 F16EX Grip - TrackIR 5 - Tobii 5C - JetPad FSE - ]
Link to comment
Share on other sites

  • 2 weeks later...

I think @twistkingmakes a good summary. In a bubble, having quality is the most important to an artist but that bubble doesnt extend to cover the user bases needs.

User perspective:
We see examples in some cases where there doesnt seem to be a gain from such large textures, and in fact these are hurting performance with no gain.

Artist perspective:
There are specific cases of rounded materials and shiny surfaces where we must use the 32bit normals to achieve good quality

And ideally we would live in the intersection of both the artists and the users needs, but we appear to be living only in the artists and this ignores issues of optimisation and good performance.

I think there is a disconnect, I think the perspective of the user is missed or diregarded because the line between 32bit texture size and GPU performance in the average consumers machine is not publically and absolutely proven to create performance issues. And I beleive myself that there is a problem and it's evidenced by mine, and others simulations running badly and performing badly because we play the game, we don't test it in these bubbles.

I don't know what else there is to do but keep showing the link between performance and texture size. It's not easily done.

I think it causes this:
https://forums.eagle.ru/topic/246886-dcs-causing-steamvr-fail-description-evidence-and-how-to-replicate/page/6/?tab=comments
and this
https://forums.eagle.ru/topic/264164-extensive-vr-bug-my-research-around-it-f10-other-view-changes-kicks-you-out-of-vr/

 

which looks like this


which is explained by this post and also this post:


And I evidence it with reproducing more often with textures set to high than textures set to low.

But if I'm wrong, then how is this proven that I am wrong? Because it seems that the burden of proof shows there is an issue with large textures and they are causing poor simulation performance, especially for VR users.
Obviously these texture sizes hit VR players harder - symptoms of issues are not correlating to VR users because of the relatively low proportion of the user base comapred to the 2D folks.
This is the extent of my subjectivity on this matter, I've been judged for trying to keep people data driven, but I know for a fact if you offer subjectivity, anecdotes and no data, you will get mocked in this forum, hence I've kept my final conclusions away from posts like this. I dont have the knowledge to link the symptom to the data, not many people do. But I can offer my experience as data, and my experience is in general agreement that textures are the root of quite severe issue in performance that can cause the entire sim to stop functioning normally.

  • Like 3

___________________________________________________________________________

SIMPLE SCENERY SAVING * SIMPLE GROUP SAVING * SIMPLE STATIC SAVING *

Link to comment
Share on other sites

Has anyone done some sorting on the data?  Are there a few very large files with 32bit normals?  Or a huge number of very small files with them?  Is it possible to sort out everything "round and shiny" from candidates for automated conversion?

Link to comment
Share on other sites

1 hour ago, glide said:

Has anyone done some sorting on the data?  Are there a few very large files with 32bit normals?  Or a huge number of very small files with them?  Is it possible to sort out everything "round and shiny" from candidates for automated conversion?

No I wouldnt expect so, one of the main statements in favour of wholesale replacement was that you dont see them in cockpits, mostly large exterior pieces. But the sorting is also a stumbling factor. As Taz said, he optimised his, tested it and liked what he saw. Yet there is an element of "manual" about the testing and specific about the type (A-10C II). I dont think a single answer fits all. And it was pointed out that the A-10C was in fact an anomaly, and that was also used for much of the OP's testing. So its not such an easy topic to wholesale provide answers to.
One question i would have is to the people producing textures with the modules, how hard is it to be more selective about what you target for what bit size? Is this all about process and saving time?

  • Like 2

___________________________________________________________________________

SIMPLE SCENERY SAVING * SIMPLE GROUP SAVING * SIMPLE STATIC SAVING *

Link to comment
Share on other sites

1 hour ago, Pikey said:

No I wouldnt expect so, one of the main statements in favour of wholesale replacement was that you dont see them in cockpits, mostly large exterior pieces. But the sorting is also a stumbling factor. As Taz said, he optimised his, tested it and liked what he saw. Yet there is an element of "manual" about the testing and specific about the type (A-10C II). I dont think a single answer fits all. And it was pointed out that the A-10C was in fact an anomaly, and that was also used for much of the OP's testing. So its not such an easy topic to wholesale provide answers to.
One question i would have is to the people producing textures with the modules, how hard is it to be more selective about what you target for what bit size? Is this all about process and saving time?

 

You will need to use the profiling tools to test exact impact but you need the SDK.  Without the SDK, best tool available is AfterBurner's per process monitor which is what I used.

I believe I've provided enough evidence for developers to investigate which I hope they are doing.

As for how to find the 32 bit normals, all texture files of specific resolution and bit depth have specific file sizes.  Because they're uncompressed (not talking about block compression).  So you can identify them just by looking at file sizes.

  • Like 3
Link to comment
Share on other sites

I think we should be patient.  I reported this tree normal issue on August 4th last year.

 

And it was fixed on November 4th patch.  It was a simple fix too but took 3 months.  I don't know how ED prioritizes or if Wishlist is lower priority but I expect at least 3 months for something like this to be looked at.


Edited by Taz1004
  • Like 4
Link to comment
Share on other sites

21 minutes ago, danperin said:

@Taz1004 An option as well, that keeps it as 32 bits but reduces its size to half, is the DXT1 (no alpha) format.

For external details isn't that good, but for internals, no problem.

 

Screenshot_1.png

 

BC7

* 21,3MB

 

Screenshot_3.png

 

BC1 (no alpha)

* 10,6MB

 

Screenshot_2.png

 

 

I don't want this to get off track and get into block compression but that's actually not correct.  As Cobra said above, DCS uses BC3 and BC5.  BC7 also works but not for older assets.  Even tho BC1 may be lower size, it is inefficient and no longer used in game engines.

 

https://docs.microsoft.com/en-us/windows/win32/direct3d11/texture-block-compression-in-direct3d-11

 


Edited by Taz1004
  • Like 2
Link to comment
Share on other sites

  • 2 months later...
  • 2 weeks later...

This is in the wishlist thread. It might as well be on the moon as far as ED is concerned. I am afraid that 32 bit normal maps are here to stay. Indefinitely. 

  • Like 1

Specs: Win10, i5-13600KF, 32GB DDR4 RAM 3200XMP, 1 TB M2 NVMe SSD, KFA2 RTX3090, VR G2 Headset, Warthog Throttle+Saitek Pedals+MSFFB2  Joystick. 

Link to comment
Share on other sites

  • 3 weeks later...
  • 1 month later...
  • 3 weeks later...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...