32 bpc Textures

[Taz1004]

This may be shelved in Wishlist like last time but I really believe this needs to be considered a bug.  If an unnecessary line of code costs 10% of VRAM, that'd be considered a bug.  I don't see how this is different.

I mentioned this in another thread looong time ago and looks like nothing has changed.  In fact, has gotten worse it seems.  I optimize most of the textures and below is comparison between my optimized textures and default.  VRAM usage is at bottom right corner.  There is absolutely zero quality difference.  Different shade of glass tint is just my personal preference.

 

Black Shark 3 Syria Shooting Range Instant Mission

 

A10C-2 Operation Piercing Fury Mission 1

 

FA-18 Supercarrier PG Cold Start

 

As you can see, newer the module, bigger benefit there are from optimization.  I don't have Apache but its textures look huge.

All I did to optimize is reduce all RoughMet and Normal maps from 4K to 2K.  No change to Diffuse maps.  And converted ALL 32bpc textures to 8bpc.  And all FLIR textures, I downsized to 1K.  I'm not sure why something you see in small MFD screen needs to be 4K textures.

Edited March 21, 2023 by Taz1004

[Flappie]

Hi @Taz1004. Many thanks for this thread. I'll report this solution to devs.

Can you please give us a link to said mods so devs can see the difference right away?

[Taz1004]

On 3/21/2023 at 3:51 AM, Flappie said:

Hi @Taz1004. Many thanks for this thread. I'll report this solution to devs.

Can you please give us a link to said mods so devs can see the difference right away?

It's for my own as it's too large to share.  Currently over 4GB.  Plus, I can only optimize the modules I own and I don't own everything.  Below is one of the zip file from the mod as an example of what I changed.

Removed

Original is 335MB with 32bpc textures.  My optimized version is 105MB.  File exists in /CoreMods/tech/TechWeaponPack/Textures/

Edited May 5, 2023 by Taz1004

[lefuneste01]

Could you please post a short tutorial to do it with gimp ?

[Flappie]

@Taz1004 That will do. Many thanks.

Edited March 21, 2023 by Flappie

[Taz1004]

On 3/21/2023 at 4:03 AM, lefuneste01 said:

Could you please post a short tutorial to do it with gimp ?

I don't use gimp but anyone can do it with MS Visual Studio Community which is free.

1. Unzip the texture file first.
2. Sort by file size.
3. Start with the largest file.  Roughmet files should say "Roughmet" and Normal files should say "Normal" or "_N" or "Nrm" or "NM" so you can also search by name.
4. I don't optimize diffuse or color textures (unless they're really unreasonable like 4K texture for pilot's face) but you can if you're really starving for VRAM.  But this would reduce image quality.
5. I usually make Roughmet and Normals half (1/4 actually) of diffuse.  So if diffuse is 4096x4096, I reduce Roughmet and Normals to 2048x2048.  (Refer to image below).  If diffuse is 1024, I make Roughmet and Normal 512 etc.  All resolution must be power of two (512, 1024, 2048... etc)
6. If it's 32 bits per pixel as in the image below, then change the format to "BC3_UNORM" or "BC7_UNORM".  It should automatically convert to 8bpc
7. If the texture is for ground unit, search for FLIR and reduce them to 1024x1024.  There's also separate FLIR.zip file in /Bazar/World/textures/
8. Then just save, re-zip the files and make sure zip file name matches original.
9. Make it into a mod and install using mod manager.

Roughmet are for defining surface finish.  Whether it's glossy, matte, metallic, etc.  They really don't need to be 4K nor 32bpc.

Normals are for bumps.  Usually used for rivets, bevels and grooves.  Imagine 8bpc as 3D printer that prints the rivet in 256 layers.  And 32bpc would be printing the rivet with 16 million layers.

 

Original (65MB)

 

Optimized (5MB)

 

Currently, I optimized texture files in:

1. /Mods/aircraft/(aircraft name)/Cockpit/Textures/
2. /Bazar/World/textures/
3. /CoreMods/aircraft/(aircraft name)/Textures/
4. /CoreMods/tech/TechWeaponPack/Textures/
5. /CoreMods/tech/USS_Nimitz/Textures/

Starting with the largest ones.

Edited April 8, 2023 by Taz1004

[lefuneste01]

thanks !

[Cleric812]

Hello all,

I just spent some time writing a batch file for texconv.exe (https://github.com/Microsoft/DirectXTex/wiki/Texconv) to try to streamline the process. The thing that struck me most during this is that there seem to be some absolutely huge normal maps and roughmet textures, some as large as 80+ MB. The other thing that I noticed was the apparent lack of standardization of the texture sizes and names. some normal maps have "normal" in their name while others have "nrm" or "NM" in there names. I understand they were most likely created at different times and by different developers. However I think it might be worth it for someone to sit down for a day or two to go through these textures and standardize them.

I cannot tell any difference between a 4096x4096 32 bit roughtmet or normal map and a 2048x2048 DXT5 file does the job just as well, often cutting total size of texture for the models in half at times.

 

Just my 2 cents

[Azorath]

On 3/21/2023 at 10:59 AM, Taz1004 said:

It's for my own as it's too large to share.  Currently over 4GB.  Plus, I can only optimize the modules I own and I don't own everything.  Below is one of the zip file from the mod as an example of what I changed.

Optimized VAB_Mephisto.zip

Original is 335MB with 32bpc textures.  My optimized version is 105MB.  File exists in /CoreMods/tech/TechWeaponPack/Textures/

 

there  must be a way to automatizing the process without needing to constantly uploading the edited files. lemme search Taz. 

Ah, also; You, a valuable modder of the community, should've been given access to all modules.  As you will add more to the community with those. I just wanted to share my opinion.. 

[twistking]

thanks @Taz1004  you're doing great work. here's hoping that this thread wakes up the responsible devs: 5/5 stars!

texture sizes are ridiculously oversized. i bet that you could squeeze them even more before quality would degrade to a point where it was really notable.

[Marco Schaap]

I've manually resized all of the cockpit and external textures for the F16 using your method. I'm eager to test what difference it makes, in terms of load, quality and / or performance. Great find! 

[RuskyV]

I bet this would help with stuttering that’s been seen by people, hope ED have a good look at this as I think it could really help a lot of people. 
 

Thanks Taz.

[Flappie]

This was reported internally for a number of aircraft. I added this excellent thread to one of the internal reports. Thanks @Taz1004.

[Luse]

@Taz1004

Would you be willing to host your edited files via Google Drive or elsewhere? Huge fan of all your work. 

Thank you! 

[Taz1004]

20 hours ago, Luse said:

@Taz1004

Would you be willing to host your edited files via Google Drive or elsewhere? Huge fan of all your work. 

Thank you! 

I might once I'm done with it.  Keeping it updated scares me tho.

[Taz1004]

On 4/4/2023 at 8:57 PM, Luse said:

@Taz1004

Would you be willing to host your edited files via Google Drive or elsewhere? Huge fan of all your work. 

Thank you! 

Posted here.

 

[Taz1004]

Whoever did these South Carolina National Guard Liveries for Apache had them optimized exactly the way I would.  So there is someone at ED who agrees with me.

 

[lefuneste01]

I tried to convert textures for the AH64 (my god they provide a 65 GB normal texture for..the seat !) and had this issue (see picture below) So I think the file ah-64d_cpt_glass_refl_RoughMet.dds should not be converted...

The command I used : texconv.exe -flist file_list.txt -o convert -w 2048 -h 2048 -f BC3_UNORM

I put all _*n* and *Roughmet* file names into file_list.txt, for size greater than 2048x20248.

I need to restore the ah-64d_cpt_glass_refl_RoughMet.dds and check in VR.

Texture folder size divided by 3....

after:

Before

Edited April 7, 2023 by lefuneste01

[Taz1004]

On 4/7/2023 at 7:43 AM, lefuneste01 said:

I tried to convert textures for the AH64 (my god they provide a 65 GB normal texture for..the seat !) and had this issue (see picture below) So I think the file ah-64d_cpt_glass_refl_RoughMet.dds should not be converted...

The command I used : texconv.exe -flist file_list.txt -o convert -w 2048 -h 2048 -f BC3_UNORM

I put all _*n* and *Roughmet* file names into file_list.txt, for size greater than 2048x20248.

I need to restore the ah-64d_cpt_glass_refl_RoughMet.dds and check in VR.

Texture folder size divided by 3....

after:

Before

 

 

Yea, I try not to touch textures for complex shaders.  Damage map being another one.  You'll find out which ones they are as you do more as you're already finding out.

And whenever I see regular image files like tga, jpg, png, bmp, I do try to convert them to dds.  Those regular image files do not support mips and they shouldn't be used for diffuse.  Converting them usually work as long as you use same file name.  And you have to create mip levels using tools menu in Visual Studio.

Edited April 8, 2023 by Taz1004

[Recoil16]

On 4/7/2023 at 1:45 AM, Taz1004 said:

Whoever did these South Carolina National Guard Liveries for Apache had them optimized exactly the way I would.  So there is someone at ED who agrees with me.

They weren't done by ED, but by @static_actual as part of the AH-64D livery competition if I recall correctly.

[static_actual]

On 4/6/2023 at 7:45 PM, Taz1004 said:

Whoever did these South Carolina National Guard Liveries for Apache had them optimized exactly the way I would.  So there is someone at ED who agrees with me.

 

@Recoil16 is correct, those are my work, though that was on my old forums account. If you take a look at the description.luas, though, you will see a comment labeling me the maker.

I do wonder why your install seems to show the old naming convention; the naming conventions were changed when I updated them to remove the new default patches (which covered up my custom patch work). I've attached a screenshot showing the updated folders.

If you're talking about the heavy use of shared files, I am a bit of a sucker for those, and you'll see those on pretty much all of my skinpacks (including my 605 sqn one from the recent Mosquito competition)null; if you're talking about the use of 2k roughmets, that was actually only so I could keep it under 300mb for the user files upload limit while using the higher quality BC7 compression format, though it certainly does have the added benefits you've already mentioned here.

[Taz1004]

On 3/26/2023 at 6:58 PM, Flappie said:

This was reported internally for a number of aircraft. I added this excellent thread to one of the internal reports. Thanks @Taz1004.

 Just by remote chance somebody internally at ED insists that there is benefit to 32bpp normal maps, referencing videos like below, I'd like to make one things clear.

 

One thing common about these videos showing benefit of 32bpp normal maps is that they almost always show it very close up on highly reflective surfaces like a chrome or mirror object.  Because that is the only way to see the difference.  On real world game scenarios like DCS where nearly everything is matte and weathered, you will not be able to tell the difference between 8bpp and 32bpp.

Below is another video that starts out by showing the benefits of 32bpp normal maps on chrome material but towards the end at 11:49, he shows 8bpp is just as good for matte materials.  And as he said in 12:22, if you bake in 32bpp out of 3D software and then reduce it to 8bpp in Photoshop, it works well even on chrome materials.  This may be viewed as another step in the pipeline but all this can be automated with batch, command line, and even scripting.

 

And if somebody does make a chrome plane, it still should be 16bpp as the video shows.  Not 32bpp.

Edited April 12, 2023 by Taz1004

[fab.13]

Hello Taz and lefuneste,

I can see in VR multiplayer session that some plane generate ghosting whereas other plane are clean

By exemple, when in VR formation flying in my cockpit, F-86 is ghosting whereas SU27 or Harrier are not ghosting at all.
In this session, Su27 and F-86 and me were flying together behind a Harrier formation flying, i just had to turn my head left or right to see each different plane

So i suspect F-86 textures to be one of source of this ghosting problem ( by looking in the f86 folder, textures are very huge ... )

My Question :

Could it be a good test that a I try to optimize F-86 textures, as you decribed,  and check if the VR ghosting problem is solved or not ?
The multiplayer server doesn't need 'pure client' ( IC check )

 

Edited May 2, 2023 by fab.13

[Cleric812]

Good afternoon all,

i was wondering if there had been any progress on this from the dev’s side? Perhaps @Flappie can give us an update?

[The_Nephilim]

Thank you Taz    yes ED Needs to incorporate these into the game hopefully having Taz blessing..

 

I had a question most of those files in the Textures folder are zipped, would the game load quicker if I had them all unzipped? I know it would take up more space on the Drive that is not my concern.. I may test it if it is even possible..

 

Well I unzipped all those texture files and it seems to take longer to load the mission after I hit fly?? Not sure if it helped or got worse..

Edited November 19, 2023 by The_Nephilim