DCS on separate drive performance?

[kksnowbear]

1 hour ago, SharpeXB said:

Can a faster SSD give you better frame rates? I’m going to guess no. 

No, and that's commonly understood.

However, as I already tried to explain to you, recently: Frame rates are *not* the be-all/end-all indicator of gaming PC performance that you seem to think they are.

[SharpeXB]

1 hour ago, kksnowbear said:

No, and that's commonly understood.

However, as I already tried to explain to you, recently: Frame rates are *not* the be-all/end-all indicator of gaming PC performance that you seem to think they are.

How about this clarification:

Performance = Frame Rate

Stability = stuttering, crashing... whatever else. 

But arguably frame rate is usually the general measure of performance. In all the bug threads here about performance issues, stuttering etc. I don't recall seeing any feedback from ED that the game being on same drive as the OS is a culprit.

And the chief benefit most people are going to notice from an SSD is the loading time, really... maybe stability

Edited February 26 by SharpeXB

[BIGNEWY]

folks please keep the discussion friendly. 

You can put DCS on any drive, the fastest drive you have will always be the best drive, just watch for bottlenecks if you are having issues. 

thank you  

[Baldrick33]

9 hours ago, kksnowbear said:

Again, the fact that you continue to use indefinite phrases like "seem most unlikely" means you understand that there's more to it.

Not sure what "IT" you worked in or how long, but if you don't understand resource contention in computer systems, then it's difficult to imagine what exactly your training and experience involved.  I mean, sure, we had guys who "worked in IT" whose job was pretty much changing backup media... sorry if it wasn't clear but I don't think any of them ever gave a moment's thought to things like resource contention. I guess I'm referring more to people who are in a professional capacity responsible for systems design, commissioning, and maintenance.

But here's my favorite part...just to be safe, you turn around and say this:

LOL if it's not an issue and zero gain..then why would anyone ever have to worry about it?

Either it's a problem or it's not.

 

Totally understand resource contention. Having applied the best practice with separate drives and also tested with a dual boot on the same hardware but with a build constrained to a single drive for OS and DCS it made no discernible difference.

The point is that in some applications like intensive database the setup of drives and optimisation of access is hugely important. With games like DCS much less so such that any gain is at best negligible.

If you have two SSDs then sure, it rules out any possibility of resource contention with the OS and DCS but the probability seems so incredibly low to make it a very poor return on investment if you were to add another SSD just for DCS.

You can be right that resource contention is a thing but in terms of advice to fellow players adding a dedicated SSD for DCS is poor value. A single SSD works well enough in my experience and others reporting on these forums.

[LucShep]

15 hours ago, The_Nephilim said:

Hey What would be a good thing to do with the Saved Games folder, put that on a 2nd SSD then the OS Drive?

Good point. And, well, it's again same principle as the game installation folder.
The files inside the DCS Saved Games folder can be considered also part of the game files, being somewhat frequently loaded from there too.  

So yes, if you already have a separate drive for DCS, and if so inclined, moving the Saved Games files of DCS away from the OS drive can be another good move.
If you want to experiment with it, you can do that by using symbolic links for folders (SimLinks). It works fine. 

Actually, I think ED should consider moving the Saved Games files of DCS into the game installation folder, altogether (just my opinion).
It can get quite big (especially with mods), and it would help those who are already using a separate drive (or even a separate partition) for the installation of DCS.

Edited February 26 by LucShep

[kksnowbear]

5 hours ago, LucShep said:

...it would help those who are already using a separate drive (or even a separate partition) for the installation of DCS.

Now, who in their right mind would bother spending money on separate drives? I mean, what with it being such a poor value, after all.

Crazy as it sounds to spend a whopping $50 on a system worth thousands, to help ensure technically superior performance...

...yet plenty have done exactly that. 

I wonder if they all know how bad an investment they made.

<SMH> Remarkable.

 

Edited February 26 by kksnowbear

[blkspade]

Direct Storage doesn't fix a supposed contention problem, it fixes a utilization problem. Which I mentioned SSDs being underutilized. Most software, and especially games have been relying on a legacy method of disk access, that just isn't optimized for the increasing potential of SSDs. Devs have been having to hand code ways to improve IO, but DS is providing an API to remove their need to do so. 

NVME was mainly a server targeted technology, where hundreds/thousands of people could be making requests of the system. No amount of mostly idle single user usage is putting that much load on one that it's making the difference in the gaming experience. The only time there would ever be contention from multiple processes is when they're moving large chunks of data in to memory. Most of what's in the background on any single user system is already working from memory, but will hit a couple CPU threads from time to time. SSDs in general are already so much better than HDDs because there is no seek time, the minute things hitting storage can take place so fast. I've only worked in IT/Systems admin for 20 years. I've upgraded a number of low core count devices from HDDs, just to have the CPU utilization go through the roof as Windows goes to maximize SSD throughput.

[kksnowbear]

Well, again, I can't imagine why people would spend a horrific $50 to incorporate technically superior performance, on a machine that only costs a few thousand. Makes no sense to me.

I wonder if all the people who did exactly that really understand how bad an idea it was.  I mean, after all, they could probably have gotten a much better GPU with that extra $50.

It really does make sense, of course, to have a machine that's top of the line in every other respect...and then choke the storage subsystem to death by only using a single drive, to save $50.

Everyone knows single drive systems are better.  No reason at all that technologies like RAID or SSDs should have ever existed in the first place, since there's no need to increase storage throughput.

Thank goodness we have experts here to set us straight, because otherwise an entire industry might be wasting its resources, pursuing better storage performance.

If technology like DirectStorage was appropriate to gaming, it would have been deployed on gaming consoles long before PCs.

Oh, wait...

(Oh, btw...for the record, I've spent over 40 years in actual professionally compensated roles doing computerized systems design, installation, and maintenance. Pretty much before there was such a thing as "IT" and well before Windows networks existed in any meaningful sense. If you ain't been around long enough to have worked on magnetic core storage, I suspect you're not gonna come up with anything I haven't seen or done already)

Edited February 27 by kksnowbear

[blkspade]

A RAID volume is still a single resource, so that doesn't actually promote your point. It's literally centralizing storage intended to be accessed by many clients simultaneously, along with potentially providing redundancy. All those drives are operating as one, not being dedicated to different tasks. SSDs have boosted the throughput well beyond most single user needs. A single gen 3 NVME has the equivalent throughput of like a 12-16 HDD array, but with near instantaneous access time. The only tangible thing you're getting with the $50 is additional storage.

The Direct Storage console point still does nothing for you, as those are still single drive systems. You are conflating the idea of contention with optimization. A single NVME has untapped potential, which is what Direct Storage is designed to unlock. Every NVME is marketed on it's sequential performance, but it's practically a lie because neither the data nor the software has typical access patterns that work that way. The random access performance is the real performance, because most software is written with HDDs in mind as its the lowest common denominator. Microsoft is changing that with Direct Storage for games and making it so being Windows certified in the future will require a SSD for basically all other software.

[kksnowbear]

5 hours ago, blkspade said:

A RAID volume is still a single resource...

Ummm..no.  Sorry, but that's completely misleading.

I do understand you're trying hard to say all this stuff like you understand, but TBH every time you post, it becomes more clear that you really don't understand much at all about storage.

RAID volumes were in use before solid state storage came around on any scale, and were among the early steps at trying to improve storage availability (note I don't say speed, but more on that later).  I only mentioned RAID to illustrate that there are other technologies that have come about for the same purpose (whether in part or in whole).

It's still a perfectly valid example, however:  While redundancy is part of the benefit of RAID, many people (myself included) built RAID arrays for performance (which I was probably doing before your 20 years in IT even started).  In fact, one of the most basic forms of RAID is intended strictly for performance, and has nothing at all to do with redundancy.

And your 'single resource' assertion is misguided.

In storage subsystems, availability and contention have an inversely proportional relationship.  An increase in availability is also a proportional decrease in contention. (It is clear you don't know this already, because if you did, you'd never make the comment about a 'single resource'.)

So, the fact that it's 'seen' as a single drive by the host system has nothing to do with contention.  RAID arrays, when configured for performance, factually reduce the time it takes for data requests to be satisfied (in proportion to the number of drives used).

And the absolute, brutal fact is, by using more drives, we decrease the amount of time it takes the "single resource" to satisfy any request.

So, back to the relationship between availability and contention:

Faster processing of requests = more availability = less contention.  Period.

In lay terms, it's like a factory, where 4 people can produce the same work as 1, in 1/4 the time.

People are still building RAID arrays today, with even very fast NVMe drives, even for gaming - because they understand the inversely proportional relationship between availability and contention.

Simple as that.

And yes DirectStorage absolutey *does* mitigate contention, for exactly the same reason as any other storage performance improvement does:  Sooner we can get the requested data out the door, the less time we are unavailable to other requests.  More availability = less contention.  (Besides that, all I implied was it wouldn't be used in gaming consoles if it weren't appropriate to games.  It is used in consoles, because it is appropriate to improving gaming performance.  Again, simple.)

So I'm afraid you're wrong on that point as well.

And those are just the facts of how storage works.  More you argue, the more you're just...well, I think it's obvious.

Edited February 28 by kksnowbear

[kksnowbear]

On 2/27/2024 at 1:04 PM, blkspade said:

NVME was mainly a server targeted technology, where hundreds/thousands of people could be making requests of the system.

Also, I'm gonna call "BS" on this comment, too.

This podcast ( >LINK< ) features Amber Huffman, a Principal Engineer and Storage Architect at Intel, and one of the two designers of NVMe.  Amber was also involved in SATA development before NVMe, and is therefore unquestionably the world's foremost expert on NVMe as it evolved after SATA.

She states specifically it was designed in order to take advantage of continued speed improvements in non-volatile memory devices (SSDs) - meaning it didn't have anything to do with multiple users or servers particularly as you claim.  It came about basically because SATA couldn't keep up anymore.

Further, she explains NVMe was developed as a replacement for SATA at the client side (that means not servers).

"NVMe and PCIe SSDs are intended to replace (or be the next generation) for client devices"

The intro for the podcast states outright: On the client side PCIe/NVMe SSDs will eventually replace SATA drives and on the enterprise side will be a complementary technology for SAS.  Again, replacement in clients, not servers.  Usage in servers, but not targeted as a replacement (as it was with desktop).

Amber goes on to say that, while there are uses for NVMe in enterprise (i.e. servers), that there will also continue to be technologies like Serial Attached SCSI (SAS) drives - meaning NVMe is not intended to replace server storage.

It wasn't 'mainly a server targeted technology" as you stated.  It has use in both areas, but was actually intended as a replacement in desktops, not servers.

And that's directly from the person who invented NVMe.

So, unless you're prepared to contradict the words of an absolutely unimpugnable source, I'd say your information was inaccurate.

It happens I've been following Amber's NVMe work since the early days, so I'm fairly familiar with the entire evolution of that technology and what it was designed and intended to do.

I'm not sure what your 20 years of IT experience involved, it certainly was not as a Storage Architect, and apparently did not involve actually learning how storage subsystems work.

Seriously, as much as I'm enjoying the chance to educate people who actually want to learn by reading this, I'd like to offer a friendly recommendation that you study more in this area if you're going to represent yourself as an expert.  It really doesn't help anyone when you spread information that is clearly at odds with the facts according to genuine experts.

You might consider there are people out here, even on flightsim forums, that have industry experience and backgrounds well beyond your own.  Just sayin' (I trust you're smart enough to do the math)

Edited February 28 by kksnowbear

[Art-J]

^ It's all very interesting in its own right from technology development point of view (and I do mean it), but also getting quite irrelevant to topic originally discussed. Moreover it's certainly not a valid reason for this D-waving contest of who allegedly worked in IT on what and for how many years. What's the use of these 40 years if they result in superiority complex mixed with being stuck with single obsession while conveniently ignoring any different aspects and opinions? "Enjoying the chance to educate people"? Yeah.... rrrrright. 

Could you just please just not look at it from IT geek/nerd perspective for a moment and look at it from average flight simmer perspective for a change? At least if you don't mind getting off that high horse of yours.

For average DCS player framerate IS be-all/end-all performance indicator, followed by stuttering or lack thereof and ending with mission load times maybe. Mind you, by "average player" I mean anyone who just fires up DCS client on his machine to do some short SP or MP flying, WITHOUT runnig dedicated server in the background, or live-streaming, or video processing, or crypto mining, or whatever else that would make storage solution (and the rest of his PC for that matter) more busy than usual.

I know from my own experience that switching from two separate system & DCS drives to a single one didn't affect performance understood as above in any practical way. Framerate didn't change obviously, but neither did the other two factors, at least I couldn't see it. Who knows, maybe even got "improved" (so to speak) performance on average as that single SSD of a different brand I bought was theoretically somewhat faster than two smaller ones? I didn't know and didn't care, 'cause I didn't notice stuttering and/or load time changes one way or the other, and thus I never bothered to do any benchmarking. Not to say there were no changes - I suspect  there probably was something out there, but anything single-digit-percentage is not worth attention. For the same reason buying two new bigger SSDs just to separate DCS from the system didn't make any sense either. It doesn't really matter if a disk costs 50 bucks or 50 cents, why should I throw money away? I tend to choose and buy my PC components as tools for the basic job, trimming off unnecessary glitter and gimmicks.

Look, I understand that your're super passionate about storage solutions but whatever you mean by "technically superior" solution and performance, even if very important in previous professional applications you have experience with, here in DCS doesn't matter all that much and thus - well, nobody cares. Unless the person is in a small group of experts and enthusiasts who are crazy about minute details. It's almost like GPU geeks arguing over which new card is better when one card pulls 140 fps and the other pulls 150. C'mon, really? I get it, for them it must be life-or-death confrontation, but normal user is going to be like "Ummm... Yeah, whatever, I need 60".

For average guy and not a tech nerd, alternative storage solution must be "practically" superior, not just "technically". Like in real gaming application, not in benchmarks or professional non-gaming applications. Like in not-so-old days of switching from HDD to SSD when even folks who knew nothing about PC/console internals noticed THAT performance increase immediately. Switching from combined OS/game SSD to two separate ones is just not comparable magnitude of change.

In either case, OP specifically wrote in his first post that he does have his system and root DCS folder on separate drives, just like you strongly recommend, so what is this storm in a teacup about anyway?

[kksnowbear]

1 hour ago, Art-J said:

^ It's all very interesting in its own right from technology development point of view (and I do mean it), but also getting quite irrelevant to topic originally discussed. Moreover it's certainly not a valid reason for this D-waving contest of who allegedly worked in IT on what and for how many years. What's the use of these 40 years if they result in superiority complex mixed with being stuck with single obsession while conveniently ignoring any different aspects and opinions? "Enjoying the chance to educate people"? Yeah.... rrrrright. 

Could you just please just not look at it from IT geek/nerd perspective for a moment and look at it from average flight simmer perspective for a change? At least if you don't mind getting off that high horse of yours.

For average DCS player framerate IS be-all/end-all performance indicator, followed by stuttering or lack thereof and ending with mission load times maybe. Mind you, by "average player" I mean anyone who just fires up DCS client on his machine to do some short SP or MP flying, WITHOUT runnig dedicated server in the background, or live-streaming, or video processing, or crypto mining, or whatever else that would make storage solution (and the rest of his PC for that matter) more busy than usual.

I know from my own experience that switching from two separate system & DCS drives to a single one didn't affect performance understood as above in any practical way. Framerate didn't change obviously, but neither did the other two factors, at least I couldn't see it. Who knows, maybe even got "improved" (so to speak) performance on average as that single SSD of a different brand I bought was theoretically somewhat faster than two smaller ones? I didn't know and didn't care, 'cause I didn't notice stuttering and/or load time changes one way or the other, and thus I never bothered to do any benchmarking. Not to say there were no changes - I suspect  there probably was something out there, but anything single-digit-percentage is not worth attention. For the same reason buying two new bigger SSDs just to separate DCS from the system didn't make any sense either. It doesn't really matter if a disk costs 50 bucks or 50 cents, why should I throw money away? I tend to choose and buy my PC components as tools for the basic job, trimming off unnecessary glitter and gimmicks.

Look, I understand that your're super passionate about storage solutions but whatever you mean by "technically superior" solution and performance, even if very important in previous professional applications you have experience with, here in DCS doesn't matter all that much and thus - well, nobody cares. Unless the person is in a small group of experts and enthusiasts who are crazy about minute details. It's almost like GPU geeks arguing over which new card is better when one card pulls 140 fps and the other pulls 150. C'mon, really? I get it, for them it must be life-or-death confrontation, but normal user is going to be like "Ummm... Yeah, whatever, I need 60".

For average guy and not a tech nerd, alternative storage solution must be "practically" superior, not just "technically". Like in real gaming application, not in benchmarks or professional non-gaming applications. Like in not-so-old days of switching from HDD to SSD when even folks who knew nothing about PC/console internals noticed THAT performance increase immediately. Switching from combined OS/game SSD to two separate ones is just not comparable magnitude of change.

In either case, OP specifically wrote in his first post that he does have his system and root DCS folder on separate drives, just like you strongly recommend, so what is this storm in a teacup about anyway?

You see it as "D waving".  I see it as an important need to promote fact according to empirical data and genuine experts - not BS and personal opinion.  My references to experience in the field are there to demonstrate that there are different levels of expertise and different levels of 'experts' **

Moreover, it is entirely possible (likely) that the 'average DCS user' causes themselves problems by doing a lot of this stuff wrong.  All you have to do is look at the forums just about any time to know this.  And the reality is that most of the 'experts' that show up - while they might know more than the 'average DCS user' - are not really experts.  They look smarter because they know more compared to the other guy.  Oldest trick in a military inspection...you wanna look better, stand next to someone who looks worse.  Fall in between two guys who look worse and you're golden.

As I said above, it does no one any good to spread BS.  It hurts the sim, the community, and the users.  Whether you notice something in single digits is a matter of your own opinion, and it doesn't give anyone the right to mislead others.  Just because you don't notice or don't care doesn't mean no one else does - that's up to the reader, not you.

I'm posting facts, with references (like the Podcast link above)...so if you're interested in learning from *real* experts (as I have) then I'm offering the resources, that's it.  If you're not interested, then don't read - simple.

Others are just getting all pissed off because the facts don't agree with them - plain and simple.

BTW "Technically superior" wasn't my phrase.  It was taken from LucShep's post earlier in this thread.  Apparently at least he understands and cares at least enough to know the facts.

The "storm in a teacup" is because (again) it matters when people spread misleading information about how things work.  If someone wants to wallow in ignorance, that's certainly their prerogative.  But they're not entitled to force everyone else to, by squelching detailed, factual information just because it steps on some toes.  I'm not hurting anyone by sticking up for fact here - and in reality, it helps those who actually want to learn.

** Edit:  Incidentally, a good part of my more recent experience is with building custom gaming machines for several people right here on this forum.  They trust my advice (and spend their money) where they can see real expertise.  So it's not just "d waving" online.  I'm out here, making it happen - in reality - every day.  If you choose not to listen, that's totally up to you.  But don't dismiss those people who have trusted me as fools.  I'm gonna go ahead and bet right now they'd beg to differ - and I know for a fact they are 100% satisfied with my actual, real work.  On a PC.  Running DCS.  I have two DCS user builds in the shop right now, along with several others for other sims...so someone's getting it done.

I don't know what it is....but a guy builds himself a PC to run DCS one time (or even a few)...he's a freakin' expert.  If he's got enough money to buy a 4090, he's a genius LOL  But I can assure you, for every one of those times that someone actually managed to make it work, I have a client walking in my shop, willing to pay me after they did it themselves (or bought a pre-built for that matter).  If I had a dime for every time I've had to work to undo mistakes and misconceptions people had from reading on the forums and places like Reddit....wow.

There are a ton of opinions on the internet.  How many of them are backing it up, every day, with actual experience and production?

Edited February 28 by kksnowbear

[blkspade]

U.2 is NVME on a different connector currently used for servers. The increase in IOPS is what promotes muti-client throughput, which is also something you get from RAID. The benefits of these technologies just flows from one space to another. It shouldn't need to be explicitly stated for you to gather.

[kksnowbear]

4 minutes ago, blkspade said:

U.2 is NVME on a different connector currently used for servers. The increase in IOPS is what promotes muti-client throughput, which is also something you get from RAID. The benefits of these technologies just flows from one space to another. It shouldn't need to be explicitly stated for you to gather.

Yup, but the actual expert made it clear: NVMe was *not* designed or intended to replace storage technologies in servers at the time it was invented (by her).  Pretty much opposite what you said.

It was targeted at plain old desktops because SATA couldn't keep up.

Edited February 28 by kksnowbear

[blkspade]

Keep up with what exactly? If we remove gaming, as the bulk of PC space isn't that, most end users don't do things that actually benefit from speeds faster than SATA. The random QD1-1T performance of most NVME SSDs is still below theoretical max of SATA. Queue depth 32 1 thread gets slightly above SATA capability (+150-250MBps). That is representative of how most client software behaves and has so for as long as SSDs have been an option. For gaming, Direct Storage comparisons exists and even they barely show a difference between NVME and SATA SSDs. So what she is describing (in 2012) could only be for extremely highly demanding situations or edge cases, and not the market as a whole. Not everything engineers gets excited about is as big as they make it sound at the ground level, because it takes forever for software to catch up. You didn't grasp context of her statement, and ignored that everyone else speaking were enterprise pros. SATA wasn't enough for the potential of NAND flash.

See you clearly looked up this clip in an effort to prove me wrong. I actually understand what's being said. They are talking about high parallelism in 2012 when the highest end client CPUs were quad cores. NVME wasn't even really an option in client computers until 2015 (which still topped out at 4c/8t CPUs for consumers). There were some M.2 PCIe based SSDs that predate the NVME protocol. I had a Z97 board from 2014 with M.2 slot that wouldn't even know what to do with NVME. See you'd want to grow interest in things like this in the consumer space because it leads to mass production and cheaper prices. So many low-mid range end-user OEM computers  between the inception of NVME until maybe 2 years ago still had HDDs as standard. You upgrade any one of those with less than 12 threads to an NVME SSD on a clean clean install, the CPU is pegged at 80-100% just setting it up and running Windows updates. So that argument doesn't add up with anyone that's been hands on with thousand of client computers. NVME drive costs only really started coming down when the real lowest common denominator needed them, Consoles. The PS5 having a user accessible M.2 slot has been the best thing for PC users with high-end desires. So late 2020 created the use case for the drives to start becoming more cost effective to be considered in lower end computers, which will have the knock on effect of more client software being written to properly address them. Yeah I guess your 2012 clip makes all the difference.

[kksnowbear]

You said:

NVME was mainly a server targeted technology, where hundreds/thousands of people could be making requests of the system. (Direct quote)

The industry expert who invented it says otherwise:

NVMe and PCIe SSDs are intended to replace (or be the next generation) for client devices (Also a direct quote)

(Those other people in that PodCast didn't create NVMe.  She did, and those are her exact words.  Since she actually invented it, I'm going to say she's the expert on why, not them...but thanks for the subterfuge.)

Those were exact quotes from you and her.  I didn't put words in anyone's mouth, nor make them say anything.

So you're not arguing with me, you're contradicting the very person who invented the technology.  Pretty straightforward.  Doesn't take all that rambling on about queue depth and whatever to see that.

And yet, you're still arguing.

Wow.

As I said, I've followed this work since the beginning, and I'm very familiar with it.  I actually learned about this stuff back then; I didn't just look anything up now.  I linked the article to show the facts, not to prove you wrong.

You made the statements that actually proved you wrong.  Don't blame me.

You've learned to use a bunch of really technical sounding terms, but your original comments show your actual level of understanding.  Your original statement about NVMe was incorrect, and your original comment about RAID performance was misguided as well, proving that you don't actually understand how availability and contention work in storage subsystems.

See, you're just piling on technical terms to obscure these few basic facts.  'Muddying up the water', as it were.  Years ago, I used to go to meetings where VPs and PMs sat at a table, tossing around technical terms just like that, when they really had zero idea how the stuff actually works - which they'd also eventually prove with their own statements.  Seems familiar in this discussion.

But hey, you keep on arguing.  Good for you

(Incidentally, to be clear: You're not by a million miles the only person to be "hands on with with thousand<sic> of client computers". That still doesn't prove you understand any of the storage concepts.  We had production guys making $8 an hour who were "hands on" with thousands of client machines, yet they were NOT in any way trained, skilled, or experienced professional IT people.  They were assemblers, that's it.)

Edited March 2 by kksnowbear

[blkspade]

My argument is that it clearly played out differently than what was claimed, as the execution was there to be witnessed. It obviously would benefit client eventually, but it continues to be far better than most can make use of. The first NVME drives were announced specifically for enterprise. That was both where they were most needed, applicable. Client CPUs were in no way ready to handle them at the time. You are quoting a person from Intel (somewhat out of context), and taking that at face value when evidence of what they did with it proves otherwise. Its literally actions speaking louder than words. Intel has said and done so many disingenuous things, that anyone paying attention to tech long enough would pick up on. It's not wise to ignore the actual actions of major corporations in general. That goes beyond our little debate.

I'm not using technical terms out of nowhere, as its about providing context. How could one with an understanding of tech knowing what benefits are provided and come to the conclusion that it is in reality a consumer oriented product? That "technical jargon" defines what benefit said tech provides in relation to exactly where it could be most immediately applied. Most bleeding edge high performance tech is in fact for enterprise first and filters down to consumer. How can her statement be true when no consumer class systems existed that needed or could even handle a single device with equivalent performance to a small array of HDDs? Not for like another 6 years after enterprise had been using them? Are you really not seeing the logical failing there? Even today the fastest consumer CPU (7950x) at general file decompression tops out at ~2.9GBps. I use that particular example because most client needs for grabbing data from storage aren't just dumping it to memory or a wire, but actually doing some level of work with it. Not everyone is sitting on a 16 core CPU. The first version of Direct Storage even still had the CPU doing the texture decompression. Just to go back around to NVME storage not really being the bottleneck in consumer applications. DCS is still handling IO with its own coding and not the DS API.

[kksnowbear]

Quote, per the inventor: It was invented for client.

You said it was invented for servers.  Opposite what the actual inventor said.  What part of you contradicted a genuine expert are we not getting?

And you're off and running again, with a bunch of stuff to distract from the fact that your stated opinion is wrong according to the industry expert who actually created the technology.

What it would benefit eventually has nothing to do with that fact.  Straw man.

5 hours ago, blkspade said:

How can her statement be true...

How can her statement be true...?  Seriously?  She invented the technology.  I'm gonna guess she must know what the heck she's talking about.  She was professionally compensated by one of the biggest tech firms on the planet to invent it...but we're supposed to believe you instead?  Wow.

Don't forget, she worked on SATA before NVMe.  So I'm pretty sure she would know where storage was coming from, and where it was going (that whole Principle Engineer/Storage Architect thing).

But you're right...why should we accept the words of the paid professional who invented the technology?  We have you to tell us what everything is really all about.

Wow. Just...wow.

And now we're adding in Intel being disingenuous?  Really obvious you're just muddying up the water now.

You keep bringing up a bunch of stuff that has nothing to do with the fact that the exact words of the actual inventor prove the opinion you stated is incorrect.

Your stated understanding of the reason NVMe was invented?  Incorrect.

Your statement concerning RAID improving availability and thus decreasing contention?  Misguided.

Dude, it can only be said so many ways.  All this other stuff you keep throwing into it has nothing to do with what you said being inaccurate, and really only proved your understanding of storage technology is limited to using a lot of terms in writing, and the same volume of 'hands on' experience that assembly workers also have.

Edited March 1 by kksnowbear

[blkspade]

I was making an effort to be concise but thorough. I wanted to give you the benefit of the doubt about you happening to misinterpret me somewhat abbreviating my points, as that wasn't exactly what I was saying about RAID. However your subtle (no so) attempt to just attack me seems to suggest you're just doing it on purpose. Amber Huffman wasn't exactly the sole entity that was developing NVME, while at Intel. It was a consortium of companies that had their hands in co-developing the spec. I don't think it's a secret that Intel's most important segment is data center. https://nvmexpress.org/why-nvm-express-in-3-minutes/ , https://unpacked.network/shows/storage-unpacked/206-nvme-2-point-0/ . Client is a part of the conversation for sure, every segment is, but the clues do point to Data Center being the primary driver. 

I was really having trouble grasping how you interpret that statement at that time as an absolute, when Data center is so clearly favored. Saying "The inventor said it", seems a bit of a cop out when those words are running counter to reality. Taking her words along with the actual execution. One could reasonably reconcile the two as her looking forward to the future. She is an inventor after all. In that moment in 2012 there were no client systems that would be able to take advantage of NVME, and there wouldn't be for many years to come. She would obviously have some ideas as to how consumer class tech would continue to evolve, being at the forefront of it. I could see how it could maybe be open to interpretation if you can ignore that it was used in enterprise first along with it's costs. It's 10 years after the 1st NVME devices shipped and we're finally getting client software solutions starting to tap into their true potential. Which still needs the latest top end client CPUs to do so, that move the least units at laughable lower margins than server chips. Fine I'll concede that this must be what it was all about. Data center was a total afterthought. 

You get your Gen 5 NVME drive yet? I haven't but I think I'll buy 2. Maybe throw in some Optanes for good measure. I'm gonna have all the frames. FPS=Yes!

[kksnowbear]

I posted direct, word for word quotes from her and you. Simple as that. You contradicted an undisputed expert. 

She invented NVMe with one other person, pretty sure that's been stated.  No one said she was the only person working on it - straw man (which would also be totally ludicrous to suggest).  She was the Principal Engineer (maybe you ought to look that up).  Plenty of inventors and scientists had teams but the name credited by history as the creator is the lead.

No one's questioned what Intel's most important segment is - again, straw man.  Seriously, stop trying to drag servers back into it just so it sounds like you weren't wrong.  You were.

Engineers invent stuff all the time regardless of whether there's an existing use.  Doesn't make it any less valid, and (again) she stated clearly what it was intended for: To replace SATA in clients. 

When it was invented, and whatever else it has use for, is immaterial.    This is just more subterfuge, attempting to obscure the bottom line.

She invented it, and she stated clearly why.  Per the actual industry expert (who stated exactly what I quoted), it was not invented for servers as you said.  Period.

Doesn't matter how much other stuff you stir up trying to obscure it, that remains the bottom line.

It is absolutely absurd that you continue trying to make it seem like she didn't know exactly what she was doing or why.  Intel paid her for her skill as a Storage Architect, which your posts make it obvious you are not.  Anyone with any sense can see who the expert clearly is. (But we're all supposed to listen to you.)

Your comment about RAID also showed you don't understand how RAID arrays work to improve availability, and thus reduce contention.

As I said, there are people today still building RAID arrays, from even the fastest NVMe drives, because using both technologies allows approaching the maximum in availability.  They understand that availability, as the inverse of contention, is key in storage.  They further understand that improvement in storage availability is critical to overall system responsiveness.  And, in gaming specifically, responsiveness is among the most important goals of better performance.

So: Increased storage availability factually equals decreased contention, which conclusively results in better system responsiveness - a crucial factor in gaming performance.

Anyone who doesn't believe in or understand the significance of responsiveness, need only ask a competitive gamer why responsiveness in systems is critical (or, you know...an actual fighter pilot).

Attacking you personally is of no importance to me and is not what's happening here.  I'm just presenting conclusive evidence that your opinions are inaccurate (and there's a history, at that.)  I get that you don't like your comments being proved wrong, but don't claim I'm attacking you personally.  It's your own stated opinions that I'm rebutting, and it's your own comments - directly quoted, word for word - that are proving you wrong.  Again, don't blame me.

I've already explained what matters here is calling BS what it is, and trying to present genuine fact, according to real experts, to help people who might actually want to learn (rather than just repeatedly acting as if they know more than the people who actually created the technologies being discussed.)

Edited March 1 by kksnowbear

[blkspade]

I get it now, its context that you really aren't grasping. You misinterpreted my point about RAID, but it was specifically in relation to your previous statements. In a shared resource, contention would be inevitable, but you'd ideally have built-in the headroom to mitigate it. Obviously demand could grow beyond the headroom. A bigger faster array obviously gives you more headroom, but an array while potentially giant is still a (typically) shared resource that could still eventually reach a point of contention. It's many drives, but the data is spanned across them, so they are effectively doing the same work simultaneously. The load is just distributed across them all. Each drive isn't handling a separate dedicated task, application, or user. RAID was a poor example for you to use in that context. I hope that clarifies that for you.

Which goes back to my point that a single decent NVME drive has so much headroom built-in, most casual single user computer use isn't going to cause the contention that would genuinely warrant expansion. The act of actually reasonably taxing just one, shifts the point of contention to the CPU. To go back to a PS5 example, (for context) it relies on heavy compression, so it has dedicated silicon separate from the CPU to handle decompression because its little Zen 2 based 8C/16t CPU isn't up to the task, in conjunction with other system responsibilities and running the game. That's according to Mark Cerny, Lead System Architect of the PS5. If you add-in a NVME it has to be Gen 4 capable of 5GBps. As you know those data center NVME arrays require lots of cores/threads just to shuttle the data around. Work to be done on/with that data is commonly on a completely separate set of servers/CPUs.

Most home/office computers are 8 cores or less, and not even the latest gen. So their CPUs are doing the shuttling and the actual work at the same time. A lot of them are still too slow to move or process data at speeds above what SATA offers. Any respectable gaming PC should, but only recent high end ones can push more than what a Gen 3 drive offers. A steam game file verification will use up to 8 of my 32 threads to only read the files at up to 630MBps, on a drive that benches 7000. This is obviously a built-in software limitation. At some point in the last 24hrs hwinfo on my PC recorded reads and writes at 4.3/5.4GBps respectively, but max CPU utilization of 70%. The disk throughput is likely from 1 of the 2 games I played in that period, and most don't push anywhere near that. The write I could probably attribute to a 6hr MP DCS track file. The game sessions alone obviously aren't the source of 22 threads worth of utilization though. Only because I wasn't playing Star Citizen. So that's most likely tied to the high write source too. My computer is overbuilt, but isn't just for gaming. That's just the most intensive use it had this day. There are still computers sold within the last 5 years that hit 100% CPU utilization just doing an Internet speed test. I won't say that most are that bad, but cheap computers are very ubiquitous.

[Mr_sukebe]

I’m with KKS on this, and use separate NVME drives for DCS and the OS/saved games.

Whilst the debate has been a bit heated, my take is that KKS is trying to prevent a misunderstanding that contention with the use of a single drive is irrelevant.

My opinion is that the potential for contention is increased with the use of a single drive.  That contention may be infrequent and hard to measure, but it’s still likely to exist.

The key point being that I believe it to incorrect to claim that it doesn’t exist, which is what I believe was the point that KKS has been trying to say.

For ref, over 25 years in IT

 

Edited March 3 by Mr_sukebe

[kksnowbear]

On 2/28/2024 at 11:59 AM, blkspade said:

A RAID volume is still a single resource, so that doesn't actually promote your point. It's literally centralizing storage intended to be accessed by many clients simultaneously, along with potentially providing redundancy. All those drives are operating as one, not being dedicated to different tasks.

Direct quote there, and misguided opinion.

It "promotes my point" exactly:  As I've repeatedly explained, RAID is one of many technologies developed to increase availability, thus decreasing resource contention.  Faster data requests are handled, sooner the storage subsystem is available to other requests and thus contention is reduced.  In this manner, the entire array is actually being devoted to different tasks (other requests), by way of increased availability.  Storage Architects understand availability is crucial.  Yet you're still attempting to argue it was a bad example.

It was a perfect example, if you understand the importance of responsiveness and availability in storage subsystems.  The fact that you keep arguing simply goes to show you really (really) don't understand this concept.

And plenty of single users have/had RAID arrays.  It was intended to improve responsiveness by increasing availability thus decreasing contention.  It's just not a matter of how many users, simply because contention will occur in any system - and, as I said in the very beginning of this, it doesn't matter how much or how often, when it happens (and it will), it's a bad thing.  The point behind *any* storage performance improvement - RAID, NVMe, whatever - it to improve responsiveness by increasing availability thus decreasing contention.

That's what you don't get.  But I urge you, by all means: Go ahead, ask a fighter pilot: Laying aside everything else, would you want a system that was more responsive or less responsive?  Ask a competitive gamer the same question.  I'm absolutely, 100% certain of the answers you'll get.

And yet you go off again, arguing a bunch of stuff no one is arguing.  It's just straw man.  If a system suffers bottlenecks elsewhere, that's a different issue with a different solution.  You just wanted a way to throw in some person's name and title to make it seem as if it reinforces your argument.  How many cores a PS5 has isn't even remotely at issue; this conversation is and has been about storage responsiveness in and of itself.  The point behind *any* storage performance improvement - including DirectStorage, even in a PS5 - it to improve responsiveness by increasing availability thus decreasing contention.

Obviously you need to stir up a lot of mud in the water to distract from the facts, but those are the bottom line:  You contradicted the industry's leading expert on NVMe, and you continue to prove your opinions in general regarding storage subsystems are misguided.  Everything else you're posting honestly is subterfuge, and straw man arguments which are not at issue, to distract from the obvious inaccuracies in your own statements.

Edited March 3 by kksnowbear

[kksnowbear]

3 hours ago, Mr_sukebe said:

I’m with KKS on this, and use separate NVME drives for DCS and the OS/saved games.

Whilst the debate has been a bit heated, my take is that KKS is trying to prevent a misunderstanding that contention with the use of a single drive is irrelevant.

My opinion is that the potential for contention is increased with the use of a single drive.  That contention may be infrequent and hard to measure, but it’s still likely to exist.

The key point being that I believe it to incorrect to claim that it doesn’t exist, which is what I believe was the point that KKS has been trying to say.

For ref, over 25 years in IT

Thank you, at least someone gets it...

I would take it a step further, however, and say that how the concept is expressed is crucially important.  You're absolutely right in that it does exist, of course.

That said, there will always be those who claim that, because it's infrequent or difficult to observe, it's not a problem.  That's the foundation for their entire argument.   Unfortunately, this gets turned into the very argument in this thread:  That it doesn't matter.

It absolutely does matter.

What matters to them is entirely their business, and not for me to say. But it is wrong and misleading to say it makes no difference at all, just because it doesn't matter to you.

My point is it makes zero sense to spend multiple thousands to build/buy high-end systems, another thousand or two on peripherals...and then 'build in' contention (thus factually decreasing system responsiveness), by refusing to invest another 1-2% into avoiding the contention that will occur (hard to observe or not).  That's like buying a million dollar sports car and burning cheap gas in it

And it also makes zero sense to carry on like the contention issue doesn't exist.  It's well known, and as I've explained (or tried to) it is a key factor in storage improvements of any type.

I'm just now wrapping up yet another DCS build for a member of the forum here, and I can tell you that even though there's a fairly constrained budget, it's a 'no brainer' to include separate storage for DCS.  And in this case, it's actually a much bigger percentage of the cost (~10%).  Here's a guy who gets it:  He doesn't even have $5000 to throw at a system ...yet he's not going to create a problem that can easily be avoided.

Something else I mentioned earlier that has been completely obscured by all the subterfuge is that the engineers who design the motherboards we use are fairly smart people.  They build the boards with multiple controllers/slots for storage, because even they understand that a properly implemented storage subsystem can use multiple drives to improve responsiveness by increasing availability thus decreasing contention.  They understand that (for people who can actually wrap their heads around it) this is a key part of optimal overall system design.

Of course, there will always be those who don't understand the concept, and will continue to argue that it doesn't matter, simply because they don't understand it.

I'll be honest, a lot of nuclear physics is lost on me because I don't understand it.  I do, however, have more sense than to argue with the direct statements of the folks who invented the technology.

Edited March 3 by kksnowbear