Blog

AI and HPC Workloads Demand Highest Performance Data – DDN is the Undisputed Leader on IO500

11x better performance. 7 of the top 10 scores. DDN leads the way for AI & HPC data platforms.

“Capturing 7 of 10 spots in the 10-Client Production list is more than a technical achievement – it’s proof that leading enterprises count on DDN to deliver the speed, scale, and reliability needed to turn data into competitive advantage”

Summary:

DDN dominates the IO500 benchmark, outperforming VAST and Weka by up to 11x across real-world AI and HPC workloads.
DDN delivers unmatched bandwidth, metadata handling, and efficiency – capturing 7 of the top 10 spots in the IO500 10-node production benchmark.
Leading enterprises achieve breakthrough results with DDN, from 70% faster fraud detection to 80% shorter genomics runtimes.

If you rely on GPU’s, AI workloads or HPC simulations, you need the best technology to achieve the best outcome. Whether your mission is curing diseases, protecting financial markets, or transforming entire industries with generative AI, your outcomes hinge entirely on Data Center and Cloud infrastructure performance. IO500 confirms what top innovators already know: DDN isn’t just leading the pack – we’ve left it behind entirely.

IO500 Proves It: DDN Leads and Delivers 3x to 11x More Value Than Other Data Platform Technologies

IO500 isn’t just another benchmark – it’s the gold standard for assessing real-world storage performance for AI and HPC. It is vital to assess which technologies you should deploy and which vendors you should rely on. Unlike synthetic benchmarks that only test hypothetical and isolated conditions which rarely exist in the real world, IO500 evaluates real data center GPU and Compute infrastructure under realistic, demanding conditions with mixed workloads, complex I/O patterns, metadata-intensive operations, and concurrent tasks – exactly what your business outcome and scientific real-world AI and HPC environments face daily. This comprehensive test provides clear, credible insights into how infrastructure will perform when it matters most.

In the vital 10-node production category, which is where most customer use cases are found, and where real-world AI workloads run under intense pressure, DDN isn’t just ahead – it’s redefining what “ahead” means. And that translates into significant value for you. Value in innovating, creating better products and services, and pushing the envelope of what is possible.

Why This Matters for Your AI and HPC Workloads

AI and HPC cannot tolerate slowdowns—every minute of wasted compute time drains both budget and innovation momentum. Competitors offer theory; DDN delivers measurable, transformative results. Topping the IO500 rankings proves DDN delivers exceptional end-to-end performance that translates into real business benefits:

Huge cost savings through hardware efficiency: A large U.S. hedge fund transitioned to DDN and saw a 3x improvement in algorithm development speeds, drastically reducing runtime costs and accelerating ROI, while also slashing fraud detection latency by 70%, saving millions in operational costs and significantly reducing financial risk.
Estimated impact: Increasing throughput from, say, 10 backtests per day to 30–35 backtests—saving hundreds of thousands annually in GPU and engineer time.
Simulation and data pipeline acceleration: DDN-powered genomics workflows—such as those at the Translational Genomics Research Institute (TGen)—have slashed pipeline times from 12 hours to under 2 hours, enabling researchers to iterate faster, increase productivity, and reduce compute costs by over 80%.
Boosting Sovereign AI: A national defense program sustained data flows of 2TB/s during simultaneous model training and data ingestion, boosting efficiency, slashing operational costs, and drastically improving time-to-insight.
Massively enhanced cluster efficiency: Supercomputing centers like CINECA (used for tsunami forecasting), Helmholtz Munich, and Bitdeer AI leverage DDN EXAScaler to maximize GPU utilization and drive unprecedented AI/HPC throughput —allowing customers to pack more workloads per cluster, lowering both CapEx and OpEx across the board.

These examples aren’t theoretical—they’re real, quantifiable impacts enabled exclusively by DDN

Here’s the Tech That Leaves Competitors Behind

DDN’s infrastructure isn’t retrofitted – it’s built specifically for AI and HPC workloads:

Blistering Multi-Terabyte Bandwidth: Proven in massive-scale environments like NVIDIA’s Selene and CINECA’s supercomputing clusters, enabling real-time training, inference, and simulation.
Unmatched Metadata Handling: Seamlessly handles billions of files and random I/O at scale, unlike competitors who stumble and delay your critical workflows.
Native AI Parallelism: Zero-compromise parallel data access designed specifically for GPU-intensive workloads – no shim layers, no workarounds, pure performance.
Intelligent Pipeline-Aware Tiering: Optimized caching ensures maximum GPU utilization, dramatically reducing latency and idle times.
Bulletproof Enterprise Resilience: Built-in redundancy and robust data protection trusted by industry leaders including NVIDIA, HPE, Dell, Lenovo, and others.

Why Competitors Fall Short

Here’s why other providers struggle:

WEKA requires ongoing manual tuning, leading to unpredictable performance, wasted time, and unnecessary operational expenses.
VAST handles some read-heavy scenarios adequately but severely lags in crucial metadata operations and write-heavy AI workloads, hindering performance and limiting scalability.
Hammerspace promotes orchestration but lacks significant deployments in high-pressure AI/HPC environments, as their single IO500 submission clearly demonstrates.

Your GPUs Deserve the Best—DDN

Every investment in AI infrastructure should maximize innovation and accelerate your goals, not bog you down in performance issues.

DDN ensures you achieve:

Maximum GPU Efficiency
Fastest Time from Data to Insight
Proven Global Leadership and Scalability

Don’t settle for outdated infrastructure. Demand better. Demand DDN.

Explore the full IO500 results or contact our team today to see how powerful your AI infrastructure can truly become.

Last Updated

Jun 25, 2025 8:44 AM

Blog

The AI Infrastructure Bottleneck No One Talks About: Why Smart Storage Is the Missing Layer

In our opinion, Gartner’s Hype Cycle™ for Storage Technologies, 2025 (subscription required) offers critical insights into which storage innovations are poised to transform enterprise IT strategies. According to Gartner®: “this research examines the influence of emerging storage technologies on infrastructure outcomes, as well as their adoption rates and maturity levels. With these insights, I&O leaders can develop strategies to deliver innovative and future-proof storage platforms that meet business requirements.”

This year, DDN is named a Sample Vendor in two of the most forward-looking categories we believe:

Storage Platforms for Generative AI
Open-Source Storage Software

To us, these recognitions are more than accolades — they underscore DDN’s role as a foundational force in the future of enterprise AI and data platforms.

But First: Why AI Initiatives Are Stalling—And What’s Really Missing

For many teams racing to build production AI systems, the signs of strain are all too familiar:

GPUs sitting idle while waiting for data
Pipelines breaking across environments
Massive unstructured datasets too slow to be usable
Cold archives that GenAI needs – but can’t access in time

From the outside, these look like GPU scaling problems. In reality, they’re data layer failures.

Storage – specifically, smart, AI-optimized storage – isn’t just a performance layer. It’s the foundation that determines whether your AI systems can actually scale, learn, and respond in real time.

And in most organizations, it hasn’t caught up.

What the New AI Data Layer Needs to Do

The storage architectures of the past weren’t built for what AI demands now: Fast, frequent, intelligent access to vast volumes of data across distributed environments – at the exact moment it’s needed.

In our opinion, that’s why Gartner’s Hype Cycle for Storage Technologies, 2025 recognizes Storage Platforms for Generative AI and Open-Source Storage Platforms as innovation triggers. And it’s why we believe the AI Data Layer is now the most important missing piece of modern infrastructure.

To work at scale, it must:

Understand how data moves across training, tuning, and inference
Adapt automatically to changes in workload, priority, and location
Serve up cold or archived data as if it were hot – especially for RAG and LLMs
Span edge, core, and cloud with consistent performance and intelligence

This is not what traditional storage delivers. It’s what intelligent, self-optimizing platforms like DDN’s do.

Storage Platforms for GenAI: Real-Time AI, Real Business Outcomes

Generative AI (GenAI) workloads – including retrieval-augmented generation (RAG), fine-tuning, inference, and large model training – are reshaping enterprise compute needs. These applications demand not just powerful GPUs, but storage systems capable of feeding them massive volumes of data with minimal latency.

Gartner defines Storage Platforms for GenAI as solutions optimized for performance and data management to support AI workflows efficiently – without requiring separate infrastructure for GenAI vs. traditional enterprise workloads.

We believe, DDN’s recognition as a Sample Vendor in this space validates its position as the AI data platform of choice. With solutions like DDN Infinia and EXAScaler®, enterprises benefit from:

Blistering performance at scale to saturate GPU pipelines
Concurrent workload execution – training, inference, analytics, and ETL
Native support for RAG and vector-aware architectures, reducing stale responses and data duplication

The business impact is real: customers using DDN have cut fraud detection times by 70%, tripled AI throughput, and slashed model training costs through higher GPU utilization and zero-copy data access. Simply put, DDN enables AI infrastructure that is faster, leaner, and smarter.

Open-Source Storage Software: Agility Meets Performance

Open-source storage software is gaining momentum as enterprises look for greater flexibility, cost efficiency, and integration into DevOps workflows. We conclude that Gartner highlights this category for its potential to drive innovation in multicloud, containerized, and performance-intensive environments.

DDN’s leadership in this space is built on decades of HPC and AI expertise, most notably through its active role in the Lustre community. As a top contributor and maintainer of Lustre – the world’s most performant parallel file system – DDN enables:

Open, programmable access to extreme-scale storage
Optimizations for AI and HPC workflows with proven performance in production
Flexibility to run across bare metal, containers, hybrid, and cloud environments

Organizations building modern data platforms – from sovereign AI initiatives to national labs and private enterprises – rely on DDN’s open-source contributions to balance cost control with cutting-edge performance.

The Bottom Line: DDN Is the Infrastructure Standard for the AI Era

We believe, being listed by Gartner in both Storage Platforms for GenAI and Open-Source Storage Software is a clear signal that DDN stands at the intersection of AI acceleration and infrastructure flexibility.

For enterprises seeking to move faster, scale smarter, and drive value from data, DDN delivers the storage foundation that transforms innovation into outcome – with the openness, performance, and vision to lead the next era of intelligent infrastructure.

Let’s talk about your AI infrastructure strategy

Connect with a DDN AI expert

Gartner, Hype Cycle for Storage Technologies, 2025, 19 June 2025

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Last Updated

Jun 30, 2025 9:22 AM

Blog

Unlocking AI Infrastructure Efficiency: Rethinking Storage Strategies

Rethinking your storage architecture might just be the key to efficiently unlocking the full potential of your accelerated computing.

How do I reduce the number of watts consumed for a given productive output? How do I make these very large, very critical production AI systems robust? Answering these questions is our mission at DDN and I think about them a lot.

At the end of the day, an AI system comprises compute, network, middleware (containers, schedulers, monitoring), AI frameworks, and storage. These components are all part of the answer regarding system efficiency.

Overcoming AI Storage Bottlenecks for Enhanced Productivity

This is why it is frustrating that shared storage within AI Infrastructure remains an often overlooked element of the stack. We come across this a lot, especially with customers starting from a smaller scale. But the important points that people often miss are:

The right storage simplifies everything and makes a system more robust
Storage can vastly improve the efficiency of the entire system

Ultimately, purchasing optimized storage for AI workloads might be a bit more costly, but if you get it right, you gain 2X that back in higher productivity from the whole infrastructure.

Streamlining Data Movement in AI Infrastructure

How does that work? Well, moving data is, of course, an inherent part of AI model training (and inference for that matter). This includes moving datasets into GPUs, moving models into GPU memory, and saving hyperparameters to storage (checkpoints).

All of this data movement is overhead. It can be compared to waiting at the checkout at a supermarket. Standing there waiting adds no value and is just a step that needs to be completed before moving forward. If we could reduce our checkout wait time to zero, supermarkets would be more efficient, shoppers would be happier, and nobody would lose.

Just like a long checkout line can delay dinner, waiting for data can seriously stall AI projects. A high percentage of AI infrastructure is wasted as it burns up capital value along with electricity, cooling, and precious time.

Organizations are trying myriad approaches to optimize the training of foundation models, often by reducing the size of the computed problem without impacting the efficacy. Popular techniques include reducing the parameter search space for training, moving to asynchronous data movements, reducing model size, reducing effective size of training data, etc.

Racing Towards Efficiency: The Need for Balanced Infrastructure

While we’ve seen these kinds of optimizations before in HPC, they are never able to completely negate the need for a balanced hardware and software architecture. To maximize efficiency, the system’s GPU performance, CPU performance and memory bandwidth, PCI architecture, network bandwidth and latency, filesystem performance and storage capabilities all need to be balanced against the workload. And that efficiency is measured in reduced power, greater capital returns, less operational spending and better use of people’s time.

I’ll also note that things are not getting easier. With the huge race now for building the next generation of foundation models, and even moving towards AGI, the problem sizes are growing faster than the optimizations can reduce them. On top of that, GPU power, memory footprint and memory bandwidth are rapidly increasing, with NVIDIA doing an astounding job of maintaining momentum in hardware and software advances. The company is not showing any signs of slowing down.

From Jensen’s recent GTC keynote, I noted the following as key for us at DDN, being a contributing part of part of the NVIDIA-driven AI Infrastructure:

“And that’s our goal—to continuously drive down the cost and energy associated with computing so that we can continue to expand and scale up the computation that we have to do to train the next generation of models.”

It was about efficiency five years ago when NVIDIA bought this first DDN solution to drive the data for its Selene supercomputer. It was the same one year ago when, again, NVIDIA turned to a DDN solution to drive the data for their Eos system, and it is going to be the same for future systems when driving down cost and energy while driving up productive output is the target.

From Hyperscalers to End Users: Prioritizing Efficiency

And it’s not just the hyperscalers who are seriously thinking about the efficiency question. The end-users are too. An interesting post from a couple of weeks ago highlighted this in the light of these new Cloud implementations that are helping organizations gain access to large scale AI infrastructure quickly.

Yi Tay, Co-founder & Chief Scientist, Reka (Past: Senior Research Scientist, Google Brain) wrote a blog that I thought very important for the new wave of GPU cloud providers out there. Tay’s post highlights the challenges and variability faced when acquiring compute resources from different providers for training LLMs.

Tay notes that even clusters with stable nodes are not immune to efficiency problems:

“Meanwhile, even though some other clusters could have significantly more stable nodes, they might suffer from poor I/O and file system that even saving checkpoints could result in timeouts or ungodly amounts of time chipping away at cluster utilisation.”

“Did I mention you’ll also get a different Model Flop Utilisation (MFU) for different clusters!? This was a non-negligible amount of compute wasted if one is unlucky enough to find a provider with badly cabled nodes or some other issues. Systems with very sub-optimal file systems would have the MFU of training runs tank the moment a teammate starts transferring large amounts of data across clusters.”

(Source)

Tay’s insights show that end users are definitely noticing the overall efficiency and output corresponding to their time and spend on cloud resources. For their GPU-Hour dollars, they notice the useful productive output, and they are comparing between the good, bad, and ugly out there.

Balanced architectures and integrated solutions are important to efficiency. NVIDIA has talked about accelerated computing being more than the sum of its parts, but rather emergent from cross-stack integration as well as the speed of the individual components.

DDN’s Integral Role in Streamlining AI Architecture

At DDN, our storage is not just storage. It is an integral piece of your AI architecture that simplifies and accelerates while keeping your data safe. Our A³I storage solutions are purpose-built for AI workloads, delivering unmatched performance, comprehensive enterprise features, and limitless scaling.

These offerings are just the beginning. DDN provides a comprehensive suite of products and solutions to streamline your entire AI data lifecycle. From our EXAScaler parallel file system for high-performance workloads to our DataFlow software for seamless data movement and management across diverse environments, our solutions enable you to harness the full potential of your data.

Explore our online TCO tool to see the tangible impact of DDN solutions on your bottom line and discover how our intelligent data solutions can transform your accelerated computing architecture.

Last Updated

Oct 2, 2024 2:43 AM

Blog

Harnessing Accelerated Computing for AI & LLMs with DDN Storage

Whether you are on the business side of the house or deeply entrenched in IT, there’s a good chance you were at least intrigued, if not taken aback, by the introduction of ChatGPT a little over a year ago.

Yeah, we knew that AI was rapidly evolving and that something like this was coming. But when we actually heard about – or directly witnessed – the types of queries to which it could respond, the amount of data it was able to gather and the quality of the information presented (very good, though not always 100% cogent) it was a remarkable milestone. What astonished most of us was the speed with which ChatGPT, powered by accelerated computing, generated results.

At 8:00 AM you ask an AI system to produce a report that predicts the quantitative economic impact of AI on each of the G20 nations by the year 2030. At 8:15 AM you receive an intelligently written, grammatically correct, well-researched, 10-page report with detailed analysis, charts and an annotated bibliography. You can’t help but be impressed – and maybe a little worried. More so when you realize this effort might have taken a small team of economists and data scientists days or weeks to pull together. Of course, accuracy is important…we know that part has to get better, and it certainly will. Extensive training is still needed to realize the promise of many AI systems, regardless of their output.

But speed is really the name of this AI game, isn’t it? Everything is about accelerated results. Faster ROI on AI investments and shorter time to market for AI-enabled products are just part of the pay-off in operational speed.

How Accelerated Computing Improves Data Center EfficiencyLearn more

The Power of GPUs: Elevating AI & Reducing Costs

The connection between speed (let’s say accelerated computing) and operational efficiency is indisputable. It’s not just the obvious things like how fast projects get completed or products get to market when AI is used. It’s also the impact of AI on data center efficiency… we’re talking about savings on things like hardware, energy, floor space and operating costs. It may not be intuitive, but getting AI systems to run faster actually makes data centers more energy- and cost-efficient – on-premises and in the cloud.

Again, whether you’re on the business side or the IT side, you’ve become familiar with companies like NVIDIA, the power of GPUs and their crucial role in taking AI mainstream. You may have also heard NVIDIA’s CEO, Jensen Huang, state that “Accelerated computing is the best way to do more with less”.

What he’s getting at is the fact that GPU-based systems can accelerate processing by up to 50X compared to CPU-based systems. The implication is that tens of thousands of CPU-based servers can now be replaced by a few hundred GPU-based systems. Given that about 55 percent of data center energy today is used to power hardware systems, such as servers and storage, and over 40 percent is used to cool these resources, such a reduction in compute resources would slash data center energy costs and space requirements considerably. Additionally, while storage is typically a small fraction of the overall power requirement – a storage system’s capabilities can have a huge knock-on effect.

AI Data Management Accelerated with DDN A³I & EXAScaler

AI Infrastructure Architects and IT teams more broadly are also beginning to realize the importance of AI storage in enabling accelerated computing. They understand that GPU-based systems run faster when paired with parallel file systems that offer super-fast data ingest and can fully saturate GPUs to maximize both AI performance and resource utilization.

This type of parallel data management architecture, as seen in DDN’s A³I appliances, enables efficient LLM offloading from GPUs and delivers throughput that greatly outperforms NFS and other enterprise storage solutions. DDN’s optimized IO processes can accelerate GPU performance by 10X versus our closest competitors.

In fact, DDN systems deliver the fastest and most responsive small IO, random IO and metadata performance in the market – all critical for AI models. At the same time, the architecture can scale performance linearly to hundreds of petabytes to support the latest LLMs that might juggle billions of parameters. The underlying EXAScaler parallel file system uniquely addresses the explosive demand for performance-hungry GPU- and DPU-based environments, processing AI data from diverse sources ranging from numerical models to high-resolution sensors.

The reality is that DDN powers more AI systems, representing more than 250,000 GPUs, across more markets than any other storage vendor in the world and is instrumental in improving data center efficiency via accelerated GPU performance.

Optimizing AI Performance: Power and Storage Improvements

So far we’ve talked about how DDN storage helps GPUs run faster. We should note that DDN technologies help accelerate all layers of the AI stack, networks and of course, filesystems and storage media – all of which have an impact on data center efficiency.

Let’s step down a notch and talk about some of the energy- and space-saving benefits that DDN storage delivers directly for Generative AI and LLM environments. This includes accelerated training for the largest and most complex LLM frameworks in the world, enabling transformer models like GPT, Bert, and Megatron LM.

DDN’s EXAScaler parallel file system can drive 10X to 15X more data per watt, delivering outstanding results using only a fraction of the power and rack space of conventional storage systems.
Machine learning is both read- AND write-intensive. DDN delivers 30X faster random read and write throughput than our competitors. EXAScaler systems deliver the best IOPS and throughput in the industry per rack – up to 70M IOPS and 1.8T/s and 1.4TB/s of read and write throughout, respectively.
We offer the best compression and data reduction performance. And because data is compressed directly from the application, less data moves over the wire for reads and writes.
Our systems reduce storage wait times for data loads by 4X.
Thousands of checkpoints are required for AI. Due to far superior write speed, ours run 15X faster, slashing training cycle run time.
DDN’s Hot Nodes feature automatically caches data sets on internal NVMe devices. This greatly reduced network and storage load during multi-epoch AI training.
These efficiencies reduce data center runtime by 5% to 12%, delivering faster training and higher productivity to LLMs and Generative AI.
Essentially, for every $1 you spend on DDN storage, you can recoup $2 in infrastructure productivity.

Elevate Your AI Journey with DDN

So, you now have a little more information about the connections from storage to accelerated computing to data center efficiency; a relationship which will become increasingly evident and stronger as LLMs get even larger, and AI becomes more ubiquitous.

Yes, the replacement of countless CPU-based systems with far fewer, more powerful GPUs and DPUs can be expected to greatly reduce energy and space consumption for many organizations. However, the optimal energy and space efficiency of accelerated computing can only be realized when it includes a complementary data management platform. Today this platform requires a parallel storage architecture like DDN EXAScaler that can process data for large, highly complex AI models, fully unleash the power of GPU-based systems and provide innovative storage-driven performance and space-saving advantages across the AI stack.

Embark on a journey to unparalleled efficiency in your GPU deployments and discover which leading GPU Cloud vendors trust DDN to supercharge their operations. If you’re ready to unlock the full potential of your AI and computing infrastructure with the industry’s most advanced storage solutions, it’s time to take action. Connect with us today, and let’s explore how DDN can transform your technological landscape, propelling you into a future of unmatched performance and efficiency.

Last Updated

Oct 2, 2024 2:59 AM