It architects tasked with the design of Storage Systems for Artificial Intelligence (AI) Need to balance capacity, performance and cost.
AI Systems, Especially that Based on Large Language Models (LLMS), Consume Vast Amounts of Data. In Fact, LLMS or Generative AI (Genai) Models often Work Better The More Data They have. The training phase of ai in particular is very data hungry.
The infection phase of ai, however, needs high performance to avoid ai systems that feel unresponsive or fail to work at all. They need throughput and low latency.
So, a key question is, to what extent can we use a mix of on-love and cloud storageOn-Premise Storage Brings Higher Performance and Greater Security. Cloud storage offers the ability to scale, lower costs and potentially, better integration with cloud-based ai models and cloud data sources.
In this article, we look at the pros and cons of each and how best to Optimise them for Storage for Ai,
AI Storage: On-PreMise vs Cloud?
Enterprises Typically Look to on-Premise Storage for the Best Speed, Performance and Security-And AI Workloads are no exception. Local storage can also be easier to fin tune to the needs of ai models, and will likely suffer less from from network bottlenecks.
Then there are the advantages of keeping ai models close to source data. For Enterprise Applications, this is often a real database that runs on block storage.
As a result, systems designers need to consider the impact of ai on the performance of a system of record. The business will not want key packages such as ERP or CRM Slowed Down BeCause they also feed data into an ai system. There are also strong security, privacy and compliance reasons for keeping core data records on site raather than moving them to the cloud.
Even so, cloud storage also offers advantages for ai projects. Cloud storage is easy to scale, and customers only pay for what they use. For some ai use cases, source data will alredy be in the cloud, in a data lake or a cloud-based, Saas application, for example.
Cloud storage is largely based Around Object Storage, which is Well-Suited to the UnstruCred Data which makes up the Bulk of Information Consuined by Large Language Models.
At the same time, the growth of storage systems that can run object storage on-lovemise makes it Easier for Enterprises to have a Single Storage Layer-even a single a single global Namespace-to serve on-trust and cloudy Infrastructure, Including ai. This is essentially relevant for firms that expect to move workloads Between Local and Cloud Infrastructure, or Operate “Hybrid” Systems.
Ai Storage, and Cloud Options
Cloud storage is often the first choice for enterprises that Want to run ai proofs-of-concept (POCS). It removers the need for up-front capital investment and can be spun down at the end of the project.
In other cases, firms have designed ai systems to “Burst” from the datacentre to the cloudThis makes use of public cloud resources for compute and storage to cover peaks in demand. Bursting is most effective for ai projects with relatively short peak worklines, such as that there that that run on a seasonal business cycle.
But the Arrival of Generative Ai Based on Large Language Models Has Tipped The Balance More Towards Cloud Storage Simply because of the data volumes involved.
At the same time, cloud provides now offer a wider range dedicated data storage options focused on ai workloads. This include storage provision tailored to different stages of an ai workload, namely: prepare, train, service and archive.
As google's engineers put it: “Each stage in the ml [machine learning] Lifecycle has different storage requirements. For example, when you upload the training dataset, you might prioritise storage capacity for training and high throughput for large datsets. Similarly, the training, tuning, serving and archiving stages have different requirements ”
Although this is written for Google Cloud Platform, The Same Principles Apply to Microsoft Azure and Amazon Web Services. All three hyperscals, plus vendors such as ibm and oracle, offer cloud-based storage suitable for the bulk storage requires of ai. For the most part, unstructured data used by AI, Including source Material and Training Data, Will Likely Be Held in Object Storage.
This could be AWS S3, Azure Blob Storage, or Google Cloud's Cloud StorageIn addition, Third-Parthy Software Platforms, Such as Netapp's on also also available from the hyperscales, and can improve data portability betward cloud and only.
For the production, or infection stage, of AI operations, the choices are often even more complex. It Architects Can Specify Nvme and SSD Storage With Different Performance Tiers for Critical Parts of the Ai Workflow. Older “Spinning disk” storage remains on offer for tasks such as initial data engage and preparation, or for Archiving AI System Outputs.
This type of storage is also application Neutral: It Architects Can Specify their Performance Parameters and Budget for Ai as They can for any other workload. But a new generation of cloud storage is designed from the ground up for ai.
Advanced Cloud Storage for Ai
The Specific Demands of Ai Has Prompted Storage vendors to design dedicated infrastructure to avoid bottlenecks in ai workflows, some of which are found in on-love in the cloud. Key Among them are two approaches: parallelism and direct gpu memory access.
Parallelism allows storage systems to handle What Storage Supplier Cloudian descriptions As “The Concurrent Data Requests Characteristic of Ai and Ml Workloads”. This makes model training and infection faster. In this way, AI Storage Systems are enabled to handle multiple data streams in parallel.
An example here is google's parallelstore, which launched last year To provide a managed parallel file storage service aimed at intensive input/output for artificial intelligence applications.
GPU Access to Memory, Meanwhile, Sets Out to Remove Bottlenecks Between Storage Cache and GPUS – GPUS are expensive and can be scarce. According to John Woolley, Chief Commercial Officer at Vendor Insurgo Media, Storage must Deliver at Least 10gbps of Sustained Throughput to Prevent “GPU Starvation”.
Protocols such as gpudirect – developed by nvidia – Allow GPUS to Access Nvme Drive Memory Directly, Similarly to the way the way RDMA Access Direct Access Betwen Systems with Involving CPUOT OS. It also goes by the name DGS or Direct GPU Support (DGS).
Local cache layers between the gpu and shared storage can use block storage on nvme ssds to provide “Bandwidth Saturation” to Each GPU, at 60GBPS or more. As a result, cloud suppliers plan a new generation of ssd, optimized for dgs and likely to be based on Slc nand.
“Inference Workloads Require a Combination of Traditional Enterprise Bulk Storage and AI-Optimized DGS Storage,” Says sebastien jean, cto at phison us, a nand manufacturer. “The new gpu-concentric workload requires small i/o access and very low latency.”
As a result, the market is likely to see more ai-optimized storage systems, include with Nvidia DGX BASEPOD and Superpod Certification, and AI Integration.
Options include Nutanix Enterprise Ai, Pure's Evergreen One For Ai, Dell Powerscale, Vast's Vast Data Platform, Weka, A Cloud Hybrid Nas Provider, and offerings from Offerings from HPE, Hitachi Vantara, IBM and Netapp.