Blog Posts Business Management

Computational Storage: Pushing the Frontiers of Big Data

Blog: NASSCOM Official Blog

Analysts expect the global big data market to grow from $138.9 billion in 2020 to $229.4 billion by 2025, at a CAGR of 10.6%.  As the volume of data being generated continues to grow at an astonishing rate, managing this snowballing data is becoming extremely difficult. This is especially true for IoT applications where data must be analyzed, and insights must be generated as quickly as possible.

This is where computational storage comes into the picture. It brings the power of high-performance compute to traditional storage systems, enabling organizations to process and analyze data as and when it is generated to extract valuable insights close to the source of the data and in real time.

The need for computational storage

With today’s data sets growing in size and complexity, traditional big data and advanced analytics techniques are feeling the heat. Computational storage enables data to be processed at the storage level, thereby reducing the time taken for insights to emerge. This also promises to reduce the amount of data moving from storage to compute. It facilitates real-time data analysis, reduces processing bottlenecks, and improves the time and speed with which data is processed.

In contrast to traditional storage models where data is constantly moved between storage and compute resources – resulting in high energy consumption and degraded performance of big data applications – computational storage brings processing capability close to where data is stored. This overcomes the time and cost involved in moving millions of gigabytes of information around, paving the way for more efficient, accurate, and timely in-situ processing.

Computational storage helps:

Its significance in the big data age

In the age of big data applications, the demand for sophisticated data processing capabilities is increasing dramatically. As computational storage helps in minimizing the time taken to fetch data from storage devices, it helps in processing data as quickly and efficiently as possible for quick results. Its significance in the big data age is profound:

  1. Helps pre-process big data: Big data brings with it big challenges; from capturing the growing volume of data from IoT and other devices to storing it, processing it, and unearthing insights – all in a matter of a few seconds. That’s where computational storage hits a home run. By bringing one or more multi-core processors near storage, it helps perform many of the pre-processing tasks such as indexing data, cleansing it, and supporting it for sophisticated big data programs.
  2. Analyzes data in real-time: Most smart applications like wearable health monitors and connected cars need to be able to analyze data in real time; any latency can be the cause of considerable harm. Computational storage helps store and analyze data in real time, allowing these devices to deliver outcomes almost instantly.
  3. Removes the storage-to-compute bottleneck: With traditional storage applications, there is always an issue of a mismatch between the storage capacity and the amount of memory needed for analysis. This means stored data must be moved in phases from one location to another for analysis. Computational storage offers the ability to store and process data simultaneously – without requiring big data to be exported from the storage device to the CPU for analysis.
  4. Improves application performance: Conventional storage architectures consume a considerable amount of time and resources just to move data from one system to the other. Computational storage helps eliminate this movement, resulting in lower latencies and better application performance. By bringing some of the compute operations directly to where data is stored and carrying out parallel processing, it helps in faster and more efficient processing of big data.
  5. Minimizes the strain on processors and networks: In traditional storage-compute models, data must constantly move from storage to memory as new data becomes available, which puts immense strain on the processor. Computational storage, on the other hand, performs analysis tasks in-situ, minimizing the impact on network bandwidth and compute resources and freeing them up for other, more critical loads.

Enabling real-time analysis in the big data world has become a key necessity for improving the performance of connected applications. Traditional storage systems face several challenges across latency, bandwidth, and efficiency, requiring organizations to adopt a concept that overcomes all these issues with ease. Computational storage brings compute resources close to where data is stored, helping pre-process big data quickly and efficiently.

Talk to our storage experts at SNIA SDC India 2020

The post Computational Storage: Pushing the Frontiers of Big Data appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples