Streaming Big Data Analytics – Are We There Yet?
Earlier this month I attended the Big Data Innovation Summit in Santa Clara that brought together over 600 data scientists, enterprise and data architects, business analysts, BI executives, customer experience professionals and C-level executives. It seemed like the perfect opportunity to poll the attendees to gauge their perceptions of real-time Big Data Analytics, and in particular, streaming Big Data Analytics. The three key questions we posed to the attendees were as follows:
- What does real-time Big Data Analytics really mean to you? And how important is streaming analytics?
- What tools are you considering for real-time analytics?
- What benefits do you hope to gain from streaming analytics?
Here’s what we found:
- Streaming Analytics is Essential for Real-time Analytics, Say 42%
We asked the attendees to weigh in on what real-time analytics meant to them. Not surprisingly, 56% indicated that real-time analytics still equates to either Hadoop-based analytics or some form of stored data analysis.This means that the query and response is in real-time or near real-time but the analysis is being done on historical data, on-demand. However, there is growing recognition of the need for streaming data analysis for true real-time analytics as was evident from 42% of the responses.
- Open-source and Home-grown Solutions are Popular but Firms Recognize the Limitations
While over one quarter of respondents said that they were considering using open-source or in-house solutions for real-time analytics, several attendees that had pursued this option admitted that this approach can limit advanced functionality and that the expense of maintaining custom code over time was a key area of concern. A survey participant from a top gaming company, for instance, said, “We built our own streaming analytics capabilities but it doesn’t scale. We are now looking for a commercially available offering to replace it.” And, with 25% of survey participants reporting that they planned to use stream processing and in-memory technologies for real-time analytics, it’s clear that there is a growing interest and understanding around the value of being able to glean real-time insight that can be acted upon immediately – in seconds and minutes.
- Real-time Customer Experience Monitoring Tops the Streaming Analytics Benefits List
About a third of respondents ranked real-time customer experience monitoring as the #1 benefit associated with streaming analytics. Not surprisingly, real-time fraud detection and infrastructure monitoring were also cited as important. Real-time 1:1 marketing trailed as a benefit in the survey, indicating that the awareness around the potential of building new revenue streams through personalized, real-time offers hasn’t yet hit the radar screen for many organizations.
I’d like to invite you to check out the survey infographic that summarizes our findings as well as the full research report. I hope you find the information useful.
Leave a Comment
You must be logged in to post a comment.
