The Internet of Analytics: 8 Predictions
Blog: The Tibco Blog
This year’s Forrester Big Data Streaming Analytics Wave for 2016, co-authored by Mike Gualtieri and Rowan Curran, is an important report that analyzes the streaming analytics market, which is projected to grow over 40% in coming years. Rowan blogged that streaming analytics will transform the Internet of Things into the Internet of Analytics. For sure, the Internet of Analytics will change the field of analytics and cognitive computing—but how?
In a recent IBM ad for cognitive computing, Serena Williams receives tennis coaching from an unmanned computer. This advocacy about the future of is not only wrong, it’s dangerous. Streaming analytics isn’t going to replace humans, it will augment their intelligence and increase efficacy.
IBM is wrong: Computers aren’t going to replace humans; it’s going to enhance their ability to…
Click To Tweet
First of all, what is streaming analytics? Forrester defines it this way: Streaming Analytics is…] “Software that can filter, aggregate, enrich, and analyze a high throughput of data from multiple, disparate live data sources and in any data format to identify simple and complex patterns to provide applications with context to detect opportune situations, automate immediate actions, and dynamically adapt.”
Put more plainly, streaming analytics is about putting algorithms into streams of data for the purpose of automation and improved human insight.
We’ve already seen the dramatic adoption of streaming analytics in one area: capital markets. In the past 15 years, algorithmic trading has grown from 3% to more than 70%, and many of those systems use streaming analytics in some form. What did we learn? We learned that automation threatens people. We learned that adopting automation requires vision. And we learned that businesses can leap-frog each other by getting automation right.
It took Wall Street 15 years to go from 3% to over 70% adoption. So what’s gonna give as analytics…
Click To Tweet
Not all of what happened on Wall Street will happen on Main Street, but we can learn from what happened. Here are 8 predictions about what’s gonna give as Main Street adopts the Internet of Analytics:
1. CIOs will adopt streaming analytics to capitalize on their big data investments. According to Gartner Group, more than 60% of big data projects fail to go beyond the pilot phase. Why? A CIO I met with last week said it well: “I have every analytics product ever made—I have enough insights—I need to act on those insights.” Streaming analytics automate decisions and augments human reactions based on historical, predictive analytics. For example, Oil & Gas CIOs are increasingly using analytics to discover what patterns in their data lakes of IoT sensor readings from production equipment and artificial lift systems—visual analytics helps them find patterns in sensor data that predict equipment failure, which can cause millions in revenue loss from production stoppage. By making those analytics streaming, operating equipment can sense and respond in real-time, before the production is interrupted. Similarly, retail CIOs use analytics to predict which customers will respond to a catalog of offers; by making those analytics streaming, sales associates and systems can make an offer when a customer walks into a store, visits the website, or opens their mobile app. Manufacturing and utilities use analytics to understand network outages and machine shutdowns; by making those analytics streaming, they can stop outages before it’s too late. These Main Street applications will propel streaming analytics to be as important as traditional analytics.
2. Streaming analytics will become a fundamental topic in computer science. Forrester’s Streaming Analytics Wave defines a set of computer science criteria to define streaming analytics: time windowing, aggregation, correlation, and integration with interactive analytics. These fundamentals are not well understand by the computer science community, are not yet taught in school, and are therefore not yet well known. This will change dramatically as IoT continues to drive the need for serious computer science about information in motion.
3. Data streams will be as important as data lakes. Data lakes contain data at rest; data streams contain data in motion. But most IT applications today are designed around data at rest. In the coming decade, data streams will become as important as data at rest. Data lakes help a utility company understand when and why outages happen; data streams help stop the outage before its too late. Data lakes help CMOs predict which customers will respond to an offer; data streams help them make the offer in real-time. Data lakes help oil and gas companies learn why production interruptions occur; data streams help avoid the interruptions. As IoT continues to create more and more data streams, the importance of data in motion will rise.
4. Steaming analytics will be dominated by enterprise software, not open source. Dan Woods wrote in Forbes that in the next phase of the big data market, products with large proprietary layers, more than pure open source, will provide the path forward. That is, the big rush will be in platforms that provide fast and easy access to the value in Big Data, rather than pure open source, developer-centric business models.
5. New CIOs will rise by leading the adoption of streaming analytics. In 2012, one firm, Knight Capital, lost $460 million in under 40 minutes because a market-making algorithm had an error and started, essentially, trading against itself. Heads rolled and just 4 months later, Knight Capital was acquired by Getco. Throughout Wall Street, budget immediately shifted to systems of streaming insight: trade risk, real-time risk management, real-time surveillance, and the technologist that were leading their adoption.
6. Streaming analytics and traditional analytics will become increasingly intertwined. In order to apply analytics to streams, you need to know what to look for. Traditional analytics help you look through the rearview mirror at the past, and predict important conditions. Streaming analytics are about looking forward, through your windshield, looking at real-time conditions, and acting. From a technology perspective, this means traditional analytics and streaming analytics will become increasingly integrated. For example, a data scientist must be able to test and deploy analytics against history, then inject the models they discover into the stream with a push of a button.
7. Business solution developers will rise in importance—a lot. The term “Business Solution Developer” came to me from John Rymer at Forrester. Business Solution Developers are a new breed of developers whose primary motivation is to create effective business solutions, NOT to develop elegant software code. They care about solving business problems, not honing their technical chops. These are developers that use Force.com, ServiceNow, and are beginning to rely on visual streaming analytics tools as well. Business Solution Developers have great technical skills, such as the ability to deliver applications, understand abstract logic, and complex data structures. But they are also passionate about business solutions. Business Solution Developers need serious developer tools, but they also need higher level tools beyond the low-level compiler. They need open source solution accelerators and don’t have the time to build everything from scratch.
8. Automation will enhance human insight, not replace it. When Knight Capital failed, a flurry of investment happened on Wall Street on in-memory data warehousing techniques. This was driven by the need to better balance the control that humans have over systems that can make billions of decisions a day. Forrester analyst Boris Evelson calls these “Systems of Operational Insight.” Wall Street calls them real-time risk management, real-time surveillance, and real-time trading operations. This technology is still largely undiscovered today, but it provides operational users a live view of streaming analytics in real-time. For example, drilling operations can explore streaming signals of production outages, marketing operations can analyze real-time sentiment and inventory and tweak promotions on Black Friday, and banks can investigate and stop fraud while its happening.
Although many will leap to the conclusion that machines will replace humans in the Internet of Analytics, they’re wrong; machines will make humans better.
Leave a Comment
You must be logged in to post a comment.
