IoT: Big Data in Disguise
Blog: Software AG Blog - Reality Check
In the Internet of Things (IoT) revolution, it appears there are some similarities between IoT projects and Big Data projects. But are the challenges the same?
In my last blog I discussed why we can call IoT a revolution and I mentioned that, as with nearly any revolution, there will be threats to existing business models and opportunities for new ones. And, as with any previous revolutions, before you can profit from it there will undoubtedly be some hurdles to jump. In this blog I would like to discuss some of the hurdles I came across while working with some of the early adopters across different industries.
The growth pains of IoT are very similar to the ones that developers face when they develop Big Data solutions. Yes, IoT projects and Big Data projects do have a lot in common. In Big Data the challenges are often discussed in terms of the V’s associated with it. Initially there were being three – Velocity, Volume and Variety – then quickly it became four, and after some debate it became as many as seven, depending who you spoke about it. Today I want to take a closer look at the Volume aspect. In my next blogs I will tune in to Velocity, Variety and Veracity.
The dazzling number of – well – numbers thrown at us never fails to awe. When Big Data was the topic du jour we were constantly bombarded with the amazing growth of numbers and the same hype is popular in presenting the importance of IoT: 50 trillion devices, 1000’s of petabytes of data. The figures can be as dazzling as headlights are for a deer.
My advice is not to stare too long into the light; knowing the size of the market doesn’t really help you in your day-to-day decisions for scoping your own project. Just don’t ignore the volume aspect and be ready to embrace it.
Simplicity is a virtue when you start designing an IoT system, and figuring out how much data your sensors will generate really makes sense. How many devices will be in your projects? How many sensors on average will it have (on many devices there can be more than one sensor)? These are important considerations in helping you to understand the influx of data. You will be amazed how much data some simple sensors will be able to generate per minute (often it is megabytes).
The next thing to consider is how much of your data will need to be transferred to a central storage area. How long will the data stay there? What needs to be done with it?
Also, it will make a big difference if you just keep sparse data – like the state of a Thing (error messages, is it on or off?), time series data (vibration readings, for example) or video (like infrared heat images). Especially if you collect the latter, your data volumes might grow rapidly.
I spoke earlier this year to a customer who initially thought that 800 Terrabytes was going to cover their needs for all their IoT projects in the upcoming years. After the first project they had reconsider as they already completely depleted the storage.
It goes without saying that IoT mimics a lot of the volume criteria previously associated only with Big Data, but IoT creates an even more challenging environment. IoT is for sure Big Data in disguise and maybe even Big Data on steroids if you look to the other V’s. More on that in my next blog.
The post IoT: Big Data in Disguise appeared first on Reality Check.
Leave a Comment
You must be logged in to post a comment.
