Blog Posts Process Management

To More Efficiently Process IoT Data, Move Your Wheelbarrow Closer to Where You Dig

Blog: The Tibco Blog

Some of the easiest, most powerful lessons we learn when we are young seem obvious—usually after we learn them.  My dad taught me one the first year he let me help him plant his garden when I was a kid. Early one spring, he gave me a shovel and a wheelbarrow and said, “Go dig a row of holes and dump all the dirt in the wheelbarrow.”

So, naturally, I placed the wheelbarrow at the far edge of the garden, put my shovel in the ground at the opposite end, walked the scooped dirt all the way over to the wheelbarrow, and then walked all the way back again to my row of holes.  

My dad watched me do this a few times (laughing under his breath I’m sure) until he finally said, “Hey Kevin, don’t you think you’d get the job done faster if you moved your wheelbarrow closer to your holes?”  Brilliant.

Lost opportunities due to latency in IoT processing 

Believe it or not, this simple lesson can also be applied to the processing of data from your IoT devices. TIBCO—an innovator in microservices tooling—had the opportunity to present this lesson at this year’s Microservices World event as part of a breakout session.

The modern version of the lesson goes like this: Typically, IoT devices are connected directly to an edge device within a local network. The IoT devices send data they collect through the edge devices and up to an IoT platform hosted within a private or public cloud, such as TIBCO Cloud.  This platform processes the data—collecting, filtering, and analyzing so that appropriate action is taken. So in network terms, the IoT devices are all close to each other. But, like my wheelbarrow and the garden holes I had dug, this topology puts quite a bit of distance between IoT devices and the IoT platform.  

The effect is a high amount of latency between the time data is collected by IoT devices and the time it is acted upon—which can translate into lost opportunities if data isn’t acted upon when it is of highest value.  

How to decrease latency in IoT processing

It would seem to make sense then, as my dad would suggest, to move the processing of your IoT data closer to your IoT devices—such as on your edge devices. However, edge devices have limited compute resources (i.e. memory), which in turn limits the size (and functionality) of applications that can be deployed to them when written in traditional languages such as Java.  So instead of a wheelbarrow, imagine my dad had given me a teacup. 

That’s where the open-sourced Project Flogo can help. It enables developers to visually create event-driven microservices and functions and, most importantly, it builds application executables that are up to 50x smaller than those written in Java—perfectly scaled-down for (teacup-sized) edge devices and new types of serverless environments such as Function-as-a-Services (FaaS).  Although small in size, these applications can embed automated decision making with contextual rules and/or machine learning right into the edge device. With Project Flogo, you can process your IoT data more quickly and cost-effectively on edge devices (which are much closer to the source) rather than the cloud. You can see how through a series of short videos.

TIBCO has commercialized Project Flogo within TIBCO Cloud Integration, our market-leading enterprise Integration Platform-as-a-Service (iPaaS).  TIBCO Cloud Integration not only includes commercial technical support and a wide variety of enterprise connectors, it also includes many productivity features that simplify and accelerate event-driven application development.  You can try TIBCO Cloud Integration for a limited time to create applications in a development environment within TIBCO Cloud at no charge.

When you have the right tools, and you move them close to where work is done, you can respond with speed to valuable digital moments.  And you can plant gardens with your dad too!

Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="https://www.businessprocessincubator.com/content/to-more-efficiently-process-iot-data-move-your-wheelbarrow-closer-to-where-you-dig/?feed=html" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples

BPMN.org

XPDL.org

×