Blog Posts Process Management

How to Apply Machine Learning and Big Data Analytics to Real-Time Processing

Blog: The Tibco Blog

In March 2016, I had a talk at Voxxed Zurich about “How to Apply Machine Learning and Big Data Analytics to Real Time Processing”.


Finding Insights with TIBCO Spotfire and TERR

Big data is currently a big hype. Large amounts of historical data are stored in Hadoop or other platforms. Data discovery tools and statistical computing are used to draw new knowledge and to find patterns from this data for promotions, cross-selling, fraud detection, or predictive maintenance. TIBCO Spotfire can be used by business users for visualization and data discovery to find these insights.

In addition, a data scientist can support business users by building analytic models with tools and frameworks such as R/TERR, Apache Spark MLlib or H20. Machine learning helps create and train powerful algorithms, which can improve business processes and add business value. These analytic models can then be embedded into TIBCO Spotfire, so that the business user can leverage them without knowing the detailed logic behind an algorithm.

The key challenge is how these insights can then be integrated from historical data to new transactions in real time to make customers happy, increase revenue, prevent fraud, or replace machines before they break.

Putting Analytic Models into Action with TIBCO StreamBase and Live Datamart

Fast Data via stream processing is the solution to embed patterns—which were obtained from analyzing historical data—into future transactions in real time. The following slide deck uses several real world success stories to explain the concepts behind stream processing, respectively streaming analytics, and its relation to Apache Hadoop and other big data platforms. I discuss how patterns and statistical models of R, H20, Spark MLlib, and other technologies (using PMML standard) can be integrated into real-time processing using TIBCO StreamBase. TIBCO Live Datamart is added on top of streaming analytics to enable a live view of the data, as well as proactive human interactions.

Here is the slide deck, which explains how to realize a closed loop from big data analytics across machine learning and analytic models to real-time actions with streaming analytics:

How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real Time Streaming Analytics from Kai Wähner

Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples