Love it or Hate It: Auto ML is Here to Stay
Blog: The Tibco Blog
In the data and analytics world, trends come and go. A few years ago, the term “big data” was all the rave.Now, it is simply an implicit capability that many organizations possess. More recently, however, I’ve been in many meetings where terms like artificial intelligence (AI) and Auto ML (automated machine learning) have been making their way into the buzzword bingo cards.
For this blog, we’ll put AI aside for the moment and chat about Auto ML. Conceptually, Auto ML is an umbrella term used to describe a set of processes that are well, automatic and require little to no code. Essentially, these processes may involve automatic:
- Data prep and cleansing routines
- Creation of new features to be used in machine learning models
- Selection of which parameters are to be used in the model
- Model identification and selection
- Hyperparameter tuning
This automation is packed with promises. Some vendors claim that the intention of this technology is for it to be used by citizen data scientists (or non-experts) to address the skills shortage in the industry and that in some cases data scientists will no longer be needed. Quite simply, for a majority of use cases, it would extremely risky to deploy models without vetting and validation so we would not recommend this approach.
For organizations new to Auto ML, the thought of deploying algorithms into mission critical business systems without proper vetting and testing should throw up a red flag. In order for ML to work, data scientists are a crucial factor. There is still a use for Auto ML in many organizations, but it will not displace or replace our coveted data science unicorns.
Auto ML will augment the work of data scientists, not replace it Recently, I had the good fortune to chat with a group of experts on the topic. To sum up the conversations, we concluded that Auto ML is most useful to augment the work of data scientists and citizen data scientists. Auto ML will be most effective when organizations use it to quickly identify which areas or projects might be most valuable for further exploration by a data scientist. It can also be used as an aid to data scientists to increase their productivity. Auto ML may also help to increase the accuracy of their final solutions, by helping the data scientist quickly consider a wider range of analytic approaches.
But buyer beware, when looking at Auto ML, consider these questions:
- Is the Auto ML transparent? That is, can you explain why the model makes a particular recommendation? This is critical for many applications and is imperative for many regulations.
- Is the Auto ML extensible and flexible? Can you customize and extend the pipeline generated to suit your specific needs?
- What is the workflow and process to vet and deploy the models generated?
- After you deploy the machine learning pipelines, how will you monitor, manage, refresh, and govern the deployment?
To find out more about AutoML and how it can benefit your business, watch this YouTube tutorial video.
Next up, artificial intelligence. Stay tuned for my next blog as we discuss the growing importance of “thinking” computer systems.
Leave a Comment
You must be logged in to post a comment.
